-- Logs begin at Thu 2022-07-21 07:47:45 UTC, end at Thu 2022-07-21 08:55:03 UTC. -- Jul 21 08:15:36.058933 dv-cinder-850320-1-iscsi systemd[1]: Starting Devstack devstack@keystone.service... Jul 21 08:15:36.068484 dv-cinder-850320-1-iscsi devstack@keystone.service[80235]: [uWSGI] getting INI configuration from /etc/keystone/keystone-uwsgi-public.ini Jul 21 08:15:36.070468 dv-cinder-850320-1-iscsi devstack@keystone.service[80235]: *** Starting uWSGI 2.0.18-debian (64bit) on [Thu Jul 21 08:15:36 2022] *** Jul 21 08:15:36.070468 dv-cinder-850320-1-iscsi devstack@keystone.service[80235]: compiled with version: 10.0.1 20200405 (experimental) [master revision 0be9efad938:fcb98e4978a:705510a708d3642c9c962beb663c476167e4e8a4] on 11 April 2020 11:15:55 Jul 21 08:15:36.070468 dv-cinder-850320-1-iscsi devstack@keystone.service[80235]: os: Linux-5.4.0-72-generic #80-Ubuntu SMP Mon Apr 12 17:35:00 UTC 2021 Jul 21 08:15:36.070468 dv-cinder-850320-1-iscsi devstack@keystone.service[80235]: nodename: dv-cinder-850320-1-iscsi Jul 21 08:15:36.070468 dv-cinder-850320-1-iscsi devstack@keystone.service[80235]: machine: x86_64 Jul 21 08:15:36.070468 dv-cinder-850320-1-iscsi devstack@keystone.service[80235]: clock source: unix Jul 21 08:15:36.070468 dv-cinder-850320-1-iscsi devstack@keystone.service[80235]: pcre jit disabled Jul 21 08:15:36.070468 dv-cinder-850320-1-iscsi devstack@keystone.service[80235]: detected number of CPU cores: 4 Jul 21 08:15:36.070468 dv-cinder-850320-1-iscsi devstack@keystone.service[80235]: current working directory: / Jul 21 08:15:36.070468 dv-cinder-850320-1-iscsi devstack@keystone.service[80235]: detected binary path: /usr/bin/uwsgi-core Jul 21 08:15:36.070468 dv-cinder-850320-1-iscsi devstack@keystone.service[80235]: your processes number limit is 47680 Jul 21 08:15:36.071039 dv-cinder-850320-1-iscsi devstack@keystone.service[80235]: your memory page size is 4096 bytes Jul 21 08:15:36.071039 dv-cinder-850320-1-iscsi devstack@keystone.service[80235]: detected max file descriptor number: 2048 Jul 21 08:15:36.071039 dv-cinder-850320-1-iscsi devstack@keystone.service[80235]: lock engine: pthread robust mutexes Jul 21 08:15:36.071039 dv-cinder-850320-1-iscsi devstack@keystone.service[80235]: thunder lock: enabled Jul 21 08:15:36.071039 dv-cinder-850320-1-iscsi devstack@keystone.service[80235]: uwsgi socket 0 bound to UNIX address /var/run/uwsgi/keystone-wsgi-public.socket fd 4 Jul 21 08:15:36.071039 dv-cinder-850320-1-iscsi devstack@keystone.service[80235]: Python version: 3.8.10 (default, Jun 22 2022, 20:18:18) [GCC 9.4.0] Jul 21 08:15:36.114687 dv-cinder-850320-1-iscsi devstack@keystone.service[80235]: Python main interpreter initialized at 0x55e05a35d9c0 Jul 21 08:15:36.114687 dv-cinder-850320-1-iscsi devstack@keystone.service[80235]: python threads support enabled Jul 21 08:15:36.114687 dv-cinder-850320-1-iscsi devstack@keystone.service[80235]: your server socket listen backlog is limited to 100 connections Jul 21 08:15:36.114687 dv-cinder-850320-1-iscsi devstack@keystone.service[80235]: your mercy for graceful operations on workers is 90 seconds Jul 21 08:15:36.115602 dv-cinder-850320-1-iscsi devstack@keystone.service[80235]: mapped 403077 bytes (393 KB) for 2 cores Jul 21 08:15:36.115669 dv-cinder-850320-1-iscsi devstack@keystone.service[80235]: *** Operational MODE: preforking *** Jul 21 08:15:36.115947 dv-cinder-850320-1-iscsi devstack@keystone.service[80235]: *** uWSGI is running in multiple interpreter mode *** Jul 21 08:15:36.115947 dv-cinder-850320-1-iscsi devstack@keystone.service[80235]: spawned uWSGI master process (pid: 80235) Jul 21 08:15:36.116141 dv-cinder-850320-1-iscsi systemd[1]: Started Devstack devstack@keystone.service. Jul 21 08:15:36.116836 dv-cinder-850320-1-iscsi devstack@keystone.service[80235]: spawned uWSGI worker 1 (pid: 80244, cores: 1) Jul 21 08:15:36.117623 dv-cinder-850320-1-iscsi devstack@keystone.service[80235]: spawned uWSGI worker 2 (pid: 80245, cores: 1) Jul 21 08:15:36.117623 dv-cinder-850320-1-iscsi devstack@keystone.service[80235]: running "unix_signal:15 gracefully_kill_them_all" (master-start)... Jul 21 08:15:41.610875 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Jul 21 08:15:41.610875 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: from cryptography.utils import int_from_bytes Jul 21 08:15:41.611663 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Jul 21 08:15:41.611663 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: from cryptography.utils import int_from_bytes Jul 21 08:15:41.612052 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Jul 21 08:15:41.612052 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: from cryptography.utils import int_from_bytes Jul 21 08:15:41.612159 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Jul 21 08:15:41.612159 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: from cryptography.utils import int_from_bytes Jul 21 08:15:41.661292 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=80244) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} Jul 21 08:15:41.661476 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=80245) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} Jul 21 08:15:41.661556 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG uwsgi [-] Configuration options gathered from: {{(pid=80244) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} Jul 21 08:15:41.662209 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG uwsgi [-] command line args: [] {{(pid=80244) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} Jul 21 08:15:41.662311 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG uwsgi [-] Configuration options gathered from: {{(pid=80245) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} Jul 21 08:15:41.662446 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG uwsgi [-] command line args: [] {{(pid=80245) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} Jul 21 08:15:41.662536 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG uwsgi [-] config files: ['/etc/keystone/keystone.conf'] {{(pid=80244) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} Jul 21 08:15:41.662665 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG uwsgi [-] ================================================================================ {{(pid=80244) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} Jul 21 08:15:41.662740 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG uwsgi [-] config files: ['/etc/keystone/keystone.conf'] {{(pid=80245) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} Jul 21 08:15:41.662918 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG uwsgi [-] ================================================================================ {{(pid=80245) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} Jul 21 08:15:41.662994 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG uwsgi [-] admin_token = **** {{(pid=80244) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 21 08:15:41.663159 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG uwsgi [-] config_dir = [] {{(pid=80244) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 21 08:15:41.663242 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG uwsgi [-] admin_token = **** {{(pid=80245) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 21 08:15:41.663365 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG uwsgi [-] config_file = ['/etc/keystone/keystone.conf'] {{(pid=80244) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 21 08:15:41.663455 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG uwsgi [-] config_dir = [] {{(pid=80245) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 21 08:15:41.663554 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG uwsgi [-] config_source = [] {{(pid=80244) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 21 08:15:41.663631 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG uwsgi [-] config_file = ['/etc/keystone/keystone.conf'] {{(pid=80245) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 21 08:15:41.663735 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG uwsgi [-] debug = True {{(pid=80244) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 21 08:15:41.663821 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG uwsgi [-] config_source = [] {{(pid=80245) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 21 08:15:41.663935 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG uwsgi [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'dogpile=INFO', 'routes=INFO'] {{(pid=80244) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 21 08:15:41.664236 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG uwsgi [-] debug = True {{(pid=80245) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 21 08:15:41.664236 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG uwsgi [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'dogpile=INFO', 'routes=INFO'] {{(pid=80245) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 21 08:15:41.664549 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG uwsgi [-] default_publisher_id = None {{(pid=80244) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 21 08:15:41.664654 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG uwsgi [-] default_publisher_id = None {{(pid=80245) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 21 08:15:41.664759 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG uwsgi [-] insecure_debug = False {{(pid=80244) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 21 08:15:41.664852 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG uwsgi [-] insecure_debug = False {{(pid=80245) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 21 08:15:41.664945 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG uwsgi [-] instance_format = [instance: %(uuid)s] {{(pid=80244) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 21 08:15:41.665048 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG uwsgi [-] instance_format = [instance: %(uuid)s] {{(pid=80245) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 21 08:15:41.665150 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG uwsgi [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=80244) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 21 08:15:41.665251 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG uwsgi [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=80245) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 21 08:15:41.665350 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG uwsgi [-] list_limit = None {{(pid=80244) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 21 08:15:41.665449 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG uwsgi [-] list_limit = None {{(pid=80245) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 21 08:15:41.665548 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG uwsgi [-] log_config_append = None {{(pid=80244) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 21 08:15:41.665652 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG uwsgi [-] log_config_append = None {{(pid=80245) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 21 08:15:41.665760 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG uwsgi [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=80244) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 21 08:15:41.665854 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG uwsgi [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=80245) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 21 08:15:41.665854 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG uwsgi [-] log_dir = None {{(pid=80245) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 21 08:15:41.666034 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG uwsgi [-] log_dir = None {{(pid=80244) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 21 08:15:41.666118 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG uwsgi [-] log_file = None {{(pid=80245) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 21 08:15:41.666184 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG uwsgi [-] log_file = None {{(pid=80244) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 21 08:15:41.666294 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG uwsgi [-] log_rotate_interval = 1 {{(pid=80245) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 21 08:15:41.666407 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG uwsgi [-] log_rotate_interval = 1 {{(pid=80244) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 21 08:15:41.666518 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG uwsgi [-] log_rotate_interval_type = days {{(pid=80245) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 21 08:15:41.666630 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG uwsgi [-] log_rotate_interval_type = days {{(pid=80244) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 21 08:15:41.666710 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG uwsgi [-] log_rotation_type = none {{(pid=80245) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 21 08:15:41.666834 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG uwsgi [-] log_rotation_type = none {{(pid=80244) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 21 08:15:41.666910 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG uwsgi [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=80245) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 21 08:15:41.666990 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG uwsgi [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=80244) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 21 08:15:41.667105 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG uwsgi [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=80245) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 21 08:15:41.667225 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG uwsgi [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=80244) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 21 08:15:41.667312 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG uwsgi [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=80245) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 21 08:15:41.667429 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG uwsgi [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=80244) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 21 08:15:41.667508 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG uwsgi [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=80245) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 21 08:15:41.667628 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG uwsgi [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=80244) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 21 08:15:41.667703 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG uwsgi [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(system_scope)s %(user_domain)s %(project_domain)s {{(pid=80245) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 21 08:15:41.667816 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG uwsgi [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(system_scope)s %(user_domain)s %(project_domain)s {{(pid=80244) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 21 08:15:41.667898 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG uwsgi [-] max_logfile_count = 30 {{(pid=80245) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 21 08:15:41.667976 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG uwsgi [-] max_logfile_count = 30 {{(pid=80244) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 21 08:15:41.668089 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG uwsgi [-] max_logfile_size_mb = 200 {{(pid=80245) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 21 08:15:41.668163 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG uwsgi [-] max_logfile_size_mb = 200 {{(pid=80244) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 21 08:15:41.668317 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG uwsgi [-] max_param_size = 64 {{(pid=80245) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 21 08:15:41.668420 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG uwsgi [-] max_param_size = 64 {{(pid=80244) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 21 08:15:41.668552 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG uwsgi [-] max_project_tree_depth = 5 {{(pid=80245) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 21 08:15:41.668666 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG uwsgi [-] max_project_tree_depth = 5 {{(pid=80244) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 21 08:15:41.668800 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG uwsgi [-] max_token_size = 16384 {{(pid=80245) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 21 08:15:41.668937 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG uwsgi [-] max_token_size = 16384 {{(pid=80244) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 21 08:15:41.669048 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG uwsgi [-] notification_format = cadf {{(pid=80245) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 21 08:15:41.669198 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG uwsgi [-] notification_format = cadf {{(pid=80244) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 21 08:15:41.669326 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG uwsgi [-] notification_opt_out = ['identity.authenticate.success', 'identity.authenticate.pending', 'identity.authenticate.failed'] {{(pid=80245) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 21 08:15:41.669498 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG uwsgi [-] notification_opt_out = ['identity.authenticate.success', 'identity.authenticate.pending', 'identity.authenticate.failed'] {{(pid=80244) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 21 08:15:41.669808 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG uwsgi [-] public_endpoint = http://10.222.0.47/identity {{(pid=80245) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 21 08:15:41.669995 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG uwsgi [-] publish_errors = False {{(pid=80245) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 21 08:15:41.670110 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG uwsgi [-] public_endpoint = http://10.222.0.47/identity {{(pid=80244) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 21 08:15:41.670207 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG uwsgi [-] publish_errors = False {{(pid=80244) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 21 08:15:41.670320 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG uwsgi [-] pydev_debug_host = None {{(pid=80245) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 21 08:15:41.670394 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG uwsgi [-] pydev_debug_host = None {{(pid=80244) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 21 08:15:41.670516 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG uwsgi [-] pydev_debug_port = None {{(pid=80245) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 21 08:15:41.670618 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG uwsgi [-] rate_limit_burst = 0 {{(pid=80245) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 21 08:15:41.670698 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG uwsgi [-] pydev_debug_port = None {{(pid=80244) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 21 08:15:41.670817 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG uwsgi [-] rate_limit_burst = 0 {{(pid=80244) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 21 08:15:41.670890 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG uwsgi [-] rate_limit_except_level = CRITICAL {{(pid=80245) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 21 08:15:41.671020 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG uwsgi [-] rate_limit_except_level = CRITICAL {{(pid=80244) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 21 08:15:41.671102 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG uwsgi [-] rate_limit_interval = 0 {{(pid=80245) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 21 08:15:41.671246 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG uwsgi [-] rate_limit_interval = 0 {{(pid=80244) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 21 08:15:41.671339 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG uwsgi [-] standard_threads = False {{(pid=80245) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 21 08:15:41.671503 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG uwsgi [-] standard_threads = False {{(pid=80244) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 21 08:15:41.671590 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG uwsgi [-] strict_password_check = False {{(pid=80245) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 21 08:15:41.671726 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG uwsgi [-] strict_password_check = False {{(pid=80244) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 21 08:15:41.671799 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG uwsgi [-] syslog_log_facility = LOG_USER {{(pid=80245) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 21 08:15:41.671956 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG uwsgi [-] use_eventlog = False {{(pid=80245) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 21 08:15:41.672032 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG uwsgi [-] syslog_log_facility = LOG_USER {{(pid=80244) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 21 08:15:41.672131 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG uwsgi [-] use_eventlog = False {{(pid=80244) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 21 08:15:41.672211 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG uwsgi [-] use_journal = False {{(pid=80245) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 21 08:15:41.672293 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG uwsgi [-] use_json = False {{(pid=80245) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 21 08:15:41.672362 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG uwsgi [-] use_journal = False {{(pid=80244) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 21 08:15:41.672469 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG uwsgi [-] use_stderr = False {{(pid=80245) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 21 08:15:41.672545 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG uwsgi [-] use_json = False {{(pid=80244) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 21 08:15:41.672640 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG uwsgi [-] use_stderr = False {{(pid=80244) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 21 08:15:41.672715 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG uwsgi [-] use_syslog = False {{(pid=80245) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 21 08:15:41.672818 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG uwsgi [-] use_syslog = False {{(pid=80244) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 21 08:15:41.672895 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG uwsgi [-] watch_log_file = False {{(pid=80245) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 21 08:15:41.673050 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG uwsgi [-] watch_log_file = False {{(pid=80244) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 21 08:15:41.673126 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG uwsgi [-] oslo_concurrency.disable_process_locking = False {{(pid=80245) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 08:15:41.673306 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG uwsgi [-] oslo_concurrency.disable_process_locking = False {{(pid=80244) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 08:15:41.673390 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG uwsgi [-] oslo_concurrency.lock_path = None {{(pid=80245) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 08:15:41.673545 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG uwsgi [-] oslo_concurrency.lock_path = None {{(pid=80244) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 08:15:41.673626 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG uwsgi [-] profiler.connection_string = messaging:// {{(pid=80245) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 08:15:41.673802 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG uwsgi [-] profiler.connection_string = messaging:// {{(pid=80244) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 08:15:41.673878 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG uwsgi [-] profiler.enabled = False {{(pid=80245) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 08:15:41.674042 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG uwsgi [-] profiler.enabled = False {{(pid=80244) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 08:15:41.674128 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG uwsgi [-] profiler.es_doc_type = notification {{(pid=80245) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 08:15:41.674281 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG uwsgi [-] profiler.es_scroll_size = 10000 {{(pid=80245) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 08:15:41.674356 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG uwsgi [-] profiler.es_doc_type = notification {{(pid=80244) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 08:15:41.674562 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG uwsgi [-] profiler.es_scroll_size = 10000 {{(pid=80244) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 08:15:41.674658 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG uwsgi [-] profiler.es_scroll_time = 2m {{(pid=80245) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 08:15:41.674758 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG uwsgi [-] profiler.filter_error_trace = False {{(pid=80245) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 08:15:41.674844 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG uwsgi [-] profiler.es_scroll_time = 2m {{(pid=80244) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 08:15:41.674990 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG uwsgi [-] profiler.filter_error_trace = False {{(pid=80244) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 08:15:41.675065 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG uwsgi [-] profiler.hmac_keys = SECRET_KEY {{(pid=80245) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 08:15:41.675238 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG uwsgi [-] profiler.hmac_keys = SECRET_KEY {{(pid=80244) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 08:15:41.675313 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG uwsgi [-] profiler.sentinel_service_name = mymaster {{(pid=80245) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 08:15:41.675508 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG uwsgi [-] profiler.sentinel_service_name = mymaster {{(pid=80244) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 08:15:41.675600 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG uwsgi [-] profiler.socket_timeout = 0.1 {{(pid=80245) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 08:15:41.675744 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG uwsgi [-] profiler.trace_sqlalchemy = False {{(pid=80245) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 08:15:41.675818 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG uwsgi [-] profiler.socket_timeout = 0.1 {{(pid=80244) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 08:15:41.675979 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG uwsgi [-] profiler.trace_sqlalchemy = False {{(pid=80244) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 08:15:41.676059 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=80245) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 08:15:41.676224 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=80245) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 08:15:41.676314 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=80244) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 08:15:41.676483 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=80244) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 08:15:41.676601 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_process_name = {{(pid=80245) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 08:15:41.676709 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_process_name = {{(pid=80244) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 08:15:41.676790 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=80245) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 08:15:41.676946 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=80245) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 08:15:41.677110 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=80244) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 08:15:41.677194 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=80244) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 08:15:41.677272 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG uwsgi [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=80245) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 08:15:41.677459 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG uwsgi [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=80244) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 08:15:41.677543 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG uwsgi [-] audit.namespace = openstack {{(pid=80245) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 08:15:41.677704 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG uwsgi [-] oslo_policy.enforce_new_defaults = False {{(pid=80245) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 08:15:41.677789 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG uwsgi [-] audit.namespace = openstack {{(pid=80244) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 08:15:41.677944 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG uwsgi [-] oslo_policy.enforce_scope = False {{(pid=80245) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 08:15:41.678023 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG uwsgi [-] oslo_policy.enforce_new_defaults = False {{(pid=80244) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 08:15:41.678177 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG uwsgi [-] oslo_policy.enforce_scope = False {{(pid=80244) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 08:15:41.678253 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG uwsgi [-] oslo_policy.policy_default_rule = default {{(pid=80245) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 08:15:41.678432 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG uwsgi [-] oslo_policy.policy_default_rule = default {{(pid=80244) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 08:15:41.678518 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG uwsgi [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=80245) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 08:15:41.678689 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG uwsgi [-] oslo_policy.policy_file = policy.yaml {{(pid=80245) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 08:15:41.678766 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG uwsgi [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=80244) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 08:15:41.678947 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG uwsgi [-] oslo_policy.policy_file = policy.yaml {{(pid=80244) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 08:15:41.679070 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG uwsgi [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=80245) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 08:15:41.679176 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG uwsgi [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=80245) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 08:15:41.679263 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG uwsgi [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=80244) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 08:15:41.679433 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG uwsgi [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=80244) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 08:15:41.679530 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=80245) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 08:15:41.679669 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=80245) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 08:15:41.679768 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=80244) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 08:15:41.679896 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=80244) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 08:15:41.679983 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG uwsgi [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=80245) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 08:15:41.680132 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG uwsgi [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=80244) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 08:15:41.680211 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG uwsgi [-] keystone_authtoken.admin_password = **** {{(pid=80245) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 08:15:41.680382 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG uwsgi [-] keystone_authtoken.admin_password = **** {{(pid=80244) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 08:15:41.680465 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG uwsgi [-] keystone_authtoken.admin_tenant_name = admin {{(pid=80245) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 08:15:41.680635 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG uwsgi [-] keystone_authtoken.admin_tenant_name = admin {{(pid=80244) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 08:15:41.680724 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG uwsgi [-] keystone_authtoken.admin_token = **** {{(pid=80245) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 08:15:41.680865 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG uwsgi [-] keystone_authtoken.admin_token = **** {{(pid=80244) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 08:15:41.680950 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG uwsgi [-] keystone_authtoken.admin_user = None {{(pid=80245) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 08:15:41.681096 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG uwsgi [-] keystone_authtoken.admin_user = None {{(pid=80244) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 08:15:41.681174 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG uwsgi [-] keystone_authtoken.auth_admin_prefix = {{(pid=80245) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 08:15:41.681342 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG uwsgi [-] keystone_authtoken.auth_admin_prefix = {{(pid=80244) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 08:15:41.681420 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG uwsgi [-] keystone_authtoken.auth_host = 127.0.0.1 {{(pid=80245) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 08:15:41.681587 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG uwsgi [-] keystone_authtoken.auth_host = 127.0.0.1 {{(pid=80244) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 08:15:41.681664 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG uwsgi [-] keystone_authtoken.auth_port = 35357 {{(pid=80245) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 08:15:41.681825 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG uwsgi [-] keystone_authtoken.auth_port = 35357 {{(pid=80244) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 08:15:41.681902 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG uwsgi [-] keystone_authtoken.auth_protocol = https {{(pid=80245) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 08:15:41.682072 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG uwsgi [-] keystone_authtoken.auth_protocol = https {{(pid=80244) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 08:15:41.682150 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG uwsgi [-] keystone_authtoken.auth_section = None {{(pid=80245) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 08:15:41.682304 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG uwsgi [-] keystone_authtoken.auth_section = None {{(pid=80244) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 08:15:41.682384 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG uwsgi [-] keystone_authtoken.auth_type = None {{(pid=80245) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 08:15:41.682543 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG uwsgi [-] keystone_authtoken.auth_type = None {{(pid=80244) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 08:15:41.682621 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG uwsgi [-] keystone_authtoken.auth_uri = None {{(pid=80245) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 08:15:41.682768 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG uwsgi [-] keystone_authtoken.auth_version = None {{(pid=80245) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 08:15:41.682843 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG uwsgi [-] keystone_authtoken.auth_uri = None {{(pid=80244) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 08:15:41.683002 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG uwsgi [-] keystone_authtoken.cache = None {{(pid=80245) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 08:15:41.683086 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG uwsgi [-] keystone_authtoken.auth_version = None {{(pid=80244) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 08:15:41.683234 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG uwsgi [-] keystone_authtoken.cache = None {{(pid=80244) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 08:15:41.683316 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG uwsgi [-] keystone_authtoken.cafile = None {{(pid=80245) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 08:15:41.683501 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG uwsgi [-] keystone_authtoken.cafile = None {{(pid=80244) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 08:15:41.683626 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG uwsgi [-] keystone_authtoken.certfile = None {{(pid=80245) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 08:15:41.683734 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG uwsgi [-] keystone_authtoken.certfile = None {{(pid=80244) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 08:15:41.683812 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG uwsgi [-] keystone_authtoken.delay_auth_decision = False {{(pid=80245) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 08:15:41.683970 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG uwsgi [-] keystone_authtoken.delay_auth_decision = False {{(pid=80244) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 08:15:41.684048 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG uwsgi [-] keystone_authtoken.enforce_token_bind = permissive {{(pid=80245) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 08:15:41.684206 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG uwsgi [-] keystone_authtoken.http_connect_timeout = None {{(pid=80245) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 08:15:41.684292 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG uwsgi [-] keystone_authtoken.enforce_token_bind = permissive {{(pid=80244) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 08:15:41.684437 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG uwsgi [-] keystone_authtoken.http_connect_timeout = None {{(pid=80244) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 08:15:41.684513 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG uwsgi [-] keystone_authtoken.http_request_max_retries = 3 {{(pid=80245) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 08:15:41.684697 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG uwsgi [-] keystone_authtoken.identity_uri = None {{(pid=80245) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 08:15:41.684783 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG uwsgi [-] keystone_authtoken.http_request_max_retries = 3 {{(pid=80244) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 08:15:41.684897 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG uwsgi [-] keystone_authtoken.identity_uri = None {{(pid=80244) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 08:15:41.685178 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG uwsgi [-] keystone_authtoken.include_service_catalog = True {{(pid=80245) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 08:15:41.685320 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG uwsgi [-] keystone_authtoken.include_service_catalog = True {{(pid=80244) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 08:15:41.685450 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG uwsgi [-] keystone_authtoken.insecure = False {{(pid=80245) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 08:15:41.685576 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG uwsgi [-] keystone_authtoken.insecure = False {{(pid=80244) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 08:15:41.685711 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG uwsgi [-] keystone_authtoken.interface = internal {{(pid=80245) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 08:15:41.685846 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG uwsgi [-] keystone_authtoken.interface = internal {{(pid=80244) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 08:15:41.686006 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG uwsgi [-] keystone_authtoken.keyfile = None {{(pid=80245) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 08:15:41.686161 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG uwsgi [-] keystone_authtoken.keyfile = None {{(pid=80244) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 08:15:41.686297 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_conn_get_timeout = 10 {{(pid=80245) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 08:15:41.686421 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_conn_get_timeout = 10 {{(pid=80244) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 08:15:41.686527 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_dead_retry = 300 {{(pid=80245) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 08:15:41.686617 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_dead_retry = 300 {{(pid=80244) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 08:15:41.686689 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_maxsize = 10 {{(pid=80245) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 08:15:41.686803 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_maxsize = 10 {{(pid=80244) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 08:15:41.686881 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_socket_timeout = 3 {{(pid=80245) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 08:15:41.686993 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_socket_timeout = 3 {{(pid=80244) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 08:15:41.687066 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_unused_timeout = 60 {{(pid=80245) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 08:15:41.687186 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_unused_timeout = 60 {{(pid=80244) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 08:15:41.687261 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG uwsgi [-] keystone_authtoken.memcache_secret_key = **** {{(pid=80245) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 08:15:41.687361 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG uwsgi [-] keystone_authtoken.memcache_secret_key = **** {{(pid=80244) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 08:15:41.687466 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG uwsgi [-] keystone_authtoken.memcache_security_strategy = None {{(pid=80245) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 08:15:41.687578 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG uwsgi [-] keystone_authtoken.memcache_security_strategy = None {{(pid=80244) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 08:15:41.687661 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG uwsgi [-] keystone_authtoken.memcache_use_advanced_pool = True {{(pid=80245) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 08:15:41.687742 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG uwsgi [-] keystone_authtoken.memcache_use_advanced_pool = True {{(pid=80244) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 08:15:41.687865 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG uwsgi [-] keystone_authtoken.memcached_servers = None {{(pid=80245) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 08:15:41.687941 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG uwsgi [-] keystone_authtoken.memcached_servers = None {{(pid=80244) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 08:15:41.688062 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG uwsgi [-] keystone_authtoken.region_name = None {{(pid=80245) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 08:15:41.688144 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG uwsgi [-] keystone_authtoken.region_name = None {{(pid=80244) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 08:15:41.688227 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles = ['service'] {{(pid=80245) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 08:15:41.688299 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles = ['service'] {{(pid=80244) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 08:15:41.688418 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles_required = False {{(pid=80245) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 08:15:41.688504 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles_required = False {{(pid=80244) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 08:15:41.688606 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG uwsgi [-] keystone_authtoken.service_type = None {{(pid=80245) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 08:15:41.688738 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG uwsgi [-] keystone_authtoken.service_type = None {{(pid=80244) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 08:15:41.688848 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG uwsgi [-] keystone_authtoken.token_cache_time = 300 {{(pid=80245) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 08:15:41.688978 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG uwsgi [-] keystone_authtoken.token_cache_time = 300 {{(pid=80244) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 08:15:41.689089 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG uwsgi [-] keystone_authtoken.www_authenticate_uri = None {{(pid=80245) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 08:15:41.689218 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG uwsgi [-] keystone_authtoken.www_authenticate_uri = None {{(pid=80244) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 08:15:41.689360 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG uwsgi [-] cache.backend = dogpile.cache.memcached {{(pid=80245) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 08:15:41.689487 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG uwsgi [-] cache.backend = dogpile.cache.memcached {{(pid=80244) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 08:15:41.689610 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG uwsgi [-] cache.backend_argument = **** {{(pid=80245) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 08:15:41.689729 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG uwsgi [-] cache.backend_argument = **** {{(pid=80244) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 08:15:41.689837 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG uwsgi [-] cache.config_prefix = cache.oslo {{(pid=80245) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 08:15:41.689979 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG uwsgi [-] cache.config_prefix = cache.oslo {{(pid=80244) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 08:15:41.690085 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG uwsgi [-] cache.dead_timeout = 60.0 {{(pid=80245) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 08:15:41.690191 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG uwsgi [-] cache.dead_timeout = 60.0 {{(pid=80244) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 08:15:41.690309 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG uwsgi [-] cache.debug_cache_backend = False {{(pid=80245) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 08:15:41.690429 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG uwsgi [-] cache.debug_cache_backend = False {{(pid=80244) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 08:15:41.690556 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG uwsgi [-] cache.enable_retry_client = False {{(pid=80245) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 08:15:41.690703 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG uwsgi [-] cache.enable_retry_client = False {{(pid=80244) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 08:15:41.690807 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG uwsgi [-] cache.enable_socket_keepalive = False {{(pid=80245) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 08:15:41.690917 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG uwsgi [-] cache.enable_socket_keepalive = False {{(pid=80244) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 08:15:41.691032 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG uwsgi [-] cache.enabled = True {{(pid=80245) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 08:15:41.691112 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG uwsgi [-] cache.enabled = True {{(pid=80244) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 08:15:41.691289 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG uwsgi [-] cache.expiration_time = 600 {{(pid=80245) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 08:15:41.691401 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG uwsgi [-] cache.expiration_time = 600 {{(pid=80244) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 08:15:41.691543 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG uwsgi [-] cache.hashclient_retry_attempts = 2 {{(pid=80245) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 08:15:41.691633 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG uwsgi [-] cache.hashclient_retry_attempts = 2 {{(pid=80244) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 08:15:41.691786 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG uwsgi [-] cache.hashclient_retry_delay = 1.0 {{(pid=80245) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 08:15:41.691865 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG uwsgi [-] cache.hashclient_retry_delay = 1.0 {{(pid=80244) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 08:15:41.692025 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG uwsgi [-] cache.memcache_dead_retry = 300 {{(pid=80245) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 08:15:41.692110 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG uwsgi [-] cache.memcache_dead_retry = 300 {{(pid=80244) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 08:15:41.692266 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG uwsgi [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=80244) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 08:15:41.692350 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG uwsgi [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=80245) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 08:15:41.692502 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG uwsgi [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=80244) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 08:15:41.692581 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG uwsgi [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=80245) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 08:15:41.692738 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG uwsgi [-] cache.memcache_pool_maxsize = 10 {{(pid=80244) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 08:15:41.692843 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG uwsgi [-] cache.memcache_pool_maxsize = 10 {{(pid=80245) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 08:15:41.693003 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG uwsgi [-] cache.memcache_pool_unused_timeout = 60 {{(pid=80245) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 08:15:41.693082 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG uwsgi [-] cache.memcache_pool_unused_timeout = 60 {{(pid=80244) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 08:15:41.693262 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG uwsgi [-] cache.memcache_servers = ['localhost:11211'] {{(pid=80244) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 08:15:41.693371 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG uwsgi [-] cache.memcache_servers = ['localhost:11211'] {{(pid=80245) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 08:15:41.693502 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG uwsgi [-] cache.memcache_socket_timeout = 1.0 {{(pid=80244) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 08:15:41.693581 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG uwsgi [-] cache.memcache_socket_timeout = 1.0 {{(pid=80245) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 08:15:41.693741 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG uwsgi [-] cache.proxies = [] {{(pid=80244) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 08:15:41.693820 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG uwsgi [-] cache.proxies = [] {{(pid=80245) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 08:15:41.693978 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG uwsgi [-] cache.retry_attempts = 2 {{(pid=80245) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 08:15:41.694056 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG uwsgi [-] cache.retry_attempts = 2 {{(pid=80244) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 08:15:41.694201 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG uwsgi [-] cache.retry_delay = 0.0 {{(pid=80244) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 08:15:41.694276 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG uwsgi [-] cache.retry_delay = 0.0 {{(pid=80245) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 08:15:41.694439 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG uwsgi [-] cache.socket_keepalive_count = 1 {{(pid=80244) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 08:15:41.694518 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG uwsgi [-] cache.socket_keepalive_count = 1 {{(pid=80245) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 08:15:41.694675 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG uwsgi [-] cache.socket_keepalive_idle = 1 {{(pid=80244) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 08:15:41.694752 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG uwsgi [-] cache.socket_keepalive_idle = 1 {{(pid=80245) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 08:15:41.694912 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG uwsgi [-] cache.socket_keepalive_interval = 1 {{(pid=80244) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 08:15:41.695010 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG uwsgi [-] cache.socket_keepalive_interval = 1 {{(pid=80245) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 08:15:41.695139 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG uwsgi [-] cache.tls_allowed_ciphers = None {{(pid=80244) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 08:15:41.695220 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG uwsgi [-] cache.tls_allowed_ciphers = None {{(pid=80245) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 08:15:41.695389 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG uwsgi [-] cache.tls_cafile = None {{(pid=80244) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 08:15:41.695482 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG uwsgi [-] cache.tls_cafile = None {{(pid=80245) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 08:15:41.695622 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG uwsgi [-] cache.tls_certfile = None {{(pid=80244) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 08:15:41.695700 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG uwsgi [-] cache.tls_certfile = None {{(pid=80245) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 08:15:41.695857 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG uwsgi [-] cache.tls_enabled = False {{(pid=80245) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 08:15:41.695936 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG uwsgi [-] cache.tls_enabled = False {{(pid=80244) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 08:15:41.696088 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG uwsgi [-] cache.tls_keyfile = None {{(pid=80244) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 08:15:41.696174 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG uwsgi [-] cache.tls_keyfile = None {{(pid=80245) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 08:15:41.696326 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG uwsgi [-] application_credential.cache_time = None {{(pid=80244) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 08:15:41.696405 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG uwsgi [-] application_credential.cache_time = None {{(pid=80245) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 08:15:41.696564 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG uwsgi [-] application_credential.caching = True {{(pid=80245) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 08:15:41.696648 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG uwsgi [-] application_credential.caching = True {{(pid=80244) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 08:15:41.696812 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG uwsgi [-] application_credential.driver = sql {{(pid=80244) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 08:15:41.696889 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG uwsgi [-] application_credential.driver = sql {{(pid=80245) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 08:15:41.697048 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG uwsgi [-] application_credential.user_limit = -1 {{(pid=80244) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 08:15:41.697132 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG uwsgi [-] application_credential.user_limit = -1 {{(pid=80245) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 08:15:41.697305 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG uwsgi [-] assignment.driver = sql {{(pid=80245) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 08:15:41.697391 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG uwsgi [-] assignment.driver = sql {{(pid=80244) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 08:15:41.697567 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG uwsgi [-] assignment.prohibited_implied_role = ['admin'] {{(pid=80245) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 08:15:41.697651 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG uwsgi [-] assignment.prohibited_implied_role = ['admin'] {{(pid=80244) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 08:15:41.697810 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG uwsgi [-] auth.application_credential = None {{(pid=80244) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 08:15:41.697887 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG uwsgi [-] auth.application_credential = None {{(pid=80245) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 08:15:41.698043 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG uwsgi [-] auth.external = None {{(pid=80245) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 08:15:41.698121 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG uwsgi [-] auth.external = None {{(pid=80244) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 08:15:41.698272 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG uwsgi [-] auth.mapped = None {{(pid=80244) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 08:15:41.698349 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG uwsgi [-] auth.mapped = None {{(pid=80245) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 08:15:41.698567 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG uwsgi [-] auth.methods = ['external', 'password', 'token', 'oauth1', 'mapped', 'application_credential'] {{(pid=80245) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 08:15:41.698674 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG uwsgi [-] auth.methods = ['external', 'password', 'token', 'oauth1', 'mapped', 'application_credential'] {{(pid=80244) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 08:15:41.698799 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG uwsgi [-] auth.oauth1 = None {{(pid=80244) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 08:15:41.698886 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG uwsgi [-] auth.oauth1 = None {{(pid=80245) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 08:15:41.699031 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG uwsgi [-] auth.password = None {{(pid=80245) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 08:15:41.699108 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG uwsgi [-] auth.password = None {{(pid=80244) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 08:15:41.699259 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG uwsgi [-] auth.token = None {{(pid=80244) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 08:15:41.699352 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG uwsgi [-] auth.token = None {{(pid=80245) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 08:15:41.699527 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG uwsgi [-] catalog.cache_time = None {{(pid=80244) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 08:15:41.699615 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG uwsgi [-] catalog.cache_time = None {{(pid=80245) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 08:15:41.699764 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG uwsgi [-] catalog.caching = True {{(pid=80245) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 08:15:41.699845 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG uwsgi [-] catalog.caching = True {{(pid=80244) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 08:15:41.700009 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG uwsgi [-] catalog.driver = sql {{(pid=80244) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 08:15:41.700086 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG uwsgi [-] catalog.driver = sql {{(pid=80245) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 08:15:41.700241 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG uwsgi [-] catalog.list_limit = None {{(pid=80245) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 08:15:41.700326 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG uwsgi [-] catalog.list_limit = None {{(pid=80244) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 08:15:41.700483 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG uwsgi [-] catalog.template_file = default_catalog.templates {{(pid=80244) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 08:15:41.700562 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG uwsgi [-] catalog.template_file = default_catalog.templates {{(pid=80245) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 08:15:41.700728 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG uwsgi [-] credential.auth_ttl = 15 {{(pid=80245) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 08:15:41.700814 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG uwsgi [-] credential.auth_ttl = 15 {{(pid=80244) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 08:15:41.700959 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG uwsgi [-] credential.cache_time = None {{(pid=80244) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 08:15:41.701045 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG uwsgi [-] credential.cache_time = None {{(pid=80245) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 08:15:41.701195 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG uwsgi [-] credential.caching = True {{(pid=80245) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 08:15:41.701272 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG uwsgi [-] credential.caching = True {{(pid=80244) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 08:15:41.701441 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG uwsgi [-] credential.driver = sql {{(pid=80244) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 08:15:41.701522 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG uwsgi [-] credential.driver = sql {{(pid=80245) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 08:15:41.701691 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG uwsgi [-] credential.key_repository = /etc/keystone/credential-keys/ {{(pid=80245) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 08:15:41.701777 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG uwsgi [-] credential.key_repository = /etc/keystone/credential-keys/ {{(pid=80244) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 08:15:41.701937 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG uwsgi [-] credential.provider = fernet {{(pid=80244) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 08:15:41.702014 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG uwsgi [-] credential.provider = fernet {{(pid=80245) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 08:15:41.702174 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG uwsgi [-] credential.user_limit = -1 {{(pid=80245) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 08:15:41.702251 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG uwsgi [-] credential.user_limit = -1 {{(pid=80244) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 08:15:41.702417 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG uwsgi [-] domain_config.cache_time = 300 {{(pid=80244) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 08:15:41.702503 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG uwsgi [-] domain_config.cache_time = 300 {{(pid=80245) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 08:15:41.702652 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG uwsgi [-] domain_config.caching = True {{(pid=80245) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 08:15:41.702735 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG uwsgi [-] domain_config.caching = True {{(pid=80244) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 08:15:41.702916 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG uwsgi [-] domain_config.driver = sql {{(pid=80244) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 08:15:41.702993 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG uwsgi [-] domain_config.driver = sql {{(pid=80245) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 08:15:41.703169 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG uwsgi [-] endpoint_filter.driver = sql {{(pid=80245) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 08:15:41.703254 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG uwsgi [-] endpoint_filter.driver = sql {{(pid=80244) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 08:15:41.703428 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG uwsgi [-] endpoint_filter.return_all_endpoints_if_no_filter = True {{(pid=80244) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 08:15:41.703510 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG uwsgi [-] endpoint_filter.return_all_endpoints_if_no_filter = True {{(pid=80245) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 08:15:41.703683 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG uwsgi [-] endpoint_policy.driver = sql {{(pid=80244) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 08:15:41.703762 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG uwsgi [-] endpoint_policy.driver = sql {{(pid=80245) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 08:15:41.703967 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG uwsgi [-] eventlet_server.admin_bind_host = 0.0.0.0 {{(pid=80245) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 08:15:41.704060 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG uwsgi [-] eventlet_server.admin_bind_host = 0.0.0.0 {{(pid=80244) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 08:15:41.704214 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG uwsgi [-] eventlet_server.admin_port = 35357 {{(pid=80244) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 08:15:41.704292 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG uwsgi [-] eventlet_server.admin_port = 35357 {{(pid=80245) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 08:15:41.704475 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG uwsgi [-] eventlet_server.public_bind_host = 0.0.0.0 {{(pid=80245) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 08:15:41.704559 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG uwsgi [-] eventlet_server.public_bind_host = 0.0.0.0 {{(pid=80244) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 08:15:41.704718 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG uwsgi [-] eventlet_server.public_port = 5000 {{(pid=80244) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 08:15:41.704807 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG uwsgi [-] eventlet_server.public_port = 5000 {{(pid=80245) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 08:15:41.704976 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG uwsgi [-] federation.assertion_prefix = {{(pid=80245) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 08:15:41.705054 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG uwsgi [-] federation.assertion_prefix = {{(pid=80244) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 08:15:41.705212 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG uwsgi [-] federation.caching = True {{(pid=80244) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 08:15:41.705290 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG uwsgi [-] federation.caching = True {{(pid=80245) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 08:15:41.705450 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG uwsgi [-] federation.default_authorization_ttl = 0 {{(pid=80245) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 08:15:41.705540 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG uwsgi [-] federation.default_authorization_ttl = 0 {{(pid=80244) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 08:15:41.705695 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG uwsgi [-] federation.driver = sql {{(pid=80244) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 08:15:41.705774 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG uwsgi [-] federation.driver = sql {{(pid=80245) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 08:15:41.705942 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG uwsgi [-] federation.federated_domain_name = Federated {{(pid=80244) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 08:15:41.706027 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG uwsgi [-] federation.federated_domain_name = Federated {{(pid=80245) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 08:15:41.706174 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG uwsgi [-] federation.remote_id_attribute = None {{(pid=80244) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 08:15:41.706261 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG uwsgi [-] federation.remote_id_attribute = None {{(pid=80245) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 08:15:41.706421 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG uwsgi [-] federation.sso_callback_template = /etc/keystone/sso_callback_template.html {{(pid=80244) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 08:15:41.706507 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG uwsgi [-] federation.sso_callback_template = /etc/keystone/sso_callback_template.html {{(pid=80245) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 08:15:41.706659 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG uwsgi [-] federation.trusted_dashboard = [] {{(pid=80245) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 08:15:41.706738 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG uwsgi [-] federation.trusted_dashboard = [] {{(pid=80244) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 08:15:41.706910 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG uwsgi [-] fernet_receipts.key_repository = /etc/keystone/fernet-keys/ {{(pid=80244) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 08:15:41.706996 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG uwsgi [-] fernet_receipts.key_repository = /etc/keystone/fernet-keys/ {{(pid=80245) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 08:15:41.707149 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG uwsgi [-] fernet_receipts.max_active_keys = 3 {{(pid=80244) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 08:15:41.707228 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG uwsgi [-] fernet_receipts.max_active_keys = 3 {{(pid=80245) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 08:15:41.707427 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG uwsgi [-] fernet_tokens.key_repository = /etc/keystone/fernet-keys/ {{(pid=80244) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 08:15:41.707511 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG uwsgi [-] fernet_tokens.key_repository = /etc/keystone/fernet-keys/ {{(pid=80245) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 08:15:41.707668 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG uwsgi [-] fernet_tokens.max_active_keys = 3 {{(pid=80245) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 08:15:41.707825 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG uwsgi [-] fernet_tokens.max_active_keys = 3 {{(pid=80244) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 08:15:41.707905 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG uwsgi [-] identity.cache_time = 600 {{(pid=80244) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 08:15:41.707976 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG uwsgi [-] identity.cache_time = 600 {{(pid=80245) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 08:15:41.708132 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG uwsgi [-] identity.caching = True {{(pid=80244) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 08:15:41.708211 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG uwsgi [-] identity.caching = True {{(pid=80245) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 08:15:41.708378 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG uwsgi [-] identity.default_domain_id = default {{(pid=80244) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 08:15:41.709055 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG uwsgi [-] identity.default_domain_id = default {{(pid=80245) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 08:15:41.709162 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG uwsgi [-] identity.domain_config_dir = /etc/keystone/domains {{(pid=80245) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 08:15:41.709240 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG uwsgi [-] identity.domain_config_dir = /etc/keystone/domains {{(pid=80244) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 08:15:41.709399 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG uwsgi [-] identity.domain_configurations_from_database = False {{(pid=80244) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 08:15:41.709476 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG uwsgi [-] identity.domain_configurations_from_database = False {{(pid=80245) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 08:15:41.709635 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG uwsgi [-] identity.domain_specific_drivers_enabled = False {{(pid=80245) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 08:15:41.709721 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG uwsgi [-] identity.domain_specific_drivers_enabled = False {{(pid=80244) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 08:15:41.709993 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG uwsgi [-] identity.driver = sql {{(pid=80244) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 08:15:41.710071 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG uwsgi [-] identity.driver = sql {{(pid=80245) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 08:15:41.710233 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG uwsgi [-] identity.list_limit = None {{(pid=80245) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 08:15:41.710311 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG uwsgi [-] identity.list_limit = None {{(pid=80244) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 08:15:41.710474 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG uwsgi [-] identity.max_password_length = 4096 {{(pid=80244) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 08:15:41.710553 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG uwsgi [-] identity.max_password_length = 4096 {{(pid=80245) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 08:15:41.710727 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG uwsgi [-] identity.password_hash_algorithm = bcrypt {{(pid=80244) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 08:15:41.710806 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG uwsgi [-] identity.password_hash_algorithm = bcrypt {{(pid=80245) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 08:15:41.710982 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG uwsgi [-] identity.password_hash_rounds = 4 {{(pid=80245) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 08:15:41.711061 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG uwsgi [-] identity.password_hash_rounds = 4 {{(pid=80244) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 08:15:41.711222 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG uwsgi [-] identity.salt_bytesize = None {{(pid=80245) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 08:15:41.711303 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG uwsgi [-] identity.salt_bytesize = None {{(pid=80244) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 08:15:41.711486 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG uwsgi [-] identity.scrypt_block_size = None {{(pid=80245) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 08:15:41.711570 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG uwsgi [-] identity.scrypt_block_size = None {{(pid=80244) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 08:15:41.711724 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG uwsgi [-] identity.scrypt_parallelism = None {{(pid=80245) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 08:15:41.711859 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG uwsgi [-] identity.scrypt_parallelism = None {{(pid=80244) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 08:15:41.711976 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG uwsgi [-] identity_mapping.backward_compatible_ids = True {{(pid=80245) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 08:15:41.712057 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG uwsgi [-] identity_mapping.backward_compatible_ids = True {{(pid=80244) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 08:15:41.712228 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG uwsgi [-] identity_mapping.driver = sql {{(pid=80245) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 08:15:41.712307 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG uwsgi [-] identity_mapping.driver = sql {{(pid=80244) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 08:15:41.712478 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG uwsgi [-] identity_mapping.generator = sha256 {{(pid=80244) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 08:15:41.712556 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG uwsgi [-] identity_mapping.generator = sha256 {{(pid=80245) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 08:15:41.712735 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG uwsgi [-] jwt_tokens.jws_private_key_repository = /etc/keystone/jws-keys/private {{(pid=80244) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 08:15:41.712816 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG uwsgi [-] jwt_tokens.jws_private_key_repository = /etc/keystone/jws-keys/private {{(pid=80245) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 08:15:41.712983 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG uwsgi [-] jwt_tokens.jws_public_key_repository = /etc/keystone/jws-keys/public {{(pid=80244) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 08:15:41.713062 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG uwsgi [-] jwt_tokens.jws_public_key_repository = /etc/keystone/jws-keys/public {{(pid=80245) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 08:15:41.713255 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG uwsgi [-] ldap.alias_dereferencing = default {{(pid=80245) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 08:15:41.713333 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG uwsgi [-] ldap.alias_dereferencing = default {{(pid=80244) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 08:15:41.713499 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG uwsgi [-] ldap.auth_pool_connection_lifetime = 60 {{(pid=80245) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 08:15:41.713580 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG uwsgi [-] ldap.auth_pool_connection_lifetime = 60 {{(pid=80244) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 08:15:41.713740 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG uwsgi [-] ldap.auth_pool_size = 100 {{(pid=80244) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 08:15:41.713820 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG uwsgi [-] ldap.auth_pool_size = 100 {{(pid=80245) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 08:15:41.713976 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG uwsgi [-] ldap.chase_referrals = None {{(pid=80245) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 08:15:41.714054 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG uwsgi [-] ldap.chase_referrals = None {{(pid=80244) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 08:15:41.714213 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG uwsgi [-] ldap.connection_timeout = -1 {{(pid=80244) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 08:15:41.714292 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG uwsgi [-] ldap.connection_timeout = -1 {{(pid=80245) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 08:15:41.714446 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG uwsgi [-] ldap.debug_level = None {{(pid=80245) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 08:15:41.714524 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG uwsgi [-] ldap.debug_level = None {{(pid=80244) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 08:15:41.715886 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG uwsgi [-] ldap.group_ad_nesting = False {{(pid=80244) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 08:15:41.715886 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG uwsgi [-] ldap.group_additional_attribute_mapping = [] {{(pid=80244) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 08:15:41.716052 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG uwsgi [-] ldap.group_ad_nesting = False {{(pid=80245) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 08:15:41.716117 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG uwsgi [-] ldap.group_attribute_ignore = [] {{(pid=80244) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 08:15:41.716177 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG uwsgi [-] ldap.group_additional_attribute_mapping = [] {{(pid=80245) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 08:15:41.716235 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG uwsgi [-] ldap.group_desc_attribute = description {{(pid=80244) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 08:15:41.716294 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG uwsgi [-] ldap.group_attribute_ignore = [] {{(pid=80245) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 08:15:41.716352 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG uwsgi [-] ldap.group_filter = None {{(pid=80244) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 08:15:41.716417 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG uwsgi [-] ldap.group_desc_attribute = description {{(pid=80245) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 08:15:41.716479 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG uwsgi [-] ldap.group_id_attribute = cn {{(pid=80244) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 08:15:41.716537 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG uwsgi [-] ldap.group_filter = None {{(pid=80245) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 08:15:41.716594 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG uwsgi [-] ldap.group_member_attribute = member {{(pid=80244) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 08:15:41.716651 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG uwsgi [-] ldap.group_id_attribute = cn {{(pid=80245) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 08:15:41.716708 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG uwsgi [-] ldap.group_members_are_ids = False {{(pid=80244) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 08:15:41.716775 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG uwsgi [-] ldap.group_member_attribute = member {{(pid=80245) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 08:15:41.716837 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG uwsgi [-] ldap.group_name_attribute = ou {{(pid=80244) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 08:15:41.716837 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG uwsgi [-] ldap.group_objectclass = groupOfNames {{(pid=80244) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 08:15:41.717058 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG uwsgi [-] ldap.group_tree_dn = None {{(pid=80244) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 08:15:41.717254 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG uwsgi [-] ldap.page_size = 0 {{(pid=80244) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 08:15:41.717439 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG uwsgi [-] ldap.group_members_are_ids = False {{(pid=80245) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 08:15:41.717574 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG uwsgi [-] ldap.password = **** {{(pid=80244) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 08:15:41.717713 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG uwsgi [-] ldap.group_name_attribute = ou {{(pid=80245) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 08:15:41.717823 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG uwsgi [-] ldap.pool_connection_lifetime = 600 {{(pid=80244) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 08:15:41.717949 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG uwsgi [-] ldap.pool_connection_timeout = -1 {{(pid=80244) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 08:15:41.718029 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG uwsgi [-] ldap.group_objectclass = groupOfNames {{(pid=80245) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 08:15:41.718195 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG uwsgi [-] ldap.pool_retry_delay = 0.1 {{(pid=80244) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 08:15:41.718285 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG uwsgi [-] ldap.group_tree_dn = None {{(pid=80245) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 08:15:41.718435 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG uwsgi [-] ldap.pool_retry_max = 3 {{(pid=80244) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 08:15:41.718513 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG uwsgi [-] ldap.page_size = 0 {{(pid=80245) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 08:15:41.718669 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG uwsgi [-] ldap.password = **** {{(pid=80245) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 08:15:41.718746 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG uwsgi [-] ldap.pool_size = 10 {{(pid=80244) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 08:15:41.718914 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG uwsgi [-] ldap.pool_connection_lifetime = 600 {{(pid=80245) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 08:15:41.718999 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG uwsgi [-] ldap.query_scope = one {{(pid=80244) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 08:15:41.719156 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG uwsgi [-] ldap.pool_connection_timeout = -1 {{(pid=80245) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 08:15:41.719234 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG uwsgi [-] ldap.suffix = cn=example,cn=com {{(pid=80244) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 08:15:41.719412 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG uwsgi [-] ldap.tls_cacertdir = None {{(pid=80244) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 08:15:41.719492 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG uwsgi [-] ldap.pool_retry_delay = 0.1 {{(pid=80245) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 08:15:41.719647 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG uwsgi [-] ldap.tls_cacertfile = None {{(pid=80244) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 08:15:41.719737 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG uwsgi [-] ldap.pool_retry_max = 3 {{(pid=80245) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 08:15:41.719886 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG uwsgi [-] ldap.pool_size = 10 {{(pid=80245) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 08:15:41.719962 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG uwsgi [-] ldap.tls_req_cert = demand {{(pid=80244) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 08:15:41.720131 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG uwsgi [-] ldap.url = ldap://localhost {{(pid=80244) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 08:15:41.720206 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG uwsgi [-] ldap.query_scope = one {{(pid=80245) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 08:15:41.720369 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG uwsgi [-] ldap.use_auth_pool = True {{(pid=80244) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 08:15:41.720458 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG uwsgi [-] ldap.suffix = cn=example,cn=com {{(pid=80245) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 08:15:41.720603 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG uwsgi [-] ldap.tls_cacertdir = None {{(pid=80245) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 08:15:41.720679 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG uwsgi [-] ldap.use_pool = True {{(pid=80244) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 08:15:41.720837 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG uwsgi [-] ldap.use_tls = False {{(pid=80244) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 08:15:41.720912 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG uwsgi [-] ldap.tls_cacertfile = None {{(pid=80245) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 08:15:41.721075 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG uwsgi [-] ldap.user = None {{(pid=80244) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 08:15:41.721160 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG uwsgi [-] ldap.tls_req_cert = demand {{(pid=80245) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 08:15:41.721316 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG uwsgi [-] ldap.user_additional_attribute_mapping = [] {{(pid=80244) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 08:15:41.721394 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG uwsgi [-] ldap.url = ldap://localhost {{(pid=80245) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 08:15:41.721553 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG uwsgi [-] ldap.use_auth_pool = True {{(pid=80245) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 08:15:41.721629 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG uwsgi [-] ldap.user_attribute_ignore = ['default_project_id'] {{(pid=80244) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 08:15:41.721792 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG uwsgi [-] ldap.user_default_project_id_attribute = None {{(pid=80244) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 08:15:41.721876 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG uwsgi [-] ldap.use_pool = True {{(pid=80245) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 08:15:41.722028 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG uwsgi [-] ldap.use_tls = False {{(pid=80245) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 08:15:41.722104 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG uwsgi [-] ldap.user_description_attribute = description {{(pid=80244) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 08:15:41.722263 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG uwsgi [-] ldap.user = None {{(pid=80245) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 08:15:41.722340 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG uwsgi [-] ldap.user_enabled_attribute = enabled {{(pid=80244) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 08:15:41.722506 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG uwsgi [-] ldap.user_enabled_default = True {{(pid=80244) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 08:15:41.722590 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG uwsgi [-] ldap.user_additional_attribute_mapping = [] {{(pid=80245) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 08:15:41.722741 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG uwsgi [-] ldap.user_enabled_emulation = False {{(pid=80244) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 08:15:41.722819 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG uwsgi [-] ldap.user_attribute_ignore = ['default_project_id'] {{(pid=80245) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 08:15:41.722970 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG uwsgi [-] ldap.user_enabled_emulation_dn = None {{(pid=80244) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 08:15:41.723064 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG uwsgi [-] ldap.user_default_project_id_attribute = None {{(pid=80245) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 08:15:41.723207 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG uwsgi [-] ldap.user_enabled_emulation_use_group_config = False {{(pid=80244) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 08:15:41.723285 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG uwsgi [-] ldap.user_description_attribute = description {{(pid=80245) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 08:15:41.723466 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG uwsgi [-] ldap.user_enabled_invert = False {{(pid=80244) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 08:15:41.723563 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG uwsgi [-] ldap.user_enabled_attribute = enabled {{(pid=80245) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 08:15:41.723706 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG uwsgi [-] ldap.user_enabled_mask = 0 {{(pid=80244) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 08:15:41.723793 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG uwsgi [-] ldap.user_enabled_default = True {{(pid=80245) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 08:15:41.723936 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG uwsgi [-] ldap.user_filter = None {{(pid=80244) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 08:15:41.724022 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG uwsgi [-] ldap.user_enabled_emulation = False {{(pid=80245) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 08:15:41.724818 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG uwsgi [-] ldap.user_enabled_emulation_dn = None {{(pid=80245) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 08:15:41.724910 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG uwsgi [-] ldap.user_id_attribute = cn {{(pid=80244) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 08:15:41.725075 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG uwsgi [-] ldap.user_mail_attribute = mail {{(pid=80244) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 08:15:41.725160 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG uwsgi [-] ldap.user_enabled_emulation_use_group_config = False {{(pid=80245) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 08:15:41.725327 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG uwsgi [-] ldap.user_name_attribute = sn {{(pid=80244) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 08:15:41.725413 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG uwsgi [-] ldap.user_enabled_invert = False {{(pid=80245) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 08:15:41.725582 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG uwsgi [-] ldap.user_enabled_mask = 0 {{(pid=80245) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 08:15:41.725667 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG uwsgi [-] ldap.user_objectclass = inetOrgPerson {{(pid=80244) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 08:15:41.725818 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG uwsgi [-] ldap.user_filter = None {{(pid=80245) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 08:15:41.725904 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG uwsgi [-] ldap.user_pass_attribute = userPassword {{(pid=80244) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 08:15:41.726048 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG uwsgi [-] ldap.user_tree_dn = None {{(pid=80244) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 08:15:41.726135 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG uwsgi [-] ldap.user_id_attribute = cn {{(pid=80245) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 08:15:41.726295 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG uwsgi [-] memcache.dead_retry = 300 {{(pid=80244) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 08:15:41.726381 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG uwsgi [-] ldap.user_mail_attribute = mail {{(pid=80245) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 08:15:41.726535 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG uwsgi [-] memcache.pool_connection_get_timeout = 10 {{(pid=80244) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 08:15:41.726622 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG uwsgi [-] ldap.user_name_attribute = sn {{(pid=80245) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 08:15:41.726773 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG uwsgi [-] memcache.pool_maxsize = 10 {{(pid=80244) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 08:15:41.726859 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG uwsgi [-] ldap.user_objectclass = inetOrgPerson {{(pid=80245) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 08:15:41.727009 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG uwsgi [-] memcache.pool_unused_timeout = 60 {{(pid=80244) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 08:15:41.727095 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG uwsgi [-] ldap.user_pass_attribute = userPassword {{(pid=80245) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 08:15:41.727241 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG uwsgi [-] ldap.user_tree_dn = None {{(pid=80245) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 08:15:41.727353 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG uwsgi [-] memcache.socket_timeout = 3 {{(pid=80244) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 08:15:41.727507 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG uwsgi [-] oauth1.access_token_duration = 86400 {{(pid=80244) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 08:15:41.727593 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG uwsgi [-] memcache.dead_retry = 300 {{(pid=80245) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 08:15:41.727750 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG uwsgi [-] memcache.pool_connection_get_timeout = 10 {{(pid=80245) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 08:15:41.727836 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG uwsgi [-] oauth1.driver = sql {{(pid=80244) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 08:15:41.727986 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG uwsgi [-] oauth1.request_token_duration = 28800 {{(pid=80244) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 08:15:41.728116 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG uwsgi [-] memcache.pool_maxsize = 10 {{(pid=80245) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 08:15:41.728249 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG uwsgi [-] memcache.pool_unused_timeout = 60 {{(pid=80245) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 08:15:41.728375 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG uwsgi [-] policy.driver = sql {{(pid=80244) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 08:15:41.728499 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG uwsgi [-] policy.list_limit = None {{(pid=80244) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 08:15:41.728622 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG uwsgi [-] memcache.socket_timeout = 3 {{(pid=80245) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 08:15:41.728748 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG uwsgi [-] receipt.cache_on_issue = True {{(pid=80244) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 08:15:41.728872 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG uwsgi [-] oauth1.access_token_duration = 86400 {{(pid=80245) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 08:15:41.729005 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG uwsgi [-] receipt.cache_time = 300 {{(pid=80244) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 08:15:41.729130 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG uwsgi [-] oauth1.driver = sql {{(pid=80245) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 08:15:41.729254 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG uwsgi [-] receipt.caching = True {{(pid=80244) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 08:15:41.729379 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG uwsgi [-] oauth1.request_token_duration = 28800 {{(pid=80245) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 08:15:41.729512 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG uwsgi [-] receipt.expiration = 300 {{(pid=80244) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 08:15:41.729637 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG uwsgi [-] policy.driver = sql {{(pid=80245) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 08:15:41.729770 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG uwsgi [-] receipt.provider = fernet {{(pid=80244) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 08:15:41.729895 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG uwsgi [-] policy.list_limit = None {{(pid=80245) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 08:15:41.730019 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG uwsgi [-] resource.admin_project_domain_name = None {{(pid=80244) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 08:15:41.730148 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG uwsgi [-] receipt.cache_on_issue = True {{(pid=80245) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 08:15:41.730267 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG uwsgi [-] resource.admin_project_name = None {{(pid=80244) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 08:15:41.730393 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG uwsgi [-] receipt.cache_time = 300 {{(pid=80245) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 08:15:41.730526 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG uwsgi [-] resource.cache_time = None {{(pid=80244) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 08:15:41.730651 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG uwsgi [-] receipt.caching = True {{(pid=80245) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 08:15:41.730777 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG uwsgi [-] resource.caching = True {{(pid=80244) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 08:15:41.730902 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG uwsgi [-] receipt.expiration = 300 {{(pid=80245) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 08:15:41.731026 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG uwsgi [-] resource.domain_name_url_safe = off {{(pid=80244) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 08:15:41.731160 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG uwsgi [-] receipt.provider = fernet {{(pid=80245) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 08:15:41.731293 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG uwsgi [-] resource.driver = sql {{(pid=80244) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 08:15:41.731444 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG uwsgi [-] resource.admin_project_domain_name = None {{(pid=80245) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 08:15:41.731566 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG uwsgi [-] resource.list_limit = None {{(pid=80244) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 08:15:41.731691 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG uwsgi [-] resource.admin_project_name = None {{(pid=80245) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 08:15:41.731817 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG uwsgi [-] resource.project_name_url_safe = off {{(pid=80244) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 08:15:41.731941 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG uwsgi [-] resource.cache_time = None {{(pid=80245) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 08:15:41.732075 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG uwsgi [-] revoke.cache_time = 3600 {{(pid=80244) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 08:15:41.732201 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG uwsgi [-] resource.caching = True {{(pid=80245) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 08:15:41.732325 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG uwsgi [-] revoke.caching = True {{(pid=80244) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 08:15:41.732450 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG uwsgi [-] resource.domain_name_url_safe = off {{(pid=80245) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 08:15:41.732575 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG uwsgi [-] revoke.driver = sql {{(pid=80244) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 08:15:41.732699 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG uwsgi [-] resource.driver = sql {{(pid=80245) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 08:15:41.732831 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG uwsgi [-] revoke.expiration_buffer = 1800 {{(pid=80244) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 08:15:41.732967 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG uwsgi [-] resource.list_limit = None {{(pid=80245) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 08:15:41.733093 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG uwsgi [-] role.cache_time = None {{(pid=80244) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 08:15:41.733217 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG uwsgi [-] resource.project_name_url_safe = off {{(pid=80245) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 08:15:41.733348 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG uwsgi [-] role.caching = True {{(pid=80244) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 08:15:41.733474 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG uwsgi [-] revoke.cache_time = 3600 {{(pid=80245) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 08:15:41.733676 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG uwsgi [-] role.driver = sql {{(pid=80244) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 08:15:41.733801 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG uwsgi [-] revoke.caching = True {{(pid=80245) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 08:15:41.733927 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG uwsgi [-] role.list_limit = None {{(pid=80244) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 08:15:41.734051 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG uwsgi [-] revoke.driver = sql {{(pid=80245) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 08:15:41.734175 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG uwsgi [-] saml.assertion_expiration_time = 3600 {{(pid=80244) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 08:15:41.734300 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG uwsgi [-] revoke.expiration_buffer = 1800 {{(pid=80245) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 08:15:41.734432 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG uwsgi [-] saml.certfile = /etc/keystone/ssl/certs/signing_cert.pem {{(pid=80244) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 08:15:41.734558 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG uwsgi [-] role.cache_time = None {{(pid=80245) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 08:15:41.734683 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG uwsgi [-] saml.idp_contact_company = Example, Inc. {{(pid=80244) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 08:15:41.734816 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG uwsgi [-] role.caching = True {{(pid=80245) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 08:15:41.734942 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG uwsgi [-] saml.idp_contact_email = support@example.com {{(pid=80244) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 08:15:41.735067 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG uwsgi [-] role.driver = sql {{(pid=80245) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 08:15:41.735200 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG uwsgi [-] saml.idp_contact_name = SAML Identity Provider Support {{(pid=80244) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 08:15:41.735338 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG uwsgi [-] role.list_limit = None {{(pid=80245) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 08:15:41.735501 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG uwsgi [-] saml.idp_contact_surname = Support {{(pid=80244) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 08:15:41.735635 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG uwsgi [-] saml.assertion_expiration_time = 3600 {{(pid=80245) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 08:15:41.735760 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG uwsgi [-] saml.idp_contact_telephone = +1 800 555 0100 {{(pid=80244) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 08:15:41.735885 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG uwsgi [-] saml.certfile = /etc/keystone/ssl/certs/signing_cert.pem {{(pid=80245) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 08:15:41.736019 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG uwsgi [-] saml.idp_contact_type = other {{(pid=80244) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 08:15:41.736144 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG uwsgi [-] saml.idp_contact_company = Example, Inc. {{(pid=80245) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 08:15:41.736270 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG uwsgi [-] saml.idp_entity_id = None {{(pid=80244) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 08:15:41.736394 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG uwsgi [-] saml.idp_contact_email = support@example.com {{(pid=80245) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 08:15:41.736528 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG uwsgi [-] saml.idp_lang = en {{(pid=80244) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 08:15:41.736661 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG uwsgi [-] saml.idp_contact_name = SAML Identity Provider Support {{(pid=80245) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 08:15:41.736787 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG uwsgi [-] saml.idp_metadata_path = /etc/keystone/saml2_idp_metadata.xml {{(pid=80244) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 08:15:41.736912 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG uwsgi [-] saml.idp_contact_surname = Support {{(pid=80245) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 08:15:41.737037 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG uwsgi [-] saml.idp_organization_display_name = OpenStack SAML Identity Provider {{(pid=80244) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 08:15:41.737163 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG uwsgi [-] saml.idp_contact_telephone = +1 800 555 0100 {{(pid=80245) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 08:15:41.737288 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG uwsgi [-] saml.idp_organization_name = SAML Identity Provider {{(pid=80244) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 08:15:41.737423 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG uwsgi [-] saml.idp_contact_type = other {{(pid=80245) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 08:15:41.737423 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG uwsgi [-] saml.idp_entity_id = None {{(pid=80245) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 08:15:41.737660 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG uwsgi [-] saml.idp_organization_url = https://example.com/ {{(pid=80244) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 08:15:41.737784 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG uwsgi [-] saml.idp_lang = en {{(pid=80245) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 08:15:41.737908 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG uwsgi [-] saml.idp_sso_endpoint = None {{(pid=80244) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 08:15:41.738032 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG uwsgi [-] saml.idp_metadata_path = /etc/keystone/saml2_idp_metadata.xml {{(pid=80245) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 08:15:41.738174 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG uwsgi [-] saml.keyfile = /etc/keystone/ssl/private/signing_key.pem {{(pid=80244) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 08:15:41.738300 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG uwsgi [-] saml.idp_organization_display_name = OpenStack SAML Identity Provider {{(pid=80245) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 08:15:41.738425 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG uwsgi [-] saml.relay_state_prefix = ss:mem: {{(pid=80244) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 08:15:41.738549 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG uwsgi [-] saml.idp_organization_name = SAML Identity Provider {{(pid=80245) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 08:15:41.738674 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG uwsgi [-] saml.xmlsec1_binary = xmlsec1 {{(pid=80244) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 08:15:41.738674 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG uwsgi [-] security_compliance.change_password_upon_first_use = False {{(pid=80244) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 08:15:41.738915 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG uwsgi [-] saml.idp_organization_url = https://example.com/ {{(pid=80245) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 08:15:41.739041 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG uwsgi [-] security_compliance.disable_user_account_days_inactive = None {{(pid=80244) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 08:15:41.739166 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG uwsgi [-] saml.idp_sso_endpoint = None {{(pid=80245) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 08:15:41.739291 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG uwsgi [-] security_compliance.lockout_duration = 10 {{(pid=80244) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 08:15:41.739427 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG uwsgi [-] saml.keyfile = /etc/keystone/ssl/private/signing_key.pem {{(pid=80245) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 08:15:41.739547 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG uwsgi [-] security_compliance.lockout_failure_attempts = 2 {{(pid=80244) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 08:15:41.739683 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG uwsgi [-] saml.relay_state_prefix = ss:mem: {{(pid=80245) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 08:15:41.739855 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG uwsgi [-] security_compliance.minimum_password_age = 0 {{(pid=80244) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 08:15:41.739982 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG uwsgi [-] saml.xmlsec1_binary = xmlsec1 {{(pid=80245) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 08:15:41.740108 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG uwsgi [-] security_compliance.password_expires_days = None {{(pid=80244) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 08:15:41.740237 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG uwsgi [-] security_compliance.change_password_upon_first_use = False {{(pid=80245) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 08:15:41.740363 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG uwsgi [-] security_compliance.password_regex = None {{(pid=80244) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 08:15:41.740496 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG uwsgi [-] security_compliance.disable_user_account_days_inactive = None {{(pid=80245) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 08:15:41.740622 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG uwsgi [-] security_compliance.password_regex_description = None {{(pid=80244) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 08:15:41.740747 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG uwsgi [-] security_compliance.lockout_duration = 10 {{(pid=80245) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 08:15:41.740872 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG uwsgi [-] security_compliance.unique_last_password_count = 2 {{(pid=80244) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 08:15:41.740996 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG uwsgi [-] security_compliance.lockout_failure_attempts = 2 {{(pid=80245) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 08:15:41.741122 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG uwsgi [-] shadow_users.driver = sql {{(pid=80244) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 08:15:41.741254 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG uwsgi [-] security_compliance.minimum_password_age = 0 {{(pid=80245) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 08:15:41.741380 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG uwsgi [-] token.allow_expired_window = 172800 {{(pid=80244) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 08:15:41.741504 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG uwsgi [-] security_compliance.password_expires_days = None {{(pid=80245) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 08:15:41.741641 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG uwsgi [-] token.allow_rescope_scoped_token = True {{(pid=80244) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 08:15:41.741768 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG uwsgi [-] security_compliance.password_regex = None {{(pid=80245) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 08:15:41.741893 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG uwsgi [-] token.cache_on_issue = True {{(pid=80244) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 08:15:41.742025 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG uwsgi [-] security_compliance.password_regex_description = None {{(pid=80245) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 08:15:41.742150 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG uwsgi [-] token.cache_time = None {{(pid=80244) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 08:15:41.742275 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG uwsgi [-] security_compliance.unique_last_password_count = 2 {{(pid=80245) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 08:15:41.742400 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG uwsgi [-] token.caching = True {{(pid=80244) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 08:15:41.742524 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG uwsgi [-] shadow_users.driver = sql {{(pid=80245) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 08:15:41.742659 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG uwsgi [-] token.expiration = 3600 {{(pid=80244) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 08:15:41.742786 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG uwsgi [-] token.allow_expired_window = 172800 {{(pid=80245) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 08:15:41.742910 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG uwsgi [-] token.provider = fernet {{(pid=80244) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 08:15:41.743046 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG uwsgi [-] token.allow_rescope_scoped_token = True {{(pid=80245) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 08:15:41.743172 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG uwsgi [-] token.revoke_by_id = True {{(pid=80244) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 08:15:41.743309 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG uwsgi [-] token.cache_on_issue = True {{(pid=80245) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 08:15:41.743469 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG uwsgi [-] tokenless_auth.issuer_attribute = SSL_CLIENT_I_DN {{(pid=80244) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 08:15:41.743596 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG uwsgi [-] token.cache_time = None {{(pid=80245) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 08:15:41.743721 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG uwsgi [-] tokenless_auth.protocol = x509 {{(pid=80244) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 08:15:41.743847 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG uwsgi [-] token.caching = True {{(pid=80245) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 08:15:41.743971 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG uwsgi [-] tokenless_auth.trusted_issuer = [] {{(pid=80244) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 08:15:41.744095 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG uwsgi [-] token.expiration = 3600 {{(pid=80245) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 08:15:41.744228 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG uwsgi [-] totp.included_previous_windows = 1 {{(pid=80244) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 08:15:41.744352 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG uwsgi [-] token.provider = fernet {{(pid=80245) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 08:15:41.744478 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG uwsgi [-] trust.allow_redelegation = False {{(pid=80244) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 08:15:41.744609 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG uwsgi [-] token.revoke_by_id = True {{(pid=80245) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 08:15:41.744734 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG uwsgi [-] trust.driver = sql {{(pid=80244) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 08:15:41.744860 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG uwsgi [-] tokenless_auth.issuer_attribute = SSL_CLIENT_I_DN {{(pid=80245) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 08:15:41.744997 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG uwsgi [-] trust.max_redelegation_count = 3 {{(pid=80244) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 08:15:41.745133 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG uwsgi [-] tokenless_auth.protocol = x509 {{(pid=80245) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 08:15:41.745258 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG uwsgi [-] unified_limit.cache_time = None {{(pid=80244) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 08:15:41.745383 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG uwsgi [-] tokenless_auth.trusted_issuer = [] {{(pid=80245) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 08:15:41.745509 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG uwsgi [-] unified_limit.caching = True {{(pid=80244) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 08:15:41.745634 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG uwsgi [-] totp.included_previous_windows = 1 {{(pid=80245) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 08:15:41.745766 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG uwsgi [-] unified_limit.driver = sql {{(pid=80244) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 08:15:41.745892 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG uwsgi [-] trust.allow_redelegation = False {{(pid=80245) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 08:15:41.746017 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG uwsgi [-] unified_limit.enforcement_model = flat {{(pid=80244) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 08:15:41.746141 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG uwsgi [-] trust.driver = sql {{(pid=80245) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 08:15:41.746265 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG uwsgi [-] unified_limit.list_limit = None {{(pid=80244) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 08:15:41.746390 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG uwsgi [-] trust.max_redelegation_count = 3 {{(pid=80245) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 08:15:41.746523 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG uwsgi [-] wsgi.debug_middleware = False {{(pid=80244) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 08:15:41.746649 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG uwsgi [-] unified_limit.cache_time = None {{(pid=80245) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 08:15:41.746774 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG uwsgi [-] database.backend = sqlalchemy {{(pid=80244) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 08:15:41.746908 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG uwsgi [-] unified_limit.caching = True {{(pid=80245) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 08:15:41.747033 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG uwsgi [-] database.connection = **** {{(pid=80244) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 08:15:41.747166 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG uwsgi [-] unified_limit.driver = sql {{(pid=80245) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 08:15:41.747298 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG uwsgi [-] database.connection_debug = 0 {{(pid=80244) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 08:15:41.747437 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG uwsgi [-] unified_limit.enforcement_model = flat {{(pid=80245) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 08:15:41.747526 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG uwsgi [-] database.connection_parameters = {{(pid=80244) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 08:15:41.747600 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG uwsgi [-] unified_limit.list_limit = None {{(pid=80245) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 08:15:41.747671 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG uwsgi [-] database.connection_recycle_time = 3600 {{(pid=80244) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 08:15:41.747741 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG uwsgi [-] wsgi.debug_middleware = False {{(pid=80245) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 08:15:41.747811 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG uwsgi [-] database.connection_trace = False {{(pid=80244) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 08:15:41.747881 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG uwsgi [-] database.backend = sqlalchemy {{(pid=80245) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 08:15:41.747959 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG uwsgi [-] database.db_inc_retry_interval = True {{(pid=80244) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 08:15:41.748039 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG uwsgi [-] database.connection = **** {{(pid=80245) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 08:15:41.748121 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG uwsgi [-] database.db_max_retries = 20 {{(pid=80244) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 08:15:41.748201 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG uwsgi [-] database.connection_debug = 0 {{(pid=80245) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 08:15:41.748273 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG uwsgi [-] database.db_max_retry_interval = 10 {{(pid=80244) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 08:15:41.748343 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG uwsgi [-] database.connection_parameters = {{(pid=80245) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 08:15:41.748414 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG uwsgi [-] database.db_retry_interval = 1 {{(pid=80244) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 08:15:41.748492 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG uwsgi [-] database.connection_recycle_time = 3600 {{(pid=80245) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 08:15:41.748564 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG uwsgi [-] database.max_overflow = 50 {{(pid=80244) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 08:15:41.748634 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG uwsgi [-] database.connection_trace = False {{(pid=80245) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 08:15:41.748704 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG uwsgi [-] database.max_pool_size = 5 {{(pid=80244) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 08:15:41.748774 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG uwsgi [-] database.db_inc_retry_interval = True {{(pid=80245) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 08:15:41.748844 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG uwsgi [-] database.max_retries = 10 {{(pid=80244) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 08:15:41.748922 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG uwsgi [-] database.db_max_retries = 20 {{(pid=80245) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 08:15:41.748993 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG uwsgi [-] database.mysql_enable_ndb = False {{(pid=80244) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 08:15:41.749064 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG uwsgi [-] database.db_max_retry_interval = 10 {{(pid=80245) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 08:15:41.749134 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG uwsgi [-] database.mysql_sql_mode = TRADITIONAL {{(pid=80244) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 08:15:41.749203 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG uwsgi [-] database.db_retry_interval = 1 {{(pid=80245) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 08:15:41.749273 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG uwsgi [-] database.pool_timeout = None {{(pid=80244) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 08:15:41.749351 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG uwsgi [-] database.max_overflow = 50 {{(pid=80245) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 08:15:41.749423 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG uwsgi [-] database.retry_interval = 10 {{(pid=80244) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 08:15:41.749423 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG uwsgi [-] database.slave_connection = **** {{(pid=80244) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 08:15:41.749549 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG uwsgi [-] database.max_pool_size = 5 {{(pid=80245) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 08:15:41.749620 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG uwsgi [-] database.sqlite_synchronous = True {{(pid=80244) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 08:15:41.749690 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG uwsgi [-] database.max_retries = 10 {{(pid=80245) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 08:15:41.749767 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG uwsgi [-] database.use_db_reconnect = False {{(pid=80244) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 08:15:41.749839 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG uwsgi [-] database.mysql_enable_ndb = False {{(pid=80245) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 08:15:41.749910 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=80244) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} Jul 21 08:15:41.749980 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG uwsgi [-] database.mysql_sql_mode = TRADITIONAL {{(pid=80245) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 08:15:41.749980 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG uwsgi [-] database.pool_timeout = None {{(pid=80245) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 08:15:41.750162 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG uwsgi [-] database.retry_interval = 10 {{(pid=80245) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 08:15:41.750392 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG uwsgi [-] database.slave_connection = **** {{(pid=80245) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 08:15:41.750635 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG uwsgi [-] database.sqlite_synchronous = True {{(pid=80245) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 08:15:41.750875 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG uwsgi [-] database.use_db_reconnect = False {{(pid=80245) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 08:15:41.751047 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=80245) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} Jul 21 08:15:41.780883 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.deleted`. {{(pid=80244) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 21 08:15:41.781111 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.disabled`. {{(pid=80244) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 21 08:15:41.781336 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_assignment_removal` subscribed to event `identity.remove_application_credentials_for_user.internal`. {{(pid=80244) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 21 08:15:41.781565 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.deleted`. {{(pid=80245) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 21 08:15:41.781799 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.disabled`. {{(pid=80245) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 21 08:15:41.782024 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_assignment_removal` subscribed to event `identity.remove_application_credentials_for_user.internal`. {{(pid=80245) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 21 08:15:41.790387 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.notifications [-] Callback: `keystone.assignment.core.Manager._delete_domain_assignments` subscribed to event `identity.domain.deleted`. {{(pid=80244) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 21 08:15:41.790967 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.notifications [-] Callback: `keystone.assignment.core.Manager._delete_domain_assignments` subscribed to event `identity.domain.deleted`. {{(pid=80245) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 21 08:15:41.818285 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.project.deleted`. {{(pid=80244) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 21 08:15:41.818512 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.endpoint.deleted`. {{(pid=80244) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 21 08:15:41.819408 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.project.deleted`. {{(pid=80245) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 21 08:15:41.819647 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.endpoint.deleted`. {{(pid=80245) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 21 08:15:41.961921 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.notifications [-] Callback: `keystone.federation.core.Manager._cleanup_identity_provider` subscribed to event `identity.domain_deleted.internal`. {{(pid=80244) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 21 08:15:41.962125 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.notifications [-] Callback: `keystone.federation.core.Manager._cleanup_identity_provider` subscribed to event `identity.domain_deleted.internal`. {{(pid=80245) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 21 08:15:42.011962 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._domain_deleted` subscribed to event `identity.domain_deleted.internal`. {{(pid=80244) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 21 08:15:42.012125 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._unset_default_project` subscribed to event `identity.project.deleted`. {{(pid=80244) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 21 08:15:42.012760 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._domain_deleted` subscribed to event `identity.domain_deleted.internal`. {{(pid=80245) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 21 08:15:42.012997 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._unset_default_project` subscribed to event `identity.project.deleted`. {{(pid=80245) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 21 08:15:42.076403 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._trust_callback` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80244) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 21 08:15:42.076627 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._consumer_callback` subscribed to event `identity.OS-OAUTH1:consumer.deleted`. {{(pid=80244) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 21 08:15:42.076852 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.deleted`. {{(pid=80244) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 21 08:15:42.076970 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._trust_callback` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80245) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 21 08:15:42.077073 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._project_callback` subscribed to event `identity.project.deleted`. {{(pid=80244) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 21 08:15:42.077206 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._consumer_callback` subscribed to event `identity.OS-OAUTH1:consumer.deleted`. {{(pid=80245) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 21 08:15:42.077289 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.disabled`. {{(pid=80244) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 21 08:15:42.077433 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.deleted`. {{(pid=80245) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 21 08:15:42.077515 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.persist_revocation_event_for_user.internal`. {{(pid=80244) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 21 08:15:42.077655 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._project_callback` subscribed to event `identity.project.deleted`. {{(pid=80245) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 21 08:15:42.077877 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.disabled`. {{(pid=80245) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 21 08:15:42.078094 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.persist_revocation_event_for_user.internal`. {{(pid=80245) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 21 08:15:42.102615 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80244) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 21 08:15:42.102829 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80245) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 21 08:15:42.102916 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.deleted`. {{(pid=80244) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 21 08:15:42.103048 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.deleted`. {{(pid=80244) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 21 08:15:42.103132 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.deleted`. {{(pid=80245) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 21 08:15:42.103268 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.disabled`. {{(pid=80244) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 21 08:15:42.103369 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.deleted`. {{(pid=80245) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 21 08:15:42.103513 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.disabled`. {{(pid=80244) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 21 08:15:42.103609 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.disabled`. {{(pid=80245) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 21 08:15:42.103728 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.project.disabled`. {{(pid=80244) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 21 08:15:42.103810 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.disabled`. {{(pid=80245) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 21 08:15:42.103940 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=80244) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 21 08:15:42.104023 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.project.disabled`. {{(pid=80245) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 21 08:15:42.104156 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=80245) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 21 08:15:42.115659 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.notifications [-] Callback: `keystone.trust.core.Manager._on_user_delete` subscribed to event `identity.user.deleted`. {{(pid=80244) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 21 08:15:42.116055 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.notifications [-] Callback: `keystone.trust.core.Manager._on_user_delete` subscribed to event `identity.user.deleted`. {{(pid=80245) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 21 08:15:42.128265 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80244) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 21 08:15:42.128494 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.deleted`. {{(pid=80244) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 21 08:15:42.128662 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80245) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 21 08:15:42.128764 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.deleted`. {{(pid=80244) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 21 08:15:42.128898 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.deleted`. {{(pid=80245) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 21 08:15:42.128981 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.disabled`. {{(pid=80244) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 21 08:15:42.129117 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.disabled`. {{(pid=80244) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 21 08:15:42.129198 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.deleted`. {{(pid=80245) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 21 08:15:42.129327 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.project.disabled`. {{(pid=80244) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 21 08:15:42.129411 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.disabled`. {{(pid=80245) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 21 08:15:42.129541 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=80244) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 21 08:15:42.129624 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.disabled`. {{(pid=80245) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 21 08:15:42.129755 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.project.disabled`. {{(pid=80245) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 21 08:15:42.129969 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=80245) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 21 08:15:42.146879 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/projects' {}] {{(pid=80244) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 08:15:42.147091 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/projects' {}] {{(pid=80244) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Jul 21 08:15:42.147429 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/projects' {}] {{(pid=80245) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 08:15:42.147515 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/domains' {}] {{(pid=80244) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 08:15:42.147645 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/projects' {}] {{(pid=80245) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Jul 21 08:15:42.147741 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/domains' {}] {{(pid=80244) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Jul 21 08:15:42.147908 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/system' {}] {{(pid=80244) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 08:15:42.147987 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/domains' {}] {{(pid=80245) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 08:15:42.148163 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/domains' {}] {{(pid=80245) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Jul 21 08:15:42.148242 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/catalog' {}] {{(pid=80244) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 08:15:42.148415 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens/OS-PKI/revoked' {}] {{(pid=80244) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 08:15:42.148502 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/system' {}] {{(pid=80245) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 08:15:42.148667 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens' {}] {{(pid=80244) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 08:15:42.148758 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/catalog' {}] {{(pid=80245) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 08:15:42.148923 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens/OS-PKI/revoked' {}] {{(pid=80245) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 08:15:42.149179 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens' {}] {{(pid=80245) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 08:15:42.159730 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2' {}] {{(pid=80244) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 08:15:42.160006 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2/ecp' {}] {{(pid=80244) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 08:15:42.160275 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2' {}] {{(pid=80245) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 08:15:42.160362 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/websso/' {}] {{(pid=80244) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 08:15:42.160562 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2/ecp' {}] {{(pid=80245) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 08:15:42.160670 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/identity_providers//protocols//websso' {}] {{(pid=80244) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 08:15:42.160825 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/websso/' {}] {{(pid=80245) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 08:15:42.161108 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/identity_providers//protocols//websso' {}] {{(pid=80245) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 08:15:42.169199 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.common [-] Adding standard routes to API credentials for `MethodViewType` (API Prefix: ) [/credentials, /credentials/] {{(pid=80244) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 21 08:15:42.169774 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.common [-] Adding standard routes to API credentials for `MethodViewType` (API Prefix: ) [/credentials, /credentials/] {{(pid=80245) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 21 08:15:42.172506 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.common [-] Adding standard routes to API domains for `MethodViewType` (API Prefix: ) [/domains, /domains/] {{(pid=80244) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 21 08:15:42.172782 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config' {}] {{(pid=80244) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 08:15:42.173095 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.common [-] Adding standard routes to API domains for `MethodViewType` (API Prefix: ) [/domains, /domains/] {{(pid=80245) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 21 08:15:42.173179 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config/' {}] {{(pid=80244) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 08:15:42.173373 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config' {}] {{(pid=80245) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 08:15:42.173456 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config//' {}] {{(pid=80244) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 08:15:42.173714 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config/default' {}] {{(pid=80244) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 08:15:42.173796 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config/' {}] {{(pid=80245) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 08:15:42.173972 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config//default' {}] {{(pid=80244) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 08:15:42.174054 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config//' {}] {{(pid=80245) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 08:15:42.174237 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config///default' {}] {{(pid=80244) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 08:15:42.174329 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config/default' {}] {{(pid=80245) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 08:15:42.174507 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles' {}] {{(pid=80244) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 08:15:42.174591 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config//default' {}] {{(pid=80245) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 08:15:42.174775 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles/' {}] {{(pid=80244) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 08:15:42.174856 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config///default' {}] {{(pid=80245) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 08:15:42.175046 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles' {}] {{(pid=80244) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 08:15:42.175135 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles' {}] {{(pid=80245) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 08:15:42.175313 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles/' {}] {{(pid=80244) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 08:15:42.175412 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles/' {}] {{(pid=80245) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 08:15:42.175632 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles' {}] {{(pid=80245) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 08:15:42.175905 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles/' {}] {{(pid=80245) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 08:15:42.202469 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.common [-] Adding resource routes to API ec2tokens: ['/ec2tokens' {}] {{(pid=80244) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 08:15:42.202874 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.common [-] Adding resource routes to API ec2tokens: ['/ec2tokens' {}] {{(pid=80245) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 08:15:42.204118 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.common [-] Adding standard routes to API endpoints for `MethodViewType` (API Prefix: ) [/endpoints, /endpoints/] {{(pid=80244) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 21 08:15:42.204397 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.common [-] Adding resource routes to API endpoints: ['/endpoints//OS-ENDPOINT-POLICY/policy' {}] {{(pid=80244) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 08:15:42.204526 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.common [-] Adding standard routes to API endpoints for `MethodViewType` (API Prefix: ) [/endpoints, /endpoints/] {{(pid=80245) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 21 08:15:42.204808 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.common [-] Adding resource routes to API endpoints: ['/endpoints//OS-ENDPOINT-POLICY/policy' {}] {{(pid=80245) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 08:15:42.209724 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.common [-] Adding standard routes to API groups for `MethodViewType` (API Prefix: ) [/groups, /groups/] {{(pid=80244) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 21 08:15:42.209998 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users' {}] {{(pid=80244) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 08:15:42.210137 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.common [-] Adding standard routes to API groups for `MethodViewType` (API Prefix: ) [/groups, /groups/] {{(pid=80245) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 21 08:15:42.210279 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users/' {}] {{(pid=80244) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 08:15:42.210414 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users' {}] {{(pid=80245) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 08:15:42.210692 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users/' {}] {{(pid=80245) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 08:15:42.217567 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.common [-] Adding standard routes to API limits for `MethodViewType` (API Prefix: ) [/limits, /limits/] {{(pid=80244) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 21 08:15:42.217841 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.common [-] Adding resource routes to API limits: ['/limits/model' {}] {{(pid=80244) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 08:15:42.217990 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.common [-] Adding standard routes to API limits for `MethodViewType` (API Prefix: ) [/limits, /limits/] {{(pid=80245) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 21 08:15:42.218274 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.common [-] Adding resource routes to API limits: ['/limits/model' {}] {{(pid=80245) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 08:15:42.222465 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-EP-FILTER for `MethodViewType` (API Prefix: /OS-EP-FILTER) [/endpoint_groups, /endpoint_groups/] {{(pid=80244) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 21 08:15:42.222745 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoints//projects' {}] {{(pid=80244) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 08:15:42.222897 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-EP-FILTER for `MethodViewType` (API Prefix: /OS-EP-FILTER) [/endpoint_groups, /endpoint_groups/] {{(pid=80245) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 21 08:15:42.223027 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints/' {}] {{(pid=80244) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 08:15:42.223183 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoints//projects' {}] {{(pid=80245) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 08:15:42.223310 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints' {}] {{(pid=80244) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 08:15:42.223490 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints/' {}] {{(pid=80245) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 08:15:42.223609 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoint_groups' {}] {{(pid=80244) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 08:15:42.223774 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints' {}] {{(pid=80245) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 08:15:42.223885 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//endpoints' {}] {{(pid=80244) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 08:15:42.224536 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoint_groups' {}] {{(pid=80245) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 08:15:42.224654 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects' {}] {{(pid=80244) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 08:15:42.224837 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//endpoints' {}] {{(pid=80245) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 08:15:42.224919 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects/' {}] {{(pid=80244) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 08:15:42.225107 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects' {}] {{(pid=80245) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 08:15:42.225373 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects/' {}] {{(pid=80245) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 08:15:42.244063 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/saml2/metadata' {}] {{(pid=80244) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 08:15:42.244338 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/identity_providers//protocols//auth' {}] {{(pid=80244) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 08:15:42.244565 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/saml2/metadata' {}] {{(pid=80245) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 08:15:42.244842 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/identity_providers//protocols//auth' {}] {{(pid=80245) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 08:15:42.248969 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.common [-] Adding standard routes to API identity_providers for `MethodViewType` (API Prefix: /OS-FEDERATION) [/identity_providers, /identity_providers/] {{(pid=80244) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 21 08:15:42.249427 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.common [-] Adding standard routes to API identity_providers for `MethodViewType` (API Prefix: /OS-FEDERATION) [/identity_providers, /identity_providers/] {{(pid=80245) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 21 08:15:42.252645 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols/' {}] {{(pid=80244) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 08:15:42.252937 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols' {}] {{(pid=80244) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 08:15:42.253083 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols/' {}] {{(pid=80245) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 08:15:42.253380 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols' {}] {{(pid=80245) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 08:15:42.258252 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.common [-] Adding standard routes to API mappings for `MethodViewType` (API Prefix: /OS-FEDERATION) [/mappings, /mappings/] {{(pid=80244) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 21 08:15:42.258695 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.common [-] Adding standard routes to API mappings for `MethodViewType` (API Prefix: /OS-FEDERATION) [/mappings, /mappings/] {{(pid=80245) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 21 08:15:42.261898 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.common [-] Adding standard routes to API service_providers for `MethodViewType` (API Prefix: /OS-FEDERATION) [/service_providers, /service_providers/] {{(pid=80244) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 21 08:15:42.262394 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.common [-] Adding standard routes to API service_providers for `MethodViewType` (API Prefix: /OS-FEDERATION) [/service_providers, /service_providers/] {{(pid=80245) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 21 08:15:42.265699 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles//inherited_to_projects' {}] {{(pid=80244) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 08:15:42.265995 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles/inherited_to_projects' {}] {{(pid=80244) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 08:15:42.266143 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles//inherited_to_projects' {}] {{(pid=80245) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 08:15:42.266291 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles//inherited_to_projects' {}] {{(pid=80244) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 08:15:42.266457 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles/inherited_to_projects' {}] {{(pid=80245) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 08:15:42.266618 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles/inherited_to_projects' {}] {{(pid=80244) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 08:15:42.266754 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles//inherited_to_projects' {}] {{(pid=80245) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 08:15:42.266876 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//users//roles//inherited_to_projects' {}] {{(pid=80244) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 08:15:42.267023 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles/inherited_to_projects' {}] {{(pid=80245) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 08:15:42.267152 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//groups//roles//inherited_to_projects' {}] {{(pid=80244) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 08:15:42.267313 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//users//roles//inherited_to_projects' {}] {{(pid=80245) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 08:15:42.267629 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//groups//roles//inherited_to_projects' {}] {{(pid=80245) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 08:15:42.286089 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-OAUTH1 for `MethodViewType` (API Prefix: /OS-OAUTH1) [/consumers, /consumers/] {{(pid=80244) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 21 08:15:42.286469 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/request_token' {}] {{(pid=80244) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 08:15:42.287250 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/access_token' {}] {{(pid=80244) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 08:15:42.287370 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-OAUTH1 for `MethodViewType` (API Prefix: /OS-OAUTH1) [/consumers, /consumers/] {{(pid=80245) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 21 08:15:42.287538 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/authorize/' {}] {{(pid=80244) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 08:15:42.287733 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/request_token' {}] {{(pid=80245) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 08:15:42.288025 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/access_token' {}] {{(pid=80245) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 08:15:42.288299 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/authorize/' {}] {{(pid=80245) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 08:15:42.295489 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.common [-] Adding resource routes to API events: ['/events' {}] {{(pid=80244) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 08:15:42.296361 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.common [-] Adding resource routes to API events: ['/events' {}] {{(pid=80245) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 08:15:42.297199 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/ca' {}] {{(pid=80244) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 08:15:42.297473 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/certificates' {}] {{(pid=80244) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 08:15:42.298163 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/ca' {}] {{(pid=80245) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 08:15:42.298436 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/certificates' {}] {{(pid=80245) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 08:15:42.300547 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.common [-] Adding standard routes to API policy for `MethodViewType` (API Prefix: ) [/policies, /policies/] {{(pid=80244) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 21 08:15:42.300832 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints' {}] {{(pid=80244) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 08:15:42.301112 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints/' {}] {{(pid=80244) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 08:15:42.301390 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services/' {}] {{(pid=80244) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 08:15:42.301528 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.common [-] Adding standard routes to API policy for `MethodViewType` (API Prefix: ) [/policies, /policies/] {{(pid=80245) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 21 08:15:42.301686 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services//regions/' {}] {{(pid=80244) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 08:15:42.301809 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints' {}] {{(pid=80245) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 08:15:42.302095 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints/' {}] {{(pid=80245) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 08:15:42.302957 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services/' {}] {{(pid=80245) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 08:15:42.303245 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services//regions/' {}] {{(pid=80245) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 08:15:42.315618 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.common [-] Adding standard routes to API projects for `MethodViewType` (API Prefix: ) [/projects, /projects/] {{(pid=80244) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 21 08:15:42.315893 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags' {}] {{(pid=80244) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 08:15:42.316170 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags/' {}] {{(pid=80244) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 08:15:42.316459 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles/' {}] {{(pid=80244) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 08:15:42.316687 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.common [-] Adding standard routes to API projects for `MethodViewType` (API Prefix: ) [/projects, /projects/] {{(pid=80245) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 21 08:15:42.316769 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles' {}] {{(pid=80244) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 08:15:42.316965 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags' {}] {{(pid=80245) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 08:15:42.317046 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles/' {}] {{(pid=80244) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 08:15:42.317248 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags/' {}] {{(pid=80245) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 08:15:42.317379 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles' {}] {{(pid=80244) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 08:15:42.317537 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles/' {}] {{(pid=80245) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 08:15:42.317852 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles' {}] {{(pid=80245) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 08:15:42.318145 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles/' {}] {{(pid=80245) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 08:15:42.318426 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles' {}] {{(pid=80245) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 08:15:42.336535 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.common [-] Adding standard routes to API regions for `MethodViewType` (API Prefix: ) [/regions, /regions/] {{(pid=80244) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 21 08:15:42.337705 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.common [-] Adding standard routes to API regions for `MethodViewType` (API Prefix: ) [/regions, /regions/] {{(pid=80245) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 21 08:15:42.339810 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.common [-] Adding standard routes to API registered_limit for `MethodViewType` (API Prefix: ) [/registered_limits, /registered_limits/] {{(pid=80244) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 21 08:15:42.341029 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.common [-] Adding standard routes to API registered_limit for `MethodViewType` (API Prefix: ) [/registered_limits, /registered_limits/] {{(pid=80245) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 21 08:15:42.343123 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_assignments: ['/role_assignments' {}] {{(pid=80244) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 08:15:42.344376 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_assignments: ['/role_assignments' {}] {{(pid=80245) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 08:15:42.344748 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_inferences: ['/role_inferences' {}] {{(pid=80244) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 08:15:42.345999 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_inferences: ['/role_inferences' {}] {{(pid=80245) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 08:15:42.346382 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.common [-] Adding standard routes to API roles for `MethodViewType` (API Prefix: ) [/roles, /roles/] {{(pid=80244) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 21 08:15:42.346658 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies' {}] {{(pid=80244) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 08:15:42.346946 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies/' {}] {{(pid=80244) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 08:15:42.347671 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.common [-] Adding standard routes to API roles for `MethodViewType` (API Prefix: ) [/roles, /roles/] {{(pid=80245) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 21 08:15:42.347948 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies' {}] {{(pid=80245) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 08:15:42.348235 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies/' {}] {{(pid=80245) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 08:15:42.354952 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.common [-] Adding resource routes to API s3tokens: ['/s3tokens' {}] {{(pid=80244) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 08:15:42.356151 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.common [-] Adding resource routes to API s3tokens: ['/s3tokens' {}] {{(pid=80245) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 08:15:42.356623 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.common [-] Adding standard routes to API services for `MethodViewType` (API Prefix: ) [/services, /services/] {{(pid=80244) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 21 08:15:42.357785 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.common [-] Adding standard routes to API services for `MethodViewType` (API Prefix: ) [/services, /services/] {{(pid=80245) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 21 08:15:42.359845 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles' {}] {{(pid=80244) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 08:15:42.360133 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles/' {}] {{(pid=80244) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 08:15:42.360416 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles' {}] {{(pid=80244) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 08:15:42.360697 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles/' {}] {{(pid=80244) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 08:15:42.361037 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles' {}] {{(pid=80245) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 08:15:42.361329 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles/' {}] {{(pid=80245) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 08:15:42.361613 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles' {}] {{(pid=80245) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 08:15:42.361890 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles/' {}] {{(pid=80245) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 08:15:42.370461 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.common [-] Adding standard routes to API trusts for `MethodViewType` (API Prefix: /OS-TRUST) [/trusts, /trusts/] {{(pid=80244) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 21 08:15:42.370744 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles' {}] {{(pid=80244) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 08:15:42.371022 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles/' {}] {{(pid=80244) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 08:15:42.371804 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.common [-] Adding standard routes to API trusts for `MethodViewType` (API Prefix: /OS-TRUST) [/trusts, /trusts/] {{(pid=80245) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 21 08:15:42.372091 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles' {}] {{(pid=80245) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 08:15:42.372369 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles/' {}] {{(pid=80245) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 08:15:42.378884 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.common [-] Adding standard routes to API users for `MethodViewType` (API Prefix: ) [/users, /users/] {{(pid=80244) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 21 08:15:42.379156 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//password' {}] {{(pid=80244) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 08:15:42.379460 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//groups' {}] {{(pid=80244) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 08:15:42.379732 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//projects' {}] {{(pid=80244) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 08:15:42.380002 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2' {}] {{(pid=80244) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 08:15:42.380270 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.common [-] Adding standard routes to API users for `MethodViewType` (API Prefix: ) [/users, /users/] {{(pid=80245) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 21 08:15:42.380452 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2/' {}] {{(pid=80244) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 08:15:42.380574 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens' {}] {{(pid=80244) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 08:15:42.380654 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//password' {}] {{(pid=80245) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 08:15:42.380850 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens/' {}] {{(pid=80244) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 08:15:42.380932 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//groups' {}] {{(pid=80245) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 08:15:42.381126 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles' {}] {{(pid=80244) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 08:15:42.381205 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//projects' {}] {{(pid=80245) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 08:15:42.381398 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles/' {}] {{(pid=80244) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 08:15:42.381486 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2' {}] {{(pid=80245) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 08:15:42.381666 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials' {}] {{(pid=80244) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 08:15:42.381754 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2/' {}] {{(pid=80245) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 08:15:42.381924 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials/' {}] {{(pid=80244) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 08:15:42.382053 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens' {}] {{(pid=80245) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 08:15:42.382198 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules' {}] {{(pid=80244) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 08:15:42.382278 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens/' {}] {{(pid=80245) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 08:15:42.382471 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules/' {}] {{(pid=80244) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 08:15:42.382551 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles' {}] {{(pid=80245) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 08:15:42.383038 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles/' {}] {{(pid=80245) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 08:15:42.383282 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials' {}] {{(pid=80245) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 08:15:42.383569 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials/' {}] {{(pid=80245) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 08:15:42.383844 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules' {}] {{(pid=80245) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 08:15:42.384111 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules/' {}] {{(pid=80245) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 08:15:42.421958 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: WSGI app 0 (mountpoint='') ready in 6 seconds on interpreter 0x55e05a35d9c0 pid: 80244 (default app) Jul 21 08:15:42.423468 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: WSGI app 0 (mountpoint='') ready in 6 seconds on interpreter 0x55e05a35d9c0 pid: 80245 (default app) Jul 21 08:15:42.425388 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9f262ff-9caa-44df-9718-451c617eceee None None] REQUEST_METHOD: `GET` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:15:42.425679 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9f262ff-9caa-44df-9718-451c617eceee None None] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:15:42.425968 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9f262ff-9caa-44df-9718-451c617eceee None None] PATH_INFO: `/v3` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:15:42.430201 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 1/1] 10.222.0.47 () {50 vars in 791 bytes} [Thu Jul 21 08:15:42 2022] GET /identity/v3/ => generated 255 bytes in 8 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 08:15:50.200431 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f8e61a9-a56e-4fde-9956-dbf9b40dadc2 None None] REQUEST_METHOD: `GET` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:15:50.200990 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f8e61a9-a56e-4fde-9956-dbf9b40dadc2 None None] SCRIPT_NAME: `/identity` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:15:50.200990 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f8e61a9-a56e-4fde-9956-dbf9b40dadc2 None None] PATH_INFO: `/` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:15:50.205302 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: [pid: 80245|app: 0|req: 1/2] 10.222.0.47 () {54 vars in 926 bytes} [Thu Jul 21 08:15:50 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 08:15:50.210899 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e313282-053e-48a5-9f64-81a2804e518b None None] REQUEST_METHOD: `POST` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:15:50.211312 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e313282-053e-48a5-9f64-81a2804e518b None None] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:15:50.211492 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e313282-053e-48a5-9f64-81a2804e518b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:15:50.281013 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: INFO dbcounter [None req-0e313282-053e-48a5-9f64-81a2804e518b None None] Registered counter for database keystone Jul 21 08:15:50.303334 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG oslo_db.sqlalchemy.engines [None req-0e313282-053e-48a5-9f64-81a2804e518b None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=80244) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Jul 21 08:15:50.309634 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG dbcounter [-] [80244] Writer thread running {{(pid=80244) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Jul 21 08:15:50.484774 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG passlib.handlers.bcrypt [None req-0e313282-053e-48a5-9f64-81a2804e518b None None] detected 'bcrypt' backend, version '3.2.2' {{(pid=80244) _load_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:625}} Jul 21 08:15:50.485469 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG passlib.handlers.bcrypt [None req-0e313282-053e-48a5-9f64-81a2804e518b None None] 'bcrypt' backend lacks $2$ support, enabling workaround {{(pid=80244) _finalize_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:406}} Jul 21 08:15:50.540168 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.auth.core [None req-0e313282-053e-48a5-9f64-81a2804e518b None None] MFA Rules not processed for user `2dce40b8d74c41ec94cc36e6ff05d412`. Rule list: `[]` (Enabled: `True`). {{(pid=80244) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 08:15:50.541775 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.fernet_utils [None req-0e313282-053e-48a5-9f64-81a2804e518b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80244) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 08:15:50.666405 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 2/3] 10.222.0.47 () {58 vars in 1023 bytes} [Thu Jul 21 08:15:50 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 456 msecs (HTTP/1.1 201) 6 headers in 384 bytes (2 switches on core 0) Jul 21 08:15:50.876352 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f8e61a9-a56e-4fde-9956-dbf9b40dadc2 None None] Authenticating user token {{(pid=80245) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:15:50.947551 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: INFO dbcounter [None req-1f8e61a9-a56e-4fde-9956-dbf9b40dadc2 None None] Registered counter for database keystone Jul 21 08:15:50.967894 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG oslo_db.sqlalchemy.engines [None req-1f8e61a9-a56e-4fde-9956-dbf9b40dadc2 None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=80245) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Jul 21 08:15:50.974038 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG dbcounter [-] [80245] Writer thread running {{(pid=80245) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Jul 21 08:15:51.090832 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f8e61a9-a56e-4fde-9956-dbf9b40dadc2 None None] Validating token access rules against request {{(pid=80245) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:15:51.095094 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59fbb499-424e-49ec-9f47-78a3d38716b6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '42d15fc0515945acacbbffcf2356c7a5', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80245) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:15:51.095778 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59fbb499-424e-49ec-9f47-78a3d38716b6 admin admin] REQUEST_METHOD: `GET` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:15:51.096059 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59fbb499-424e-49ec-9f47-78a3d38716b6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:15:51.096329 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59fbb499-424e-49ec-9f47-78a3d38716b6 admin admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:15:51.104861 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59fbb499-424e-49ec-9f47-78a3d38716b6 admin admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:15:51.156671 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: WARNING keystone.common.rbac_enforcer.enforcer [None req-59fbb499-424e-49ec-9f47-78a3d38716b6 admin admin] Deprecated policy rules found. Use oslopolicy-policy-generator and oslopolicy-policy-upgrade to detect and resolve deprecated policies in your configuration. Jul 21 08:15:51.157007 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59fbb499-424e-49ec-9f47-78a3d38716b6 admin admin] RBAC: Authorization granted {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:15:51.167430 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: [pid: 80245|app: 0|req: 2/4] 10.222.0.47 () {58 vars in 1324 bytes} [Thu Jul 21 08:15:50 2022] GET /identity/v3/auth/tokens => generated 914 bytes in 292 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 21 08:15:51.173351 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e313282-053e-48a5-9f64-81a2804e518b None None] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:15:51.188325 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e313282-053e-48a5-9f64-81a2804e518b None None] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:15:51.192683 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b015a934-ac57-42b0-9f46-0c307ab096de admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '42d15fc0515945acacbbffcf2356c7a5', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:15:51.193809 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b015a934-ac57-42b0-9f46-0c307ab096de admin admin] REQUEST_METHOD: `GET` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:15:51.194095 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b015a934-ac57-42b0-9f46-0c307ab096de admin admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:15:51.194370 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b015a934-ac57-42b0-9f46-0c307ab096de admin admin] PATH_INFO: `/v3/projects/42d15fc0515945acacbbffcf2356c7a5` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:15:51.195900 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b015a934-ac57-42b0-9f46-0c307ab096de admin admin] RBAC: Authorizing `identity:get_project(project_id=42d15fc0515945acacbbffcf2356c7a5)` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:15:51.249385 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: WARNING keystone.common.rbac_enforcer.enforcer [None req-b015a934-ac57-42b0-9f46-0c307ab096de admin admin] Deprecated policy rules found. Use oslopolicy-policy-generator and oslopolicy-policy-upgrade to detect and resolve deprecated policies in your configuration. Jul 21 08:15:51.249929 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b015a934-ac57-42b0-9f46-0c307ab096de admin admin] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:15:51.254292 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 3/5] 10.222.0.47 () {56 vars in 1207 bytes} [Thu Jul 21 08:15:51 2022] GET /identity/v3/projects/42d15fc0515945acacbbffcf2356c7a5 => generated 344 bytes in 82 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 08:15:53.255832 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c0e5a66-e671-418d-bbb9-0a8f4f0a4423 None None] REQUEST_METHOD: `GET` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:15:53.256361 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c0e5a66-e671-418d-bbb9-0a8f4f0a4423 None None] SCRIPT_NAME: `/identity` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:15:53.256361 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c0e5a66-e671-418d-bbb9-0a8f4f0a4423 None None] PATH_INFO: `/` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:15:53.261271 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: [pid: 80245|app: 0|req: 3/6] 10.222.0.47 () {54 vars in 926 bytes} [Thu Jul 21 08:15:53 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 08:15:53.266633 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5790408c-0055-4b2d-b8f5-d6fa807d76f3 None None] REQUEST_METHOD: `POST` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:15:53.267057 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5790408c-0055-4b2d-b8f5-d6fa807d76f3 None None] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:15:53.267206 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5790408c-0055-4b2d-b8f5-d6fa807d76f3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:15:53.286387 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.auth.core [None req-5790408c-0055-4b2d-b8f5-d6fa807d76f3 None None] MFA Rules not processed for user `2dce40b8d74c41ec94cc36e6ff05d412`. Rule list: `[]` (Enabled: `True`). {{(pid=80244) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 08:15:53.287928 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.fernet_utils [None req-5790408c-0055-4b2d-b8f5-d6fa807d76f3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80244) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 08:15:53.295744 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 4/7] 10.222.0.47 () {58 vars in 1023 bytes} [Thu Jul 21 08:15:53 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 384 bytes (1 switches on core 0) Jul 21 08:15:53.323448 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c0e5a66-e671-418d-bbb9-0a8f4f0a4423 None None] Authenticating user token {{(pid=80245) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:15:53.338409 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c0e5a66-e671-418d-bbb9-0a8f4f0a4423 None None] Validating token access rules against request {{(pid=80245) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:15:53.342692 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8fb3f00-923b-4b60-a087-1516624d2946 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '42d15fc0515945acacbbffcf2356c7a5', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80245) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:15:53.343347 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8fb3f00-923b-4b60-a087-1516624d2946 admin admin] REQUEST_METHOD: `GET` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:15:53.343646 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8fb3f00-923b-4b60-a087-1516624d2946 admin admin] SCRIPT_NAME: `/identity` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:15:53.343922 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8fb3f00-923b-4b60-a087-1516624d2946 admin admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:15:53.351712 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8fb3f00-923b-4b60-a087-1516624d2946 admin admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:15:53.353501 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8fb3f00-923b-4b60-a087-1516624d2946 admin admin] RBAC: Authorization granted {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:15:53.362221 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: [pid: 80245|app: 0|req: 4/8] 10.222.0.47 () {58 vars in 1324 bytes} [Thu Jul 21 08:15:53 2022] GET /identity/v3/auth/tokens => generated 914 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 21 08:15:53.368012 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5790408c-0055-4b2d-b8f5-d6fa807d76f3 None None] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:15:53.377074 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5790408c-0055-4b2d-b8f5-d6fa807d76f3 None None] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:15:53.381501 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1396d2a8-57a8-4408-93bd-e57006baacd7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '42d15fc0515945acacbbffcf2356c7a5', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:15:53.383136 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1396d2a8-57a8-4408-93bd-e57006baacd7 admin admin] REQUEST_METHOD: `GET` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:15:53.383457 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1396d2a8-57a8-4408-93bd-e57006baacd7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:15:53.383735 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1396d2a8-57a8-4408-93bd-e57006baacd7 admin admin] PATH_INFO: `/v3/users/2dce40b8d74c41ec94cc36e6ff05d412` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:15:53.385132 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1396d2a8-57a8-4408-93bd-e57006baacd7 admin admin] RBAC: Authorizing `identity:get_user(user_id=2dce40b8d74c41ec94cc36e6ff05d412)` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:15:53.386959 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1396d2a8-57a8-4408-93bd-e57006baacd7 admin admin] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:15:53.388575 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 5/9] 10.222.0.47 () {56 vars in 1198 bytes} [Thu Jul 21 08:15:53 2022] GET /identity/v3/users/2dce40b8d74c41ec94cc36e6ff05d412 => generated 247 bytes in 21 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 08:15:55.484979 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-831c05ed-765f-49b2-90c4-21dad889b0e4 None None] REQUEST_METHOD: `GET` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:15:55.485536 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-831c05ed-765f-49b2-90c4-21dad889b0e4 None None] SCRIPT_NAME: `/identity` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:15:55.485536 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-831c05ed-765f-49b2-90c4-21dad889b0e4 None None] PATH_INFO: `/` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:15:55.486899 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: [pid: 80245|app: 0|req: 5/10] 10.222.0.47 () {54 vars in 926 bytes} [Thu Jul 21 08:15:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 08:15:55.492105 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41a73e6c-84d0-4c46-9f75-15d2859542c5 None None] REQUEST_METHOD: `POST` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:15:55.492504 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41a73e6c-84d0-4c46-9f75-15d2859542c5 None None] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:15:55.492671 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41a73e6c-84d0-4c46-9f75-15d2859542c5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:15:55.509429 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d15ae8c3-9a14-4f78-b4d4-becf959bb3ec None None] REQUEST_METHOD: `GET` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:15:55.509717 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d15ae8c3-9a14-4f78-b4d4-becf959bb3ec None None] SCRIPT_NAME: `/identity` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:15:55.509999 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d15ae8c3-9a14-4f78-b4d4-becf959bb3ec None None] PATH_INFO: `/` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:15:55.510243 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.auth.core [None req-41a73e6c-84d0-4c46-9f75-15d2859542c5 None None] MFA Rules not processed for user `2dce40b8d74c41ec94cc36e6ff05d412`. Rule list: `[]` (Enabled: `True`). {{(pid=80244) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 08:15:55.511368 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: [pid: 80245|app: 0|req: 6/11] 10.222.0.47 () {54 vars in 926 bytes} [Thu Jul 21 08:15:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 21 08:15:55.511468 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.fernet_utils [None req-41a73e6c-84d0-4c46-9f75-15d2859542c5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80244) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 08:15:55.516671 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ede5cba-ab7f-4099-bbed-694496204a4d None None] REQUEST_METHOD: `POST` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:15:55.516968 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ede5cba-ab7f-4099-bbed-694496204a4d None None] SCRIPT_NAME: `/identity` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:15:55.517233 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ede5cba-ab7f-4099-bbed-694496204a4d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:15:55.557131 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG passlib.handlers.bcrypt [None req-0ede5cba-ab7f-4099-bbed-694496204a4d None None] detected 'bcrypt' backend, version '3.2.2' {{(pid=80245) _load_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:625}} Jul 21 08:15:55.557547 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG passlib.handlers.bcrypt [None req-0ede5cba-ab7f-4099-bbed-694496204a4d None None] 'bcrypt' backend lacks $2$ support, enabling workaround {{(pid=80245) _finalize_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:406}} Jul 21 08:15:55.594121 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 6/12] 10.222.0.47 () {58 vars in 1023 bytes} [Thu Jul 21 08:15:55 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 21 08:15:55.603636 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.auth.core [None req-0ede5cba-ab7f-4099-bbed-694496204a4d None None] MFA Rules not processed for user `2dce40b8d74c41ec94cc36e6ff05d412`. Rule list: `[]` (Enabled: `True`). {{(pid=80245) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 08:15:55.605269 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.fernet_utils [None req-0ede5cba-ab7f-4099-bbed-694496204a4d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80245) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 08:15:55.616807 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41a73e6c-84d0-4c46-9f75-15d2859542c5 None None] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:15:55.641782 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG dbcounter [-] [80244] Writing DB stats keystone:SELECT=100 {{(pid=80244) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:15:55.707154 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: [pid: 80245|app: 0|req: 7/13] 10.222.0.47 () {58 vars in 1023 bytes} [Thu Jul 21 08:15:55 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 191 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 21 08:15:55.722599 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41a73e6c-84d0-4c46-9f75-15d2859542c5 None None] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:15:55.729825 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ede5cba-ab7f-4099-bbed-694496204a4d None None] Authenticating user token {{(pid=80245) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:15:55.755593 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6cac1ab-4deb-4b0e-ba54-cd749a04710e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:15:55.756397 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6cac1ab-4deb-4b0e-ba54-cd749a04710e None admin] REQUEST_METHOD: `GET` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:15:55.756604 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6cac1ab-4deb-4b0e-ba54-cd749a04710e None admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:15:55.756882 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6cac1ab-4deb-4b0e-ba54-cd749a04710e None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:15:55.768704 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6cac1ab-4deb-4b0e-ba54-cd749a04710e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 21 08:15:55.780496 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application [None req-f6cac1ab-4deb-4b0e-ba54-cd749a04710e None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 21 08:15:55.780496 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 08:15:55.780496 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 08:15:55.780496 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 08:15:55.780496 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 08:15:55.780496 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 08:15:55.780496 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 08:15:55.780496 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 08:15:55.780496 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 08:15:55.780496 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 08:15:55.780496 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 08:15:55.780496 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 08:15:55.780496 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 21 08:15:55.780496 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 21 08:15:55.780496 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 21 08:15:55.780496 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 21 08:15:55.781400 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 21 08:15:55.781400 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 21 08:15:55.781400 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 21 08:15:55.781400 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 21 08:15:55.781400 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 08:15:55.781400 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 08:15:55.781400 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 08:15:55.781400 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 08:15:55.781400 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 08:15:55.781400 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 08:15:55.781400 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 08:15:55.781400 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application with Lock( Jul 21 08:15:55.781400 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 08:15:55.781400 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self._enter() Jul 21 08:15:55.781400 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 08:15:55.781400 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 08:15:55.782202 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 08:15:55.782202 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self.creator() Jul 21 08:15:55.782202 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 08:15:55.782202 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application created_value = creator( Jul 21 08:15:55.782202 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 21 08:15:55.782202 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 21 08:15:55.782202 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 21 08:15:55.782202 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 21 08:15:55.782202 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 21 08:15:55.782202 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 21 08:15:55.782202 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 21 08:15:55.782202 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application Jul 21 08:15:55.782202 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 7/14] 10.222.0.47 () {56 vars in 1096 bytes} [Thu Jul 21 08:15:55 2022] GET /identity/v3/roles/admin => generated 83 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 21 08:15:55.788923 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6cac1ab-4deb-4b0e-ba54-cd749a04710e None admin] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:15:55.833931 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ede5cba-ab7f-4099-bbed-694496204a4d None None] Validating token access rules against request {{(pid=80245) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:15:55.836836 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG dbcounter [-] [80245] Writing DB stats keystone:SELECT=100 {{(pid=80245) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:15:55.871605 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG dbcounter [-] [80244] Writing DB stats keystone:SELECT=100 {{(pid=80244) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:15:55.871903 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2109ab4a-0615-41fa-852f-a40c9f37fb8a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80245) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:15:55.872813 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2109ab4a-0615-41fa-852f-a40c9f37fb8a None admin] REQUEST_METHOD: `GET` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:15:55.873100 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2109ab4a-0615-41fa-852f-a40c9f37fb8a None admin] SCRIPT_NAME: `/identity` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:15:55.873373 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2109ab4a-0615-41fa-852f-a40c9f37fb8a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:15:55.897087 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6cac1ab-4deb-4b0e-ba54-cd749a04710e None admin] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:15:55.934713 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5c2730f-e0b7-44f7-9bdb-ed114ab38d43 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:15:55.935425 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5c2730f-e0b7-44f7-9bdb-ed114ab38d43 None admin] REQUEST_METHOD: `GET` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:15:55.935718 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5c2730f-e0b7-44f7-9bdb-ed114ab38d43 None admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:15:55.936002 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5c2730f-e0b7-44f7-9bdb-ed114ab38d43 None admin] PATH_INFO: `/v3/roles` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:15:55.936766 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5c2730f-e0b7-44f7-9bdb-ed114ab38d43 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80244) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 08:15:55.937076 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5c2730f-e0b7-44f7-9bdb-ed114ab38d43 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:15:55.938490 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5c2730f-e0b7-44f7-9bdb-ed114ab38d43 None admin] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:15:55.947864 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2109ab4a-0615-41fa-852f-a40c9f37fb8a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:15:55.949417 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2109ab4a-0615-41fa-852f-a40c9f37fb8a None admin] RBAC: Authorization granted {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:15:55.950968 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 8/15] 10.222.0.47 () {56 vars in 1099 bytes} [Thu Jul 21 08:15:55 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 21 08:15:55.956624 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5c2730f-e0b7-44f7-9bdb-ed114ab38d43 None admin] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:15:56.047205 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: [pid: 80245|app: 0|req: 8/16] 10.222.0.47 () {58 vars in 1282 bytes} [Thu Jul 21 08:15:55 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 318 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Jul 21 08:15:56.054612 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2109ab4a-0615-41fa-852f-a40c9f37fb8a None admin] Authenticating user token {{(pid=80245) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:15:56.056492 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5c2730f-e0b7-44f7-9bdb-ed114ab38d43 None admin] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:15:56.057883 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG dbcounter [-] [80245] Writing DB stats keystone:SELECT=100 {{(pid=80245) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:15:56.091239 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4274841b-9b81-4ff4-942e-95b84dcd4fd1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:15:56.092148 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4274841b-9b81-4ff4-942e-95b84dcd4fd1 None admin] REQUEST_METHOD: `GET` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:15:56.092444 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4274841b-9b81-4ff4-942e-95b84dcd4fd1 None admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:15:56.092731 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4274841b-9b81-4ff4-942e-95b84dcd4fd1 None admin] PATH_INFO: `/v3/users/2dce40b8d74c41ec94cc36e6ff05d412` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:15:56.093943 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4274841b-9b81-4ff4-942e-95b84dcd4fd1 None admin] RBAC: Authorizing `identity:get_user(user_id=2dce40b8d74c41ec94cc36e6ff05d412)` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:15:56.095444 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4274841b-9b81-4ff4-942e-95b84dcd4fd1 None admin] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:15:56.097073 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 9/17] 10.222.0.47 () {56 vars in 1177 bytes} [Thu Jul 21 08:15:55 2022] GET /identity/v3/users/2dce40b8d74c41ec94cc36e6ff05d412 => generated 247 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 21 08:15:56.102576 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4274841b-9b81-4ff4-942e-95b84dcd4fd1 None admin] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:15:56.118047 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG dbcounter [-] [80244] Writing DB stats keystone:SELECT=100 {{(pid=80244) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:15:56.152699 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2109ab4a-0615-41fa-852f-a40c9f37fb8a None admin] Validating token access rules against request {{(pid=80245) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:15:56.186692 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c5efbda-38fc-44b9-9c6c-b48d76caf976 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80245) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:15:56.187540 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c5efbda-38fc-44b9-9c6c-b48d76caf976 None admin] REQUEST_METHOD: `GET` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:15:56.187829 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c5efbda-38fc-44b9-9c6c-b48d76caf976 None admin] SCRIPT_NAME: `/identity` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:15:56.188106 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c5efbda-38fc-44b9-9c6c-b48d76caf976 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:15:56.203819 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4274841b-9b81-4ff4-942e-95b84dcd4fd1 None admin] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:15:56.207477 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c5efbda-38fc-44b9-9c6c-b48d76caf976 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:15:56.208961 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c5efbda-38fc-44b9-9c6c-b48d76caf976 None admin] RBAC: Authorization granted {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:15:56.231989 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application [None req-0c5efbda-38fc-44b9-9c6c-b48d76caf976 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 08:15:56.231989 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 08:15:56.231989 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 08:15:56.231989 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 08:15:56.231989 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 08:15:56.231989 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 08:15:56.231989 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 08:15:56.231989 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 08:15:56.231989 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 08:15:56.231989 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 08:15:56.231989 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 08:15:56.231989 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 08:15:56.231989 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 21 08:15:56.231989 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 21 08:15:56.231989 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 21 08:15:56.232997 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 21 08:15:56.232997 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 08:15:56.232997 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 08:15:56.232997 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 08:15:56.232997 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 08:15:56.232997 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 08:15:56.232997 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 08:15:56.232997 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 08:15:56.232997 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application with Lock( Jul 21 08:15:56.232997 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 08:15:56.232997 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application return self._enter() Jul 21 08:15:56.232997 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 08:15:56.232997 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 08:15:56.232997 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 08:15:56.232997 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application return self.creator() Jul 21 08:15:56.232997 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 08:15:56.232997 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application created_value = creator( Jul 21 08:15:56.234076 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 21 08:15:56.234076 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 21 08:15:56.234076 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 08:15:56.234076 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application Jul 21 08:15:56.234076 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: [pid: 80245|app: 0|req: 9/18] 10.222.0.47 () {56 vars in 1108 bytes} [Thu Jul 21 08:15:56 2022] GET /identity/v3/domains/Default => generated 87 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 08:15:56.238330 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60dab689-6162-4132-81cd-238ba50e23f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:15:56.238884 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c5efbda-38fc-44b9-9c6c-b48d76caf976 None admin] Authenticating user token {{(pid=80245) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:15:56.239133 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60dab689-6162-4132-81cd-238ba50e23f1 None admin] REQUEST_METHOD: `GET` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:15:56.239456 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60dab689-6162-4132-81cd-238ba50e23f1 None admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:15:56.239739 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60dab689-6162-4132-81cd-238ba50e23f1 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:15:56.240927 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60dab689-6162-4132-81cd-238ba50e23f1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:15:56.242945 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60dab689-6162-4132-81cd-238ba50e23f1 None admin] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:15:56.244594 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 10/19] 10.222.0.47 () {56 vars in 1108 bytes} [Thu Jul 21 08:15:56 2022] GET /identity/v3/domains/default => generated 199 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 21 08:15:56.250189 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60dab689-6162-4132-81cd-238ba50e23f1 None admin] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:15:56.318556 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG dbcounter [-] [80245] Writing DB stats keystone:SELECT=100 {{(pid=80245) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:15:56.345183 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG dbcounter [-] [80244] Writing DB stats keystone:SELECT=100 {{(pid=80244) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:15:56.361714 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c5efbda-38fc-44b9-9c6c-b48d76caf976 None admin] Validating token access rules against request {{(pid=80245) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:15:56.364941 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60dab689-6162-4132-81cd-238ba50e23f1 None admin] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:15:56.396401 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b65250f8-84fb-4b50-adf0-a6b9afdb6963 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80245) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:15:56.397071 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b65250f8-84fb-4b50-adf0-a6b9afdb6963 None admin] REQUEST_METHOD: `GET` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:15:56.397354 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b65250f8-84fb-4b50-adf0-a6b9afdb6963 None admin] SCRIPT_NAME: `/identity` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:15:56.397634 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b65250f8-84fb-4b50-adf0-a6b9afdb6963 None admin] PATH_INFO: `/v3/domains` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:15:56.398363 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b65250f8-84fb-4b50-adf0-a6b9afdb6963 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80245) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 08:15:56.398560 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1c60ede-92cc-4385-9865-8fd5c0d9cec0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:15:56.398671 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b65250f8-84fb-4b50-adf0-a6b9afdb6963 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:15:56.399217 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1c60ede-92cc-4385-9865-8fd5c0d9cec0 None admin] REQUEST_METHOD: `GET` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:15:56.399528 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1c60ede-92cc-4385-9865-8fd5c0d9cec0 None admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:15:56.399805 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1c60ede-92cc-4385-9865-8fd5c0d9cec0 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:15:56.400084 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b65250f8-84fb-4b50-adf0-a6b9afdb6963 None admin] RBAC: Authorization granted {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:15:56.400638 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1c60ede-92cc-4385-9865-8fd5c0d9cec0 None admin] RBAC: Adding query filter params (role.id=e29870f6e9e2411a9db0d5bf45f32ca5, scope.domain.id=default, user.id=2dce40b8d74c41ec94cc36e6ff05d412) {{(pid=80244) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 08:15:56.400946 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1c60ede-92cc-4385-9865-8fd5c0d9cec0 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:15:56.402417 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1c60ede-92cc-4385-9865-8fd5c0d9cec0 None admin] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:15:56.409087 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 11/20] 10.222.0.47 () {56 vars in 1322 bytes} [Thu Jul 21 08:15:56 2022] GET /identity/v3/role_assignments?user.id=2dce40b8d74c41ec94cc36e6ff05d412&scope.domain.id=default&role.id=e29870f6e9e2411a9db0d5bf45f32ca5 => generated 233 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 21 08:15:56.418305 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: [pid: 80245|app: 0|req: 10/21] 10.222.0.47 () {56 vars in 1109 bytes} [Thu Jul 21 08:15:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 21 08:15:58.725057 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b886b69-9435-4207-af4e-a2ab6281c654 None None] REQUEST_METHOD: `GET` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:15:58.725587 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b886b69-9435-4207-af4e-a2ab6281c654 None None] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:15:58.725587 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b886b69-9435-4207-af4e-a2ab6281c654 None None] PATH_INFO: `/` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:15:58.727224 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 12/22] 10.222.0.47 () {54 vars in 926 bytes} [Thu Jul 21 08:15:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 08:15:58.732856 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce52dc12-2685-410b-9bf4-a2c140014198 None None] REQUEST_METHOD: `POST` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:15:58.733677 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce52dc12-2685-410b-9bf4-a2c140014198 None None] SCRIPT_NAME: `/identity` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:15:58.734104 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce52dc12-2685-410b-9bf4-a2c140014198 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:15:58.760516 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.auth.core [None req-ce52dc12-2685-410b-9bf4-a2c140014198 None None] MFA Rules not processed for user `2dce40b8d74c41ec94cc36e6ff05d412`. Rule list: `[]` (Enabled: `True`). {{(pid=80245) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 08:15:58.761979 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.fernet_utils [None req-ce52dc12-2685-410b-9bf4-a2c140014198 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80245) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 08:15:58.851588 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: [pid: 80245|app: 0|req: 11/23] 10.222.0.47 () {58 vars in 1023 bytes} [Thu Jul 21 08:15:58 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 21 08:15:58.890508 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b886b69-9435-4207-af4e-a2ab6281c654 None None] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:15:59.023243 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2325c6fe-d56c-4705-84a2-a1445a74c666 None None] REQUEST_METHOD: `GET` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:15:59.023755 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2325c6fe-d56c-4705-84a2-a1445a74c666 None None] SCRIPT_NAME: `/identity` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:15:59.024203 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2325c6fe-d56c-4705-84a2-a1445a74c666 None None] PATH_INFO: `/` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:15:59.026548 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: [pid: 80245|app: 0|req: 12/24] 10.222.0.47 () {54 vars in 926 bytes} [Thu Jul 21 08:15:59 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 21 08:15:59.031979 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad5dc2ce-8c75-4d9a-bd04-6586eabcc84d None None] REQUEST_METHOD: `POST` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:15:59.032444 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad5dc2ce-8c75-4d9a-bd04-6586eabcc84d None None] SCRIPT_NAME: `/identity` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:15:59.032869 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad5dc2ce-8c75-4d9a-bd04-6586eabcc84d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:15:59.051839 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b886b69-9435-4207-af4e-a2ab6281c654 None None] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:15:59.055761 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.auth.core [None req-ad5dc2ce-8c75-4d9a-bd04-6586eabcc84d None None] MFA Rules not processed for user `2dce40b8d74c41ec94cc36e6ff05d412`. Rule list: `[]` (Enabled: `True`). {{(pid=80245) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 08:15:59.057165 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.fernet_utils [None req-ad5dc2ce-8c75-4d9a-bd04-6586eabcc84d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80245) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 08:15:59.108575 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf27e12d-fc95-451c-88b5-ef14a6f408fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:15:59.109381 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf27e12d-fc95-451c-88b5-ef14a6f408fe None admin] REQUEST_METHOD: `GET` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:15:59.109669 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf27e12d-fc95-451c-88b5-ef14a6f408fe None admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:15:59.109950 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf27e12d-fc95-451c-88b5-ef14a6f408fe None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:15:59.123991 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG dbcounter [-] [80245] Writing DB stats keystone:SELECT=100 {{(pid=80245) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:15:59.140449 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf27e12d-fc95-451c-88b5-ef14a6f408fe None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:15:59.141969 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf27e12d-fc95-451c-88b5-ef14a6f408fe None admin] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:15:59.148828 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG dbcounter [-] [80244] Writing DB stats keystone:SELECT=100 {{(pid=80244) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:15:59.175878 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application [None req-bf27e12d-fc95-451c-88b5-ef14a6f408fe None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 08:15:59.175878 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 08:15:59.175878 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 08:15:59.175878 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 08:15:59.175878 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 08:15:59.175878 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 08:15:59.175878 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 08:15:59.175878 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 08:15:59.175878 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 08:15:59.175878 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 08:15:59.175878 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 08:15:59.175878 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 08:15:59.175878 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 21 08:15:59.175878 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 21 08:15:59.175878 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 21 08:15:59.176761 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 21 08:15:59.176761 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 08:15:59.176761 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 08:15:59.176761 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 08:15:59.176761 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 08:15:59.176761 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 08:15:59.176761 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 08:15:59.176761 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 08:15:59.176761 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application with Lock( Jul 21 08:15:59.176761 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 08:15:59.176761 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self._enter() Jul 21 08:15:59.176761 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 08:15:59.176761 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 08:15:59.176761 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 08:15:59.176761 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self.creator() Jul 21 08:15:59.176761 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 08:15:59.176761 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application created_value = creator( Jul 21 08:15:59.177542 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 21 08:15:59.177542 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 21 08:15:59.177542 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 08:15:59.177542 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application Jul 21 08:15:59.177542 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 13/25] 10.222.0.47 () {56 vars in 1108 bytes} [Thu Jul 21 08:15:58 2022] GET /identity/v3/domains/Default => generated 87 bytes in 287 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 08:15:59.181365 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: [pid: 80245|app: 0|req: 13/26] 10.222.0.47 () {58 vars in 1023 bytes} [Thu Jul 21 08:15:59 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 150 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 21 08:15:59.183945 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf27e12d-fc95-451c-88b5-ef14a6f408fe None admin] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:15:59.227871 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad5dc2ce-8c75-4d9a-bd04-6586eabcc84d None None] Authenticating user token {{(pid=80245) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:15:59.358139 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf27e12d-fc95-451c-88b5-ef14a6f408fe None admin] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:15:59.402840 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d03a93c5-8e44-493e-b6a1-96ba4c7bd348 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:15:59.403574 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d03a93c5-8e44-493e-b6a1-96ba4c7bd348 None admin] REQUEST_METHOD: `GET` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:15:59.403857 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d03a93c5-8e44-493e-b6a1-96ba4c7bd348 None admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:15:59.404132 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d03a93c5-8e44-493e-b6a1-96ba4c7bd348 None admin] PATH_INFO: `/v3/domains` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:15:59.404903 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d03a93c5-8e44-493e-b6a1-96ba4c7bd348 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80244) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 08:15:59.405212 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d03a93c5-8e44-493e-b6a1-96ba4c7bd348 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:15:59.406668 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d03a93c5-8e44-493e-b6a1-96ba4c7bd348 None admin] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:15:59.437549 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 14/27] 10.222.0.47 () {56 vars in 1109 bytes} [Thu Jul 21 08:15:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 254 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 08:15:59.439638 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ae47a66-69aa-42e4-b4b8-20714ac5eb58 None None] REQUEST_METHOD: `GET` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:15:59.440144 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ae47a66-69aa-42e4-b4b8-20714ac5eb58 None None] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:15:59.440565 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ae47a66-69aa-42e4-b4b8-20714ac5eb58 None None] PATH_INFO: `/` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:15:59.442287 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 15/28] 10.222.0.47 () {54 vars in 926 bytes} [Thu Jul 21 08:15:59 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 08:15:59.443846 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d51e4cff-87af-4348-9fbc-1035cbd37309 None None] REQUEST_METHOD: `GET` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:15:59.444290 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d51e4cff-87af-4348-9fbc-1035cbd37309 None None] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:15:59.444924 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d51e4cff-87af-4348-9fbc-1035cbd37309 None None] PATH_INFO: `/` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:15:59.446788 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 16/29] 10.222.0.47 () {54 vars in 926 bytes} [Thu Jul 21 08:15:59 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 08:15:59.447814 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d51e4cff-87af-4348-9fbc-1035cbd37309 None None] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:15:59.450180 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad5dc2ce-8c75-4d9a-bd04-6586eabcc84d None None] Validating token access rules against request {{(pid=80245) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:15:59.490110 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6662f300-f36f-4ecd-8d7f-a8a4261bc995 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80245) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:15:59.491187 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6662f300-f36f-4ecd-8d7f-a8a4261bc995 None admin] REQUEST_METHOD: `POST` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:15:59.492209 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6662f300-f36f-4ecd-8d7f-a8a4261bc995 None admin] SCRIPT_NAME: `/identity` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:15:59.492663 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6662f300-f36f-4ecd-8d7f-a8a4261bc995 None admin] PATH_INFO: `/v3/roles` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:15:59.493584 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6662f300-f36f-4ecd-8d7f-a8a4261bc995 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:15:59.495314 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6662f300-f36f-4ecd-8d7f-a8a4261bc995 None admin] RBAC: Authorization granted {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:15:59.523759 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG dbcounter [-] [80244] Writing DB stats keystone:SELECT=100 {{(pid=80244) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:15:59.572000 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d51e4cff-87af-4348-9fbc-1035cbd37309 None None] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:15:59.607964 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ac00180-2747-48df-9570-a3afae126c9f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:15:59.608793 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ac00180-2747-48df-9570-a3afae126c9f None admin] REQUEST_METHOD: `POST` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:15:59.609108 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ac00180-2747-48df-9570-a3afae126c9f None admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:15:59.609384 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ac00180-2747-48df-9570-a3afae126c9f None admin] PATH_INFO: `/v3/projects` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:15:59.610318 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ac00180-2747-48df-9570-a3afae126c9f None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:15:59.611931 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ac00180-2747-48df-9570-a3afae126c9f None admin] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:15:59.639103 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: [pid: 80245|app: 0|req: 14/30] 10.222.0.47 () {60 vars in 1131 bytes} [Thu Jul 21 08:15:59 2022] POST /identity/v3/roles => generated 219 bytes in 412 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 21 08:15:59.641117 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc17641c-2272-4372-a4cb-635f8dc7db42 None None] REQUEST_METHOD: `POST` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:15:59.641914 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc17641c-2272-4372-a4cb-635f8dc7db42 None None] SCRIPT_NAME: `/identity` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:15:59.642351 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc17641c-2272-4372-a4cb-635f8dc7db42 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:15:59.665389 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.auth.core [None req-cc17641c-2272-4372-a4cb-635f8dc7db42 None None] MFA Rules not processed for user `2dce40b8d74c41ec94cc36e6ff05d412`. Rule list: `[]` (Enabled: `True`). {{(pid=80245) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 08:15:59.667286 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.fernet_utils [None req-cc17641c-2272-4372-a4cb-635f8dc7db42 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80245) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 08:15:59.701461 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG dbcounter [-] [80245] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80245) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:15:59.747228 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: [pid: 80245|app: 0|req: 15/31] 10.222.0.47 () {58 vars in 1023 bytes} [Thu Jul 21 08:15:59 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 21 08:15:59.749347 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61dd26de-06f8-4f9c-b405-c6d053183dae None None] REQUEST_METHOD: `POST` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:15:59.749789 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61dd26de-06f8-4f9c-b405-c6d053183dae None None] SCRIPT_NAME: `/identity` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:15:59.750242 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61dd26de-06f8-4f9c-b405-c6d053183dae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:15:59.767671 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 17/32] 10.222.0.47 () {60 vars in 1141 bytes} [Thu Jul 21 08:15:59 2022] POST /identity/v3/projects => generated 301 bytes in 316 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 08:15:59.768238 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.auth.core [None req-61dd26de-06f8-4f9c-b405-c6d053183dae None None] MFA Rules not processed for user `2dce40b8d74c41ec94cc36e6ff05d412`. Rule list: `[]` (Enabled: `True`). {{(pid=80245) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 08:15:59.769779 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.fernet_utils [None req-61dd26de-06f8-4f9c-b405-c6d053183dae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80245) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 08:15:59.802188 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ac00180-2747-48df-9570-a3afae126c9f None admin] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:15:59.850241 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: [pid: 80245|app: 0|req: 16/33] 10.222.0.47 () {58 vars in 1023 bytes} [Thu Jul 21 08:15:59 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 21 08:15:59.898822 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61dd26de-06f8-4f9c-b405-c6d053183dae None None] Authenticating user token {{(pid=80245) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:16:00.009951 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ac00180-2747-48df-9570-a3afae126c9f None admin] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:16:00.023925 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG dbcounter [-] [80244] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80244) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:16:00.057466 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG dbcounter [-] [80245] Writing DB stats keystone:SELECT=100 {{(pid=80245) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:16:00.062813 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfa86496-a05e-4d45-a51d-f934c5b49d40 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:16:00.063659 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfa86496-a05e-4d45-a51d-f934c5b49d40 None admin] REQUEST_METHOD: `GET` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:16:00.063946 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfa86496-a05e-4d45-a51d-f934c5b49d40 None admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:16:00.064222 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfa86496-a05e-4d45-a51d-f934c5b49d40 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:16:00.065256 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61dd26de-06f8-4f9c-b405-c6d053183dae None None] Validating token access rules against request {{(pid=80245) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:16:00.076121 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfa86496-a05e-4d45-a51d-f934c5b49d40 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 21 08:16:00.084696 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application [None req-bfa86496-a05e-4d45-a51d-f934c5b49d40 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 21 08:16:00.084696 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 08:16:00.084696 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 08:16:00.084696 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 08:16:00.084696 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 08:16:00.084696 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 08:16:00.084696 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 08:16:00.084696 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 08:16:00.084696 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 08:16:00.084696 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 08:16:00.084696 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 08:16:00.084696 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 08:16:00.084696 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 21 08:16:00.084696 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 21 08:16:00.084696 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 21 08:16:00.084696 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 21 08:16:00.085750 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 21 08:16:00.085750 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 21 08:16:00.085750 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 21 08:16:00.085750 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 21 08:16:00.085750 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 08:16:00.085750 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 08:16:00.085750 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 08:16:00.085750 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 08:16:00.085750 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 08:16:00.085750 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 08:16:00.085750 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 08:16:00.085750 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application with Lock( Jul 21 08:16:00.085750 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 08:16:00.085750 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self._enter() Jul 21 08:16:00.085750 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 08:16:00.085750 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 08:16:00.086673 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 08:16:00.086673 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self.creator() Jul 21 08:16:00.086673 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 08:16:00.086673 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application created_value = creator( Jul 21 08:16:00.086673 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 21 08:16:00.086673 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 21 08:16:00.086673 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 21 08:16:00.086673 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 21 08:16:00.086673 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 21 08:16:00.086673 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 21 08:16:00.086673 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 21 08:16:00.086673 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application Jul 21 08:16:00.086673 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 18/34] 10.222.0.47 () {56 vars in 1096 bytes} [Thu Jul 21 08:15:59 2022] GET /identity/v3/roles/admin => generated 83 bytes in 285 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 21 08:16:00.098583 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfa86496-a05e-4d45-a51d-f934c5b49d40 None admin] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:16:00.120920 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ca0dc1a-af80-4ed0-87fa-e3f2f1f6433f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80245) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:16:00.122321 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ca0dc1a-af80-4ed0-87fa-e3f2f1f6433f None admin] REQUEST_METHOD: `POST` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:16:00.122830 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ca0dc1a-af80-4ed0-87fa-e3f2f1f6433f None admin] SCRIPT_NAME: `/identity` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:16:00.123355 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ca0dc1a-af80-4ed0-87fa-e3f2f1f6433f None admin] PATH_INFO: `/v3/roles` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:16:00.124469 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ca0dc1a-af80-4ed0-87fa-e3f2f1f6433f None admin] RBAC: Authorizing `identity:create_role()` {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:16:00.126645 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ca0dc1a-af80-4ed0-87fa-e3f2f1f6433f None admin] RBAC: Authorization granted {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:16:00.147483 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: [pid: 80245|app: 0|req: 17/35] 10.222.0.47 () {60 vars in 1131 bytes} [Thu Jul 21 08:15:59 2022] POST /identity/v3/roles => generated 225 bytes in 250 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 21 08:16:00.332494 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74a041f9-1dad-4bd9-8825-8469ebc90a99 None None] REQUEST_METHOD: `GET` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:16:00.333398 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74a041f9-1dad-4bd9-8825-8469ebc90a99 None None] SCRIPT_NAME: `/identity` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:16:00.333915 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74a041f9-1dad-4bd9-8825-8469ebc90a99 None None] PATH_INFO: `/` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:16:00.336427 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: [pid: 80245|app: 0|req: 18/36] 10.222.0.47 () {54 vars in 926 bytes} [Thu Jul 21 08:16:00 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 08:16:00.343898 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a45c907a-a00e-4a10-954c-8acecbc46aeb None None] REQUEST_METHOD: `POST` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:16:00.344237 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a45c907a-a00e-4a10-954c-8acecbc46aeb None None] SCRIPT_NAME: `/identity` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:16:00.344477 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a45c907a-a00e-4a10-954c-8acecbc46aeb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:16:00.352389 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfa86496-a05e-4d45-a51d-f934c5b49d40 None admin] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:16:00.363399 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.auth.core [None req-a45c907a-a00e-4a10-954c-8acecbc46aeb None None] MFA Rules not processed for user `2dce40b8d74c41ec94cc36e6ff05d412`. Rule list: `[]` (Enabled: `True`). {{(pid=80245) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 08:16:00.364635 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.fernet_utils [None req-a45c907a-a00e-4a10-954c-8acecbc46aeb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80245) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 08:16:00.395208 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-245eb30e-93d6-4aa0-bdcc-9b151a137159 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:16:00.396348 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-245eb30e-93d6-4aa0-bdcc-9b151a137159 None admin] REQUEST_METHOD: `GET` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:16:00.396881 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-245eb30e-93d6-4aa0-bdcc-9b151a137159 None admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:16:00.397396 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-245eb30e-93d6-4aa0-bdcc-9b151a137159 None admin] PATH_INFO: `/v3/roles` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:16:00.398208 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-245eb30e-93d6-4aa0-bdcc-9b151a137159 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80244) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 08:16:00.398534 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-245eb30e-93d6-4aa0-bdcc-9b151a137159 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:16:00.400300 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-245eb30e-93d6-4aa0-bdcc-9b151a137159 None admin] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:16:00.411183 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 19/37] 10.222.0.47 () {56 vars in 1099 bytes} [Thu Jul 21 08:16:00 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 313 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 21 08:16:00.418829 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-245eb30e-93d6-4aa0-bdcc-9b151a137159 None admin] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:16:00.444359 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: [pid: 80245|app: 0|req: 19/38] 10.222.0.47 () {58 vars in 1023 bytes} [Thu Jul 21 08:16:00 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 21 08:16:00.455579 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG dbcounter [-] [80244] Writing DB stats keystone:SELECT=100 {{(pid=80244) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:16:00.467604 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a45c907a-a00e-4a10-954c-8acecbc46aeb None None] Authenticating user token {{(pid=80245) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:16:00.541023 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-245eb30e-93d6-4aa0-bdcc-9b151a137159 None admin] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:16:00.562043 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG dbcounter [-] [80245] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80245) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:16:00.579593 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ee34df2-40a0-4671-9090-4bef2c39884b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:16:00.580479 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ee34df2-40a0-4671-9090-4bef2c39884b None admin] REQUEST_METHOD: `GET` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:16:00.580763 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ee34df2-40a0-4671-9090-4bef2c39884b None admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:16:00.581039 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ee34df2-40a0-4671-9090-4bef2c39884b None admin] PATH_INFO: `/v3/users/2dce40b8d74c41ec94cc36e6ff05d412` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:16:00.581170 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a45c907a-a00e-4a10-954c-8acecbc46aeb None None] Validating token access rules against request {{(pid=80245) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:16:00.582353 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ee34df2-40a0-4671-9090-4bef2c39884b None admin] RBAC: Authorizing `identity:get_user(user_id=2dce40b8d74c41ec94cc36e6ff05d412)` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:16:00.584456 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ee34df2-40a0-4671-9090-4bef2c39884b None admin] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:16:00.585763 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 20/39] 10.222.0.47 () {56 vars in 1177 bytes} [Thu Jul 21 08:16:00 2022] GET /identity/v3/users/2dce40b8d74c41ec94cc36e6ff05d412 => generated 247 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 21 08:16:00.591016 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ee34df2-40a0-4671-9090-4bef2c39884b None admin] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:16:00.618469 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8f77a22-f625-47f4-ab91-ec66292a9eb3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80245) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:16:00.619276 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8f77a22-f625-47f4-ab91-ec66292a9eb3 None admin] REQUEST_METHOD: `GET` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:16:00.619604 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8f77a22-f625-47f4-ab91-ec66292a9eb3 None admin] SCRIPT_NAME: `/identity` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:16:00.619882 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8f77a22-f625-47f4-ab91-ec66292a9eb3 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:16:00.621321 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8f77a22-f625-47f4-ab91-ec66292a9eb3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:16:00.622790 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8f77a22-f625-47f4-ab91-ec66292a9eb3 None admin] RBAC: Authorization granted {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:16:00.628968 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: [pid: 80245|app: 0|req: 20/40] 10.222.0.47 () {56 vars in 1108 bytes} [Thu Jul 21 08:16:00 2022] GET /identity/v3/domains/default => generated 199 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 21 08:16:00.630139 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8f77a22-f625-47f4-ab91-ec66292a9eb3 None admin] Authenticating user token {{(pid=80245) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:16:00.703027 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG dbcounter [-] [80244] Writing DB stats keystone:SELECT=100 {{(pid=80244) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:16:00.707583 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ee34df2-40a0-4671-9090-4bef2c39884b None admin] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:16:00.742757 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8f77a22-f625-47f4-ab91-ec66292a9eb3 None admin] Validating token access rules against request {{(pid=80245) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:16:00.744285 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3fd62b2-57f0-49d8-8d4e-48915f6fe9f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:16:00.745049 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3fd62b2-57f0-49d8-8d4e-48915f6fe9f8 None admin] REQUEST_METHOD: `GET` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:16:00.745341 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3fd62b2-57f0-49d8-8d4e-48915f6fe9f8 None admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:16:00.745622 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3fd62b2-57f0-49d8-8d4e-48915f6fe9f8 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:16:00.746921 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3fd62b2-57f0-49d8-8d4e-48915f6fe9f8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:16:00.748430 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3fd62b2-57f0-49d8-8d4e-48915f6fe9f8 None admin] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:16:00.750080 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 21/41] 10.222.0.47 () {56 vars in 1108 bytes} [Thu Jul 21 08:16:00 2022] GET /identity/v3/domains/default => generated 199 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 21 08:16:00.751416 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4298717e-a2fc-4030-b8a8-812827b00c97 None None] REQUEST_METHOD: `GET` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:16:00.751710 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4298717e-a2fc-4030-b8a8-812827b00c97 None None] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:16:00.751995 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4298717e-a2fc-4030-b8a8-812827b00c97 None None] PATH_INFO: `/` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:16:00.753252 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 22/42] 10.222.0.47 () {54 vars in 926 bytes} [Thu Jul 21 08:16:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 08:16:00.756216 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4298717e-a2fc-4030-b8a8-812827b00c97 None None] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:16:00.783191 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02439bb4-7a37-4488-861e-cfeda752f7c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80245) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:16:00.783907 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02439bb4-7a37-4488-861e-cfeda752f7c2 None admin] REQUEST_METHOD: `POST` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:16:00.784189 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02439bb4-7a37-4488-861e-cfeda752f7c2 None admin] SCRIPT_NAME: `/identity` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:16:00.784465 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02439bb4-7a37-4488-861e-cfeda752f7c2 None admin] PATH_INFO: `/v3/projects` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:16:00.785185 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02439bb4-7a37-4488-861e-cfeda752f7c2 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:16:00.786640 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02439bb4-7a37-4488-861e-cfeda752f7c2 None admin] RBAC: Authorization granted {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:16:00.814669 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: [pid: 80245|app: 0|req: 21/43] 10.222.0.47 () {60 vars in 1141 bytes} [Thu Jul 21 08:16:00 2022] POST /identity/v3/projects => generated 312 bytes in 185 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 08:16:00.814669 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49fcf5ec-1f18-470b-8929-0f8374fdcc4f None None] REQUEST_METHOD: `POST` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:16:00.814669 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49fcf5ec-1f18-470b-8929-0f8374fdcc4f None None] SCRIPT_NAME: `/identity` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:16:00.814669 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49fcf5ec-1f18-470b-8929-0f8374fdcc4f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:16:00.843941 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG dbcounter [-] [80245] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80245) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:16:00.851317 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.auth.core [None req-49fcf5ec-1f18-470b-8929-0f8374fdcc4f None None] MFA Rules not processed for user `2dce40b8d74c41ec94cc36e6ff05d412`. Rule list: `[]` (Enabled: `True`). {{(pid=80245) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 08:16:00.853722 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.fernet_utils [None req-49fcf5ec-1f18-470b-8929-0f8374fdcc4f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80245) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 08:16:00.872652 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4298717e-a2fc-4030-b8a8-812827b00c97 None None] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:16:00.918117 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a787ad5-63bf-4a48-95bb-dc0b6d91f0f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:16:00.919027 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a787ad5-63bf-4a48-95bb-dc0b6d91f0f2 None admin] REQUEST_METHOD: `PUT` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:16:00.919319 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a787ad5-63bf-4a48-95bb-dc0b6d91f0f2 None admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:16:00.919630 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a787ad5-63bf-4a48-95bb-dc0b6d91f0f2 None admin] PATH_INFO: `/v3/domains/default/users/2dce40b8d74c41ec94cc36e6ff05d412/roles/e29870f6e9e2411a9db0d5bf45f32ca5` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:16:00.921797 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a787ad5-63bf-4a48-95bb-dc0b6d91f0f2 None admin] RBAC: Authorizing `identity:create_grant(domain_id=default, user_id=2dce40b8d74c41ec94cc36e6ff05d412, role_id=e29870f6e9e2411a9db0d5bf45f32ca5)` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:16:00.923309 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a787ad5-63bf-4a48-95bb-dc0b6d91f0f2 None admin] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:16:00.939171 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: [pid: 80245|app: 0|req: 22/44] 10.222.0.47 () {58 vars in 1023 bytes} [Thu Jul 21 08:16:00 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 21 08:16:00.940976 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 23/45] 10.222.0.47 () {58 vars in 1361 bytes} [Thu Jul 21 08:16:00 2022] PUT /identity/v3/domains/default/users/2dce40b8d74c41ec94cc36e6ff05d412/roles/e29870f6e9e2411a9db0d5bf45f32ca5 => generated 0 bytes in 185 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 08:16:00.962666 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49fcf5ec-1f18-470b-8929-0f8374fdcc4f None None] Authenticating user token {{(pid=80245) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:16:01.081666 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49fcf5ec-1f18-470b-8929-0f8374fdcc4f None None] Validating token access rules against request {{(pid=80245) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:16:01.124542 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19a2681f-bef9-4d48-a95a-baec1f3f3594 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80245) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:16:01.125074 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19a2681f-bef9-4d48-a95a-baec1f3f3594 None admin] REQUEST_METHOD: `POST` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:16:01.125386 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19a2681f-bef9-4d48-a95a-baec1f3f3594 None admin] SCRIPT_NAME: `/identity` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:16:01.125664 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19a2681f-bef9-4d48-a95a-baec1f3f3594 None admin] PATH_INFO: `/v3/roles` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:16:01.126418 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19a2681f-bef9-4d48-a95a-baec1f3f3594 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:16:01.127861 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19a2681f-bef9-4d48-a95a-baec1f3f3594 None admin] RBAC: Authorization granted {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:16:01.131143 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG dbcounter [-] [80245] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80245) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:16:01.148360 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: [pid: 80245|app: 0|req: 23/46] 10.222.0.47 () {60 vars in 1131 bytes} [Thu Jul 21 08:16:00 2022] POST /identity/v3/roles => generated 223 bytes in 186 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 08:16:02.946852 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25bd20dc-6763-44ea-9e81-7aaba465ef36 None None] REQUEST_METHOD: `GET` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:16:02.947390 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25bd20dc-6763-44ea-9e81-7aaba465ef36 None None] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:16:02.947474 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25bd20dc-6763-44ea-9e81-7aaba465ef36 None None] PATH_INFO: `/` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:16:02.948835 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 24/47] 10.222.0.47 () {54 vars in 926 bytes} [Thu Jul 21 08:16:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 08:16:02.954075 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26dfff58-6ebe-4238-9f72-62d596e6158b None None] REQUEST_METHOD: `POST` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:16:02.954514 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26dfff58-6ebe-4238-9f72-62d596e6158b None None] SCRIPT_NAME: `/identity` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:16:02.954645 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26dfff58-6ebe-4238-9f72-62d596e6158b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:16:02.971841 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.auth.core [None req-26dfff58-6ebe-4238-9f72-62d596e6158b None None] MFA Rules not processed for user `2dce40b8d74c41ec94cc36e6ff05d412`. Rule list: `[]` (Enabled: `True`). {{(pid=80245) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 08:16:02.973028 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.fernet_utils [None req-26dfff58-6ebe-4238-9f72-62d596e6158b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80245) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 08:16:03.036784 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c55f0f2-ef82-48a1-96c8-eea95eb397d9 None None] REQUEST_METHOD: `GET` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:16:03.037308 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c55f0f2-ef82-48a1-96c8-eea95eb397d9 None None] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:16:03.037595 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c55f0f2-ef82-48a1-96c8-eea95eb397d9 None None] PATH_INFO: `/` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:16:03.039342 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 25/48] 10.222.0.47 () {54 vars in 926 bytes} [Thu Jul 21 08:16:03 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 21 08:16:03.044565 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5ea7e98-6a70-4bac-83a8-75161f92341d None None] REQUEST_METHOD: `POST` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:16:03.044857 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5ea7e98-6a70-4bac-83a8-75161f92341d None None] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:16:03.045139 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5ea7e98-6a70-4bac-83a8-75161f92341d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:16:03.047179 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: [pid: 80245|app: 0|req: 24/49] 10.222.0.47 () {58 vars in 1023 bytes} [Thu Jul 21 08:16:02 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 21 08:16:03.064743 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.auth.core [None req-c5ea7e98-6a70-4bac-83a8-75161f92341d None None] MFA Rules not processed for user `2dce40b8d74c41ec94cc36e6ff05d412`. Rule list: `[]` (Enabled: `True`). {{(pid=80244) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 08:16:03.066411 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.fernet_utils [None req-c5ea7e98-6a70-4bac-83a8-75161f92341d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80244) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 08:16:03.071854 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26dfff58-6ebe-4238-9f72-62d596e6158b None None] Authenticating user token {{(pid=80245) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:16:03.088166 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG dbcounter [-] [80244] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80244) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:16:03.140934 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 26/50] 10.222.0.47 () {58 vars in 1023 bytes} [Thu Jul 21 08:16:03 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 21 08:16:03.164178 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5ea7e98-6a70-4bac-83a8-75161f92341d None None] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:16:03.172917 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26dfff58-6ebe-4238-9f72-62d596e6158b None None] Validating token access rules against request {{(pid=80245) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:16:03.198843 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG dbcounter [-] [80245] Writing DB stats keystone:SELECT=100 {{(pid=80245) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:16:03.207941 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43bf8bd2-0080-4482-80a9-4f50f39a5771 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80245) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:16:03.208753 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43bf8bd2-0080-4482-80a9-4f50f39a5771 None admin] REQUEST_METHOD: `GET` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:16:03.209042 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43bf8bd2-0080-4482-80a9-4f50f39a5771 None admin] SCRIPT_NAME: `/identity` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:16:03.209329 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43bf8bd2-0080-4482-80a9-4f50f39a5771 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:16:03.211081 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43bf8bd2-0080-4482-80a9-4f50f39a5771 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:16:03.212580 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43bf8bd2-0080-4482-80a9-4f50f39a5771 None admin] RBAC: Authorization granted {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:16:03.214217 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: [pid: 80245|app: 0|req: 25/51] 10.222.0.47 () {56 vars in 1108 bytes} [Thu Jul 21 08:16:03 2022] GET /identity/v3/domains/default => generated 199 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 21 08:16:03.220212 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43bf8bd2-0080-4482-80a9-4f50f39a5771 None admin] Authenticating user token {{(pid=80245) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:16:03.261804 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5ea7e98-6a70-4bac-83a8-75161f92341d None None] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:16:03.295172 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79fd6cda-31fd-4b9d-8e9a-3ede575f22d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:16:03.295993 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79fd6cda-31fd-4b9d-8e9a-3ede575f22d2 None admin] REQUEST_METHOD: `GET` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:16:03.296281 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79fd6cda-31fd-4b9d-8e9a-3ede575f22d2 None admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:16:03.296555 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79fd6cda-31fd-4b9d-8e9a-3ede575f22d2 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:16:03.306755 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79fd6cda-31fd-4b9d-8e9a-3ede575f22d2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 21 08:16:03.314502 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application [None req-79fd6cda-31fd-4b9d-8e9a-3ede575f22d2 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 21 08:16:03.314502 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 08:16:03.314502 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 08:16:03.314502 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 08:16:03.314502 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 08:16:03.314502 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 08:16:03.314502 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 08:16:03.314502 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 08:16:03.314502 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 08:16:03.314502 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 08:16:03.314502 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 08:16:03.314502 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 08:16:03.314502 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 21 08:16:03.314502 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 21 08:16:03.314502 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 21 08:16:03.314502 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 21 08:16:03.315366 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 21 08:16:03.315366 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 21 08:16:03.315366 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 21 08:16:03.315366 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 21 08:16:03.315366 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 08:16:03.315366 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 08:16:03.315366 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 08:16:03.315366 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 08:16:03.315366 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 08:16:03.315366 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 08:16:03.315366 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 08:16:03.315366 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application with Lock( Jul 21 08:16:03.315366 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 08:16:03.315366 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self._enter() Jul 21 08:16:03.315366 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 08:16:03.315366 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 08:16:03.316294 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 08:16:03.316294 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self.creator() Jul 21 08:16:03.316294 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 08:16:03.316294 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application created_value = creator( Jul 21 08:16:03.316294 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 21 08:16:03.316294 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 21 08:16:03.316294 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 21 08:16:03.316294 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 21 08:16:03.316294 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 21 08:16:03.316294 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 21 08:16:03.316294 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 21 08:16:03.316294 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application Jul 21 08:16:03.316294 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 27/52] 10.222.0.47 () {56 vars in 1096 bytes} [Thu Jul 21 08:16:03 2022] GET /identity/v3/roles/admin => generated 83 bytes in 152 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 21 08:16:03.320218 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43bf8bd2-0080-4482-80a9-4f50f39a5771 None admin] Validating token access rules against request {{(pid=80245) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:16:03.322003 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79fd6cda-31fd-4b9d-8e9a-3ede575f22d2 None admin] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:16:03.341034 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG dbcounter [-] [80244] Writing DB stats keystone:SELECT=100 {{(pid=80244) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:16:03.355738 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30902729-b9cf-45c7-8e13-49e127dac1ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80245) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:16:03.356415 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30902729-b9cf-45c7-8e13-49e127dac1ed None admin] REQUEST_METHOD: `POST` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:16:03.356702 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30902729-b9cf-45c7-8e13-49e127dac1ed None admin] SCRIPT_NAME: `/identity` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:16:03.356977 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30902729-b9cf-45c7-8e13-49e127dac1ed None admin] PATH_INFO: `/v3/projects` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:16:03.357689 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30902729-b9cf-45c7-8e13-49e127dac1ed None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:16:03.359124 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30902729-b9cf-45c7-8e13-49e127dac1ed None admin] RBAC: Authorization granted {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:16:03.384797 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: [pid: 80245|app: 0|req: 26/53] 10.222.0.47 () {60 vars in 1140 bytes} [Thu Jul 21 08:16:03 2022] POST /identity/v3/projects => generated 298 bytes in 165 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 08:16:03.429718 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79fd6cda-31fd-4b9d-8e9a-3ede575f22d2 None admin] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:16:03.466686 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80d6a73c-70a7-4b0c-97f0-9d1ae2d53e58 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:16:03.467637 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80d6a73c-70a7-4b0c-97f0-9d1ae2d53e58 None admin] REQUEST_METHOD: `GET` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:16:03.467921 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80d6a73c-70a7-4b0c-97f0-9d1ae2d53e58 None admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:16:03.468198 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80d6a73c-70a7-4b0c-97f0-9d1ae2d53e58 None admin] PATH_INFO: `/v3/roles` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:16:03.469027 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80d6a73c-70a7-4b0c-97f0-9d1ae2d53e58 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80244) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 08:16:03.469338 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80d6a73c-70a7-4b0c-97f0-9d1ae2d53e58 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:16:03.470779 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80d6a73c-70a7-4b0c-97f0-9d1ae2d53e58 None admin] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:16:03.481000 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 28/54] 10.222.0.47 () {56 vars in 1099 bytes} [Thu Jul 21 08:16:03 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 21 08:16:03.487015 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30902729-b9cf-45c7-8e13-49e127dac1ed None admin] Authenticating user token {{(pid=80245) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:16:03.640722 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG dbcounter [-] [80245] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80245) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:16:03.695683 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30902729-b9cf-45c7-8e13-49e127dac1ed None admin] Validating token access rules against request {{(pid=80245) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:16:03.730302 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8188a7f-ce48-44e5-b29c-9ededa074bdd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80245) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:16:03.731119 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8188a7f-ce48-44e5-b29c-9ededa074bdd None admin] REQUEST_METHOD: `GET` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:16:03.731507 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8188a7f-ce48-44e5-b29c-9ededa074bdd None admin] SCRIPT_NAME: `/identity` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:16:03.731728 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8188a7f-ce48-44e5-b29c-9ededa074bdd None admin] PATH_INFO: `/v3/users/2dce40b8d74c41ec94cc36e6ff05d412` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:16:03.733176 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8188a7f-ce48-44e5-b29c-9ededa074bdd None admin] RBAC: Authorizing `identity:get_user(user_id=2dce40b8d74c41ec94cc36e6ff05d412)` {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:16:03.734622 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8188a7f-ce48-44e5-b29c-9ededa074bdd None admin] RBAC: Authorization granted {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:16:03.736470 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: [pid: 80245|app: 0|req: 27/55] 10.222.0.47 () {56 vars in 1177 bytes} [Thu Jul 21 08:16:03 2022] GET /identity/v3/users/2dce40b8d74c41ec94cc36e6ff05d412 => generated 247 bytes in 250 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 21 08:16:03.742431 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80d6a73c-70a7-4b0c-97f0-9d1ae2d53e58 None admin] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:16:03.820789 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG dbcounter [-] [80244] Writing DB stats keystone:SELECT=100 {{(pid=80244) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:16:03.842487 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80d6a73c-70a7-4b0c-97f0-9d1ae2d53e58 None admin] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:16:03.876697 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5676e6f1-0621-484f-94bf-c41410176e54 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:16:03.877518 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5676e6f1-0621-484f-94bf-c41410176e54 None admin] REQUEST_METHOD: `GET` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:16:03.877810 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5676e6f1-0621-484f-94bf-c41410176e54 None admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:16:03.878092 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5676e6f1-0621-484f-94bf-c41410176e54 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:16:03.879273 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5676e6f1-0621-484f-94bf-c41410176e54 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:16:03.880788 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5676e6f1-0621-484f-94bf-c41410176e54 None admin] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:16:03.882491 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 29/56] 10.222.0.47 () {56 vars in 1108 bytes} [Thu Jul 21 08:16:03 2022] GET /identity/v3/domains/default => generated 199 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 08:16:03.888596 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8188a7f-ce48-44e5-b29c-9ededa074bdd None admin] Authenticating user token {{(pid=80245) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:16:03.988087 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8188a7f-ce48-44e5-b29c-9ededa074bdd None admin] Validating token access rules against request {{(pid=80245) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:16:04.001760 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG dbcounter [-] [80245] Writing DB stats keystone:SELECT=100 {{(pid=80245) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:16:04.022648 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-697b9161-f092-4df1-adee-a4643a5a9523 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80245) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:16:04.023308 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-697b9161-f092-4df1-adee-a4643a5a9523 None admin] REQUEST_METHOD: `GET` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:16:04.023623 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-697b9161-f092-4df1-adee-a4643a5a9523 None admin] SCRIPT_NAME: `/identity` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:16:04.023902 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-697b9161-f092-4df1-adee-a4643a5a9523 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:16:04.024738 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-697b9161-f092-4df1-adee-a4643a5a9523 None admin] RBAC: Adding query filter params (role.id=e29870f6e9e2411a9db0d5bf45f32ca5, scope.domain.id=default, user.id=2dce40b8d74c41ec94cc36e6ff05d412) {{(pid=80245) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 08:16:04.025043 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-697b9161-f092-4df1-adee-a4643a5a9523 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:16:04.026471 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-697b9161-f092-4df1-adee-a4643a5a9523 None admin] RBAC: Authorization granted {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:16:04.036048 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: [pid: 80245|app: 0|req: 28/57] 10.222.0.47 () {56 vars in 1322 bytes} [Thu Jul 21 08:16:03 2022] GET /identity/v3/role_assignments?user.id=2dce40b8d74c41ec94cc36e6ff05d412&scope.domain.id=default&role.id=e29870f6e9e2411a9db0d5bf45f32ca5 => generated 530 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 08:16:05.441216 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-151f9bf6-1199-445a-a984-4252ab405bd7 None None] REQUEST_METHOD: `GET` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:16:05.441729 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-151f9bf6-1199-445a-a984-4252ab405bd7 None None] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:16:05.441812 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-151f9bf6-1199-445a-a984-4252ab405bd7 None None] PATH_INFO: `/` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:16:05.443165 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 30/58] 10.222.0.47 () {54 vars in 926 bytes} [Thu Jul 21 08:16:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 08:16:05.448815 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b9f1cce-f9e7-446a-a8ba-2f9ca9e879a5 None None] REQUEST_METHOD: `POST` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:16:05.449225 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b9f1cce-f9e7-446a-a8ba-2f9ca9e879a5 None None] SCRIPT_NAME: `/identity` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:16:05.449391 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b9f1cce-f9e7-446a-a8ba-2f9ca9e879a5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:16:05.467606 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.auth.core [None req-1b9f1cce-f9e7-446a-a8ba-2f9ca9e879a5 None None] MFA Rules not processed for user `2dce40b8d74c41ec94cc36e6ff05d412`. Rule list: `[]` (Enabled: `True`). {{(pid=80245) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 08:16:05.468817 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.fernet_utils [None req-1b9f1cce-f9e7-446a-a8ba-2f9ca9e879a5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80245) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 08:16:05.534466 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: [pid: 80245|app: 0|req: 29/59] 10.222.0.47 () {58 vars in 1023 bytes} [Thu Jul 21 08:16:05 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 21 08:16:05.558576 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-151f9bf6-1199-445a-a984-4252ab405bd7 None None] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:16:05.660431 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-151f9bf6-1199-445a-a984-4252ab405bd7 None None] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:16:05.694182 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faa32737-52c3-42ec-a7c0-116cf7a7ea15 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:16:05.695550 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faa32737-52c3-42ec-a7c0-116cf7a7ea15 None admin] REQUEST_METHOD: `GET` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:16:05.695842 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faa32737-52c3-42ec-a7c0-116cf7a7ea15 None admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:16:05.696130 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faa32737-52c3-42ec-a7c0-116cf7a7ea15 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:16:05.697318 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faa32737-52c3-42ec-a7c0-116cf7a7ea15 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:16:05.698806 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faa32737-52c3-42ec-a7c0-116cf7a7ea15 None admin] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:16:05.700467 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 31/60] 10.222.0.47 () {56 vars in 1108 bytes} [Thu Jul 21 08:16:05 2022] GET /identity/v3/domains/default => generated 199 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 08:16:05.706141 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b9f1cce-f9e7-446a-a8ba-2f9ca9e879a5 None None] Authenticating user token {{(pid=80245) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:16:05.801514 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b9f1cce-f9e7-446a-a8ba-2f9ca9e879a5 None None] Validating token access rules against request {{(pid=80245) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:16:05.810852 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG dbcounter [-] [80245] Writing DB stats keystone:SELECT=100 {{(pid=80245) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:16:05.835950 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aaaa6fa-63e4-446c-b171-b8e730070665 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80245) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:16:05.836620 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aaaa6fa-63e4-446c-b171-b8e730070665 None admin] REQUEST_METHOD: `POST` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:16:05.836931 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aaaa6fa-63e4-446c-b171-b8e730070665 None admin] SCRIPT_NAME: `/identity` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:16:05.837215 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aaaa6fa-63e4-446c-b171-b8e730070665 None admin] PATH_INFO: `/v3/users` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:16:05.837923 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4aaaa6fa-63e4-446c-b171-b8e730070665 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:16:05.839421 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4aaaa6fa-63e4-446c-b171-b8e730070665 None admin] RBAC: Authorization granted {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:16:05.863569 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: [pid: 80245|app: 0|req: 30/61] 10.222.0.47 () {60 vars in 1132 bytes} [Thu Jul 21 08:16:05 2022] POST /identity/v3/users => generated 275 bytes in 158 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 08:16:09.131999 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbff6e3f-5afd-43bd-899e-6f4054282954 None None] REQUEST_METHOD: `GET` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:16:09.131999 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbff6e3f-5afd-43bd-899e-6f4054282954 None None] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:16:09.131999 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbff6e3f-5afd-43bd-899e-6f4054282954 None None] PATH_INFO: `/` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:16:09.131999 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 32/62] 10.222.0.47 () {54 vars in 926 bytes} [Thu Jul 21 08:16:09 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 08:16:09.141157 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78126bdf-e637-4c59-8a5d-b1b348bb77a7 None None] REQUEST_METHOD: `POST` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:16:09.141835 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78126bdf-e637-4c59-8a5d-b1b348bb77a7 None None] SCRIPT_NAME: `/identity` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:16:09.141835 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78126bdf-e637-4c59-8a5d-b1b348bb77a7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:16:09.157486 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d198979c-93d3-481b-b603-2e22b0d3b1e3 None None] REQUEST_METHOD: `GET` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:16:09.158254 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d198979c-93d3-481b-b603-2e22b0d3b1e3 None None] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:16:09.158945 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d198979c-93d3-481b-b603-2e22b0d3b1e3 None None] PATH_INFO: `/` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:16:09.161744 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 33/63] 10.222.0.47 () {54 vars in 926 bytes} [Thu Jul 21 08:16:09 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 21 08:16:09.162008 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.auth.core [None req-78126bdf-e637-4c59-8a5d-b1b348bb77a7 None None] MFA Rules not processed for user `2dce40b8d74c41ec94cc36e6ff05d412`. Rule list: `[]` (Enabled: `True`). {{(pid=80245) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 08:16:09.164072 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.fernet_utils [None req-78126bdf-e637-4c59-8a5d-b1b348bb77a7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80245) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 08:16:09.174309 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-082ad66c-0bb5-430b-8db4-deba232867c0 None None] REQUEST_METHOD: `POST` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:16:09.175216 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-082ad66c-0bb5-430b-8db4-deba232867c0 None None] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:16:09.176094 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-082ad66c-0bb5-430b-8db4-deba232867c0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:16:09.208899 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.auth.core [None req-082ad66c-0bb5-430b-8db4-deba232867c0 None None] MFA Rules not processed for user `2dce40b8d74c41ec94cc36e6ff05d412`. Rule list: `[]` (Enabled: `True`). {{(pid=80244) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 08:16:09.212409 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.fernet_utils [None req-082ad66c-0bb5-430b-8db4-deba232867c0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80244) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 08:16:09.235673 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG dbcounter [-] [80244] Writing DB stats keystone:SELECT=100 {{(pid=80244) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:16:09.281405 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: [pid: 80245|app: 0|req: 31/64] 10.222.0.47 () {58 vars in 1023 bytes} [Thu Jul 21 08:16:09 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 141 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 21 08:16:09.321439 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78126bdf-e637-4c59-8a5d-b1b348bb77a7 None None] Authenticating user token {{(pid=80245) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:16:09.353888 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 34/65] 10.222.0.47 () {58 vars in 1023 bytes} [Thu Jul 21 08:16:09 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 181 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 21 08:16:09.446598 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77847337-3708-431f-9ddb-6adbc03bed04 None None] REQUEST_METHOD: `GET` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:16:09.447111 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77847337-3708-431f-9ddb-6adbc03bed04 None None] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:16:09.447641 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77847337-3708-431f-9ddb-6adbc03bed04 None None] PATH_INFO: `/` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:16:09.450159 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 35/66] 10.222.0.47 () {54 vars in 926 bytes} [Thu Jul 21 08:16:09 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 08:16:09.455714 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f66d0d0-8ab6-4959-9df7-3b77f919a0db None None] REQUEST_METHOD: `POST` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:16:09.456240 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f66d0d0-8ab6-4959-9df7-3b77f919a0db None None] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:16:09.456717 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f66d0d0-8ab6-4959-9df7-3b77f919a0db None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:16:09.487480 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.auth.core [None req-0f66d0d0-8ab6-4959-9df7-3b77f919a0db None None] MFA Rules not processed for user `2dce40b8d74c41ec94cc36e6ff05d412`. Rule list: `[]` (Enabled: `True`). {{(pid=80244) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 08:16:09.488976 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.fernet_utils [None req-0f66d0d0-8ab6-4959-9df7-3b77f919a0db None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80244) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 08:16:09.519521 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG dbcounter [-] [80245] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80245) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:16:09.528934 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78126bdf-e637-4c59-8a5d-b1b348bb77a7 None None] Validating token access rules against request {{(pid=80245) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:16:09.574048 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4f61f62-5f7e-4e0f-8d5a-bcfb9a8862ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80245) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:16:09.574860 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4f61f62-5f7e-4e0f-8d5a-bcfb9a8862ee None admin] REQUEST_METHOD: `GET` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:16:09.575145 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4f61f62-5f7e-4e0f-8d5a-bcfb9a8862ee None admin] SCRIPT_NAME: `/identity` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:16:09.575446 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4f61f62-5f7e-4e0f-8d5a-bcfb9a8862ee None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:16:09.587976 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4f61f62-5f7e-4e0f-8d5a-bcfb9a8862ee None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 21 08:16:09.588264 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 36/67] 10.222.0.47 () {58 vars in 1023 bytes} [Thu Jul 21 08:16:09 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 21 08:16:09.594798 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f66d0d0-8ab6-4959-9df7-3b77f919a0db None None] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:16:09.608264 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application [None req-b4f61f62-5f7e-4e0f-8d5a-bcfb9a8862ee None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 21 08:16:09.608264 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 08:16:09.608264 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 08:16:09.608264 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 08:16:09.608264 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 08:16:09.608264 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 08:16:09.608264 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 08:16:09.608264 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 08:16:09.608264 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 08:16:09.608264 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 08:16:09.608264 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 08:16:09.608264 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 08:16:09.608264 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 21 08:16:09.608264 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 21 08:16:09.608264 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 21 08:16:09.608264 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 21 08:16:09.609138 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 21 08:16:09.609138 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 21 08:16:09.609138 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 21 08:16:09.609138 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 21 08:16:09.609138 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 08:16:09.609138 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 08:16:09.609138 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 08:16:09.609138 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 08:16:09.609138 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 08:16:09.609138 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 08:16:09.609138 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 08:16:09.609138 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application with Lock( Jul 21 08:16:09.609138 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 08:16:09.609138 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application return self._enter() Jul 21 08:16:09.609138 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 08:16:09.609138 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 08:16:09.609888 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 08:16:09.609888 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application return self.creator() Jul 21 08:16:09.609888 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 08:16:09.609888 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application created_value = creator( Jul 21 08:16:09.609888 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 21 08:16:09.609888 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 21 08:16:09.609888 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 21 08:16:09.609888 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 21 08:16:09.609888 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 21 08:16:09.609888 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 21 08:16:09.609888 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 21 08:16:09.609888 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application Jul 21 08:16:09.612379 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: [pid: 80245|app: 0|req: 32/68] 10.222.0.47 () {56 vars in 1096 bytes} [Thu Jul 21 08:16:09 2022] GET /identity/v3/roles/admin => generated 83 bytes in 292 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 21 08:16:09.613856 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46aa0463-5772-444f-a02b-649840b58d96 None None] REQUEST_METHOD: `GET` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:16:09.614352 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46aa0463-5772-444f-a02b-649840b58d96 None None] SCRIPT_NAME: `/identity` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:16:09.614848 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46aa0463-5772-444f-a02b-649840b58d96 None None] PATH_INFO: `/` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:16:09.616930 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: [pid: 80245|app: 0|req: 33/69] 10.222.0.47 () {54 vars in 926 bytes} [Thu Jul 21 08:16:09 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 08:16:09.619081 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46aa0463-5772-444f-a02b-649840b58d96 None None] Authenticating user token {{(pid=80245) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:16:09.675803 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG dbcounter [-] [80244] Writing DB stats keystone:SELECT=100 {{(pid=80244) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:16:09.701602 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f66d0d0-8ab6-4959-9df7-3b77f919a0db None None] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:16:09.729634 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46aa0463-5772-444f-a02b-649840b58d96 None None] Validating token access rules against request {{(pid=80245) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:16:09.734551 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5f43c29-becb-43b3-bcdf-427785958fb8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:16:09.735379 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5f43c29-becb-43b3-bcdf-427785958fb8 None admin] REQUEST_METHOD: `GET` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:16:09.735702 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5f43c29-becb-43b3-bcdf-427785958fb8 None admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:16:09.735980 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5f43c29-becb-43b3-bcdf-427785958fb8 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:16:09.746099 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5f43c29-becb-43b3-bcdf-427785958fb8 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 21 08:16:09.753745 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application [None req-c5f43c29-becb-43b3-bcdf-427785958fb8 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 21 08:16:09.753745 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 08:16:09.753745 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 08:16:09.753745 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 08:16:09.753745 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 08:16:09.753745 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 08:16:09.753745 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 08:16:09.753745 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 08:16:09.753745 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 08:16:09.753745 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 08:16:09.753745 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 08:16:09.753745 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 08:16:09.753745 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 21 08:16:09.753745 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 21 08:16:09.753745 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 21 08:16:09.753745 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 21 08:16:09.754651 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 21 08:16:09.754651 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 21 08:16:09.754651 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 21 08:16:09.754651 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 21 08:16:09.754651 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 08:16:09.754651 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 08:16:09.754651 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 08:16:09.754651 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 08:16:09.754651 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 08:16:09.754651 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 08:16:09.754651 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 08:16:09.754651 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application with Lock( Jul 21 08:16:09.754651 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 08:16:09.754651 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self._enter() Jul 21 08:16:09.754651 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 08:16:09.754651 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 08:16:09.761233 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 08:16:09.761233 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self.creator() Jul 21 08:16:09.761233 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 08:16:09.761233 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application created_value = creator( Jul 21 08:16:09.761233 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 21 08:16:09.761233 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 21 08:16:09.761233 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 21 08:16:09.761233 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 21 08:16:09.761233 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 21 08:16:09.761233 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 21 08:16:09.761233 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 21 08:16:09.761233 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application Jul 21 08:16:09.761233 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 37/70] 10.222.0.47 () {56 vars in 1099 bytes} [Thu Jul 21 08:16:09 2022] GET /identity/v3/roles/member => generated 84 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 08:16:09.761233 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76fe5be2-3110-4aba-8926-1165dd052fa3 None None] REQUEST_METHOD: `POST` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:16:09.761894 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76fe5be2-3110-4aba-8926-1165dd052fa3 None None] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:16:09.761894 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76fe5be2-3110-4aba-8926-1165dd052fa3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:16:09.763582 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b8a187f-5397-43c9-8681-49a144d98575 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80245) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:16:09.764289 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b8a187f-5397-43c9-8681-49a144d98575 None admin] REQUEST_METHOD: `GET` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:16:09.764574 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b8a187f-5397-43c9-8681-49a144d98575 None admin] SCRIPT_NAME: `/identity` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:16:09.764851 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b8a187f-5397-43c9-8681-49a144d98575 None admin] PATH_INFO: `/v3/roles` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:16:09.765603 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b8a187f-5397-43c9-8681-49a144d98575 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80245) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 08:16:09.765909 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b8a187f-5397-43c9-8681-49a144d98575 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:16:09.767294 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b8a187f-5397-43c9-8681-49a144d98575 None admin] RBAC: Authorization granted {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:16:09.774196 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.auth.core [None req-76fe5be2-3110-4aba-8926-1165dd052fa3 None None] MFA Rules not processed for user `2dce40b8d74c41ec94cc36e6ff05d412`. Rule list: `[]` (Enabled: `True`). {{(pid=80244) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 08:16:09.775391 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.fernet_utils [None req-76fe5be2-3110-4aba-8926-1165dd052fa3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80244) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 08:16:09.780106 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: [pid: 80245|app: 0|req: 34/71] 10.222.0.47 () {56 vars in 1099 bytes} [Thu Jul 21 08:16:09 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 08:16:09.780945 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b8a187f-5397-43c9-8681-49a144d98575 None admin] Authenticating user token {{(pid=80245) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:16:09.798554 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG dbcounter [-] [80245] Writing DB stats keystone:SELECT=100 {{(pid=80245) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:16:09.843344 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 38/72] 10.222.0.47 () {58 vars in 1023 bytes} [Thu Jul 21 08:16:09 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 21 08:16:09.844687 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32ce0e80-d1ea-47d2-9b79-e69ff1d17b80 None None] REQUEST_METHOD: `GET` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:16:09.844984 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32ce0e80-d1ea-47d2-9b79-e69ff1d17b80 None None] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:16:09.845268 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32ce0e80-d1ea-47d2-9b79-e69ff1d17b80 None None] PATH_INFO: `/` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:16:09.846665 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 39/73] 10.222.0.47 () {54 vars in 926 bytes} [Thu Jul 21 08:16:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 08:16:09.847251 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32ce0e80-d1ea-47d2-9b79-e69ff1d17b80 None None] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:16:09.896458 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b8a187f-5397-43c9-8681-49a144d98575 None admin] Validating token access rules against request {{(pid=80245) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:16:09.911676 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG dbcounter [-] [80244] Writing DB stats keystone:SELECT=100 {{(pid=80244) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:16:09.930652 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9304210e-f44b-403f-8446-5e6b897dee86 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80245) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:16:09.931503 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9304210e-f44b-403f-8446-5e6b897dee86 None admin] REQUEST_METHOD: `GET` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:16:09.931793 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9304210e-f44b-403f-8446-5e6b897dee86 None admin] SCRIPT_NAME: `/identity` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:16:09.932082 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9304210e-f44b-403f-8446-5e6b897dee86 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:16:09.942321 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9304210e-f44b-403f-8446-5e6b897dee86 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 21 08:16:09.950849 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application [None req-9304210e-f44b-403f-8446-5e6b897dee86 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 21 08:16:09.950849 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 08:16:09.950849 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 08:16:09.950849 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 08:16:09.950849 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 08:16:09.950849 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 08:16:09.950849 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 08:16:09.950849 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 08:16:09.950849 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 08:16:09.950849 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 08:16:09.950849 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 08:16:09.950849 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 08:16:09.950849 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 21 08:16:09.950849 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 21 08:16:09.950849 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 21 08:16:09.950849 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 21 08:16:09.951760 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 21 08:16:09.951760 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 21 08:16:09.951760 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 21 08:16:09.951760 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 21 08:16:09.951760 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 08:16:09.951760 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 08:16:09.951760 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 08:16:09.951760 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 08:16:09.951760 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 08:16:09.951760 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 08:16:09.951760 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 08:16:09.951760 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application with Lock( Jul 21 08:16:09.951760 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 08:16:09.951760 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application return self._enter() Jul 21 08:16:09.951760 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 08:16:09.951760 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 08:16:09.952512 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 08:16:09.952512 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application return self.creator() Jul 21 08:16:09.952512 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 08:16:09.952512 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application created_value = creator( Jul 21 08:16:09.952512 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 21 08:16:09.952512 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 21 08:16:09.952512 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 21 08:16:09.952512 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 21 08:16:09.952512 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 21 08:16:09.952512 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 21 08:16:09.952512 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 21 08:16:09.952512 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application Jul 21 08:16:09.953081 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32ce0e80-d1ea-47d2-9b79-e69ff1d17b80 None None] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:16:09.953157 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: [pid: 80245|app: 0|req: 35/74] 10.222.0.47 () {56 vars in 1114 bytes} [Thu Jul 21 08:16:09 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 08:16:09.953157 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9304210e-f44b-403f-8446-5e6b897dee86 None admin] Authenticating user token {{(pid=80245) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:16:09.990079 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aa67454-56ea-4222-a97e-667a92024fa9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:16:09.990763 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aa67454-56ea-4222-a97e-667a92024fa9 None admin] REQUEST_METHOD: `GET` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:16:09.991112 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aa67454-56ea-4222-a97e-667a92024fa9 None admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:16:09.991432 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aa67454-56ea-4222-a97e-667a92024fa9 None admin] PATH_INFO: `/v3/roles` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:16:09.992194 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2aa67454-56ea-4222-a97e-667a92024fa9 None admin] RBAC: Adding query filter params (name=member) {{(pid=80244) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 08:16:09.992549 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2aa67454-56ea-4222-a97e-667a92024fa9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:16:09.994193 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2aa67454-56ea-4222-a97e-667a92024fa9 None admin] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:16:10.002001 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 40/75] 10.222.0.47 () {56 vars in 1101 bytes} [Thu Jul 21 08:16:09 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 08:16:10.003199 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77291407-44c0-427a-a186-989b2508d13a None None] REQUEST_METHOD: `POST` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:16:10.003526 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77291407-44c0-427a-a186-989b2508d13a None None] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:16:10.003898 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77291407-44c0-427a-a186-989b2508d13a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:16:10.020901 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.auth.core [None req-77291407-44c0-427a-a186-989b2508d13a None None] MFA Rules not processed for user `2dce40b8d74c41ec94cc36e6ff05d412`. Rule list: `[]` (Enabled: `True`). {{(pid=80244) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 08:16:10.022110 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.fernet_utils [None req-77291407-44c0-427a-a186-989b2508d13a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80244) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 08:16:10.029207 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG dbcounter [-] [80245] Writing DB stats keystone:SELECT=100 {{(pid=80245) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:16:10.052398 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9304210e-f44b-403f-8446-5e6b897dee86 None admin] Validating token access rules against request {{(pid=80245) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:16:10.085509 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1856d4d-f426-4710-9490-4c7994d4f08e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80245) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:16:10.086378 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1856d4d-f426-4710-9490-4c7994d4f08e None admin] REQUEST_METHOD: `GET` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:16:10.086663 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1856d4d-f426-4710-9490-4c7994d4f08e None admin] SCRIPT_NAME: `/identity` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:16:10.086942 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1856d4d-f426-4710-9490-4c7994d4f08e None admin] PATH_INFO: `/v3/users/2dce40b8d74c41ec94cc36e6ff05d412` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:16:10.088307 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 41/76] 10.222.0.47 () {58 vars in 1023 bytes} [Thu Jul 21 08:16:10 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 21 08:16:10.088412 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1856d4d-f426-4710-9490-4c7994d4f08e None admin] RBAC: Authorizing `identity:get_user(user_id=2dce40b8d74c41ec94cc36e6ff05d412)` {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:16:10.089055 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77291407-44c0-427a-a186-989b2508d13a None None] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:16:10.089649 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1856d4d-f426-4710-9490-4c7994d4f08e None admin] RBAC: Authorization granted {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:16:10.095579 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: [pid: 80245|app: 0|req: 36/77] 10.222.0.47 () {56 vars in 1177 bytes} [Thu Jul 21 08:16:09 2022] GET /identity/v3/users/2dce40b8d74c41ec94cc36e6ff05d412 => generated 247 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 08:16:10.095579 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1856d4d-f426-4710-9490-4c7994d4f08e None admin] Authenticating user token {{(pid=80245) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:16:10.149262 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG dbcounter [-] [80244] Writing DB stats keystone:SELECT=100 {{(pid=80244) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:16:10.192974 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77291407-44c0-427a-a186-989b2508d13a None None] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:16:10.203237 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1856d4d-f426-4710-9490-4c7994d4f08e None admin] Validating token access rules against request {{(pid=80245) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:16:10.227851 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66fc3593-5085-4f1d-bfdb-ab9fc2efa2c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:16:10.228684 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66fc3593-5085-4f1d-bfdb-ab9fc2efa2c8 None admin] REQUEST_METHOD: `GET` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:16:10.229233 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66fc3593-5085-4f1d-bfdb-ab9fc2efa2c8 None admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:16:10.229338 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66fc3593-5085-4f1d-bfdb-ab9fc2efa2c8 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:16:10.230563 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66fc3593-5085-4f1d-bfdb-ab9fc2efa2c8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:16:10.232102 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66fc3593-5085-4f1d-bfdb-ab9fc2efa2c8 None admin] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:16:10.233779 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 42/78] 10.222.0.47 () {56 vars in 1108 bytes} [Thu Jul 21 08:16:10 2022] GET /identity/v3/domains/default => generated 199 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 08:16:10.234570 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66fc3593-5085-4f1d-bfdb-ab9fc2efa2c8 None admin] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:16:10.254803 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb899291-b1f7-4043-8beb-1e17f7bd2b94 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80245) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:16:10.255527 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb899291-b1f7-4043-8beb-1e17f7bd2b94 None admin] REQUEST_METHOD: `GET` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:16:10.255819 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb899291-b1f7-4043-8beb-1e17f7bd2b94 None admin] SCRIPT_NAME: `/identity` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:16:10.256102 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb899291-b1f7-4043-8beb-1e17f7bd2b94 None admin] PATH_INFO: `/v3/roles` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:16:10.256889 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb899291-b1f7-4043-8beb-1e17f7bd2b94 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80245) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 08:16:10.257200 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb899291-b1f7-4043-8beb-1e17f7bd2b94 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:16:10.258852 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb899291-b1f7-4043-8beb-1e17f7bd2b94 None admin] RBAC: Authorization granted {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:16:10.267049 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: [pid: 80245|app: 0|req: 37/79] 10.222.0.47 () {56 vars in 1111 bytes} [Thu Jul 21 08:16:10 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 08:16:10.271809 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb899291-b1f7-4043-8beb-1e17f7bd2b94 None admin] Authenticating user token {{(pid=80245) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:16:10.278044 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG dbcounter [-] [80245] Writing DB stats keystone:SELECT=100 {{(pid=80245) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:16:10.338596 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66fc3593-5085-4f1d-bfdb-ab9fc2efa2c8 None admin] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:16:10.362361 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG dbcounter [-] [80244] Writing DB stats keystone:SELECT=100 {{(pid=80244) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:16:10.368328 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb899291-b1f7-4043-8beb-1e17f7bd2b94 None admin] Validating token access rules against request {{(pid=80245) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:16:10.374344 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ec4a961-ae5c-4ff9-86cb-38944ca51504 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:16:10.375230 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ec4a961-ae5c-4ff9-86cb-38944ca51504 None admin] REQUEST_METHOD: `GET` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:16:10.375554 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ec4a961-ae5c-4ff9-86cb-38944ca51504 None admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:16:10.375832 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ec4a961-ae5c-4ff9-86cb-38944ca51504 None admin] PATH_INFO: `/v3/users/39bf6631ce574c758a674faedecf2254` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:16:10.389091 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ec4a961-ae5c-4ff9-86cb-38944ca51504 None admin] RBAC: Authorizing `identity:get_user(user_id=39bf6631ce574c758a674faedecf2254)` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:16:10.390542 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ec4a961-ae5c-4ff9-86cb-38944ca51504 None admin] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:16:10.392231 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 43/80] 10.222.0.47 () {56 vars in 1177 bytes} [Thu Jul 21 08:16:10 2022] GET /identity/v3/users/39bf6631ce574c758a674faedecf2254 => generated 275 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 08:16:10.392993 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ec4a961-ae5c-4ff9-86cb-38944ca51504 None admin] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:16:10.406371 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78d5c707-45b3-4e95-9694-819e6156b6d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80245) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:16:10.407038 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78d5c707-45b3-4e95-9694-819e6156b6d0 None admin] REQUEST_METHOD: `GET` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:16:10.407340 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78d5c707-45b3-4e95-9694-819e6156b6d0 None admin] SCRIPT_NAME: `/identity` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:16:10.407640 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78d5c707-45b3-4e95-9694-819e6156b6d0 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:16:10.484481 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78d5c707-45b3-4e95-9694-819e6156b6d0 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:16:10.485953 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78d5c707-45b3-4e95-9694-819e6156b6d0 None admin] RBAC: Authorization granted {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:16:10.493754 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG dbcounter [-] [80245] Writing DB stats keystone:SELECT=100 {{(pid=80245) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:16:10.503757 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ec4a961-ae5c-4ff9-86cb-38944ca51504 None admin] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:16:10.538855 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c9b5012-55c6-4b0d-8f01-331c852af930 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:16:10.539680 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c9b5012-55c6-4b0d-8f01-331c852af930 None admin] REQUEST_METHOD: `GET` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:16:10.539976 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c9b5012-55c6-4b0d-8f01-331c852af930 None admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:16:10.540257 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c9b5012-55c6-4b0d-8f01-331c852af930 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:16:10.550669 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c9b5012-55c6-4b0d-8f01-331c852af930 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 21 08:16:10.558346 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application [None req-4c9b5012-55c6-4b0d-8f01-331c852af930 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 21 08:16:10.558346 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 08:16:10.558346 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 08:16:10.558346 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 08:16:10.558346 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 08:16:10.558346 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 08:16:10.558346 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 08:16:10.558346 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 08:16:10.558346 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 08:16:10.558346 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 08:16:10.558346 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 08:16:10.558346 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 08:16:10.558346 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 21 08:16:10.558346 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 21 08:16:10.558346 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 21 08:16:10.558346 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 21 08:16:10.559470 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 21 08:16:10.559470 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 21 08:16:10.559470 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 21 08:16:10.559470 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 21 08:16:10.559470 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 08:16:10.559470 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 08:16:10.559470 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 08:16:10.559470 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 08:16:10.559470 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 08:16:10.559470 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 08:16:10.559470 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 08:16:10.559470 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application with Lock( Jul 21 08:16:10.559470 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 08:16:10.559470 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self._enter() Jul 21 08:16:10.559470 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 08:16:10.559470 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 08:16:10.560252 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 08:16:10.560252 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self.creator() Jul 21 08:16:10.560252 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 08:16:10.560252 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application created_value = creator( Jul 21 08:16:10.560252 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 21 08:16:10.560252 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 21 08:16:10.560252 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 21 08:16:10.560252 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 21 08:16:10.560252 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 21 08:16:10.560252 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 21 08:16:10.560252 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 21 08:16:10.560252 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application Jul 21 08:16:10.560252 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 44/81] 10.222.0.47 () {56 vars in 1099 bytes} [Thu Jul 21 08:16:10 2022] GET /identity/v3/roles/member => generated 84 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 08:16:10.560885 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c9b5012-55c6-4b0d-8f01-331c852af930 None admin] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:16:10.588056 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: [pid: 80245|app: 0|req: 38/82] 10.222.0.47 () {58 vars in 1282 bytes} [Thu Jul 21 08:16:10 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 320 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 21 08:16:10.588880 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78d5c707-45b3-4e95-9694-819e6156b6d0 None admin] Authenticating user token {{(pid=80245) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:16:10.611635 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG dbcounter [-] [80244] Writing DB stats keystone:SELECT=100 {{(pid=80244) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:16:10.671590 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c9b5012-55c6-4b0d-8f01-331c852af930 None admin] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:16:10.689345 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78d5c707-45b3-4e95-9694-819e6156b6d0 None admin] Validating token access rules against request {{(pid=80245) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:16:10.696490 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG dbcounter [-] [80245] Writing DB stats keystone:SELECT=100 {{(pid=80245) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:16:10.706425 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0766a760-2d06-4ed1-a170-9e52f68a932c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:16:10.707095 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0766a760-2d06-4ed1-a170-9e52f68a932c None admin] REQUEST_METHOD: `POST` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:16:10.707414 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0766a760-2d06-4ed1-a170-9e52f68a932c None admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:16:10.707689 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0766a760-2d06-4ed1-a170-9e52f68a932c None admin] PATH_INFO: `/v3/users` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:16:10.708406 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0766a760-2d06-4ed1-a170-9e52f68a932c None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:16:10.709861 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0766a760-2d06-4ed1-a170-9e52f68a932c None admin] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:16:10.724422 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faf7a74a-b9b8-45f5-9886-d0d9a3b1acd8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80245) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:16:10.725822 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faf7a74a-b9b8-45f5-9886-d0d9a3b1acd8 None admin] REQUEST_METHOD: `GET` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:16:10.726112 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faf7a74a-b9b8-45f5-9886-d0d9a3b1acd8 None admin] SCRIPT_NAME: `/identity` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:16:10.726392 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faf7a74a-b9b8-45f5-9886-d0d9a3b1acd8 None admin] PATH_INFO: `/v3/users/39bf6631ce574c758a674faedecf2254` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:16:10.727675 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faf7a74a-b9b8-45f5-9886-d0d9a3b1acd8 None admin] RBAC: Authorizing `identity:get_user(user_id=39bf6631ce574c758a674faedecf2254)` {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:16:10.729111 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faf7a74a-b9b8-45f5-9886-d0d9a3b1acd8 None admin] RBAC: Authorization granted {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:16:10.730762 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: [pid: 80245|app: 0|req: 39/83] 10.222.0.47 () {56 vars in 1177 bytes} [Thu Jul 21 08:16:10 2022] GET /identity/v3/users/39bf6631ce574c758a674faedecf2254 => generated 275 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 08:16:10.731557 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faf7a74a-b9b8-45f5-9886-d0d9a3b1acd8 None admin] Authenticating user token {{(pid=80245) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:16:10.741507 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 45/84] 10.222.0.47 () {60 vars in 1132 bytes} [Thu Jul 21 08:16:10 2022] POST /identity/v3/users => generated 289 bytes in 181 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 08:16:10.746912 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0766a760-2d06-4ed1-a170-9e52f68a932c None admin] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:16:10.856994 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faf7a74a-b9b8-45f5-9886-d0d9a3b1acd8 None admin] Validating token access rules against request {{(pid=80245) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:16:10.858772 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG dbcounter [-] [80244] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80244) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:16:10.863161 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0766a760-2d06-4ed1-a170-9e52f68a932c None admin] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:16:10.899983 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6acbedda-179e-4c99-be5e-6eceb2532b1b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80245) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:16:10.900945 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6acbedda-179e-4c99-be5e-6eceb2532b1b None admin] REQUEST_METHOD: `GET` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:16:10.901421 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6acbedda-179e-4c99-be5e-6eceb2532b1b None admin] SCRIPT_NAME: `/identity` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:16:10.901708 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6acbedda-179e-4c99-be5e-6eceb2532b1b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:16:10.908770 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-898005f2-8de7-4192-b397-45de9112f9e8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:16:10.909802 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-898005f2-8de7-4192-b397-45de9112f9e8 None admin] REQUEST_METHOD: `GET` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:16:10.909802 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-898005f2-8de7-4192-b397-45de9112f9e8 None admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:16:10.910067 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-898005f2-8de7-4192-b397-45de9112f9e8 None admin] PATH_INFO: `/v3/roles` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:16:10.910763 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-898005f2-8de7-4192-b397-45de9112f9e8 None admin] RBAC: Adding query filter params (name=member) {{(pid=80244) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 08:16:10.911070 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-898005f2-8de7-4192-b397-45de9112f9e8 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:16:10.912496 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-898005f2-8de7-4192-b397-45de9112f9e8 None admin] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:16:10.921609 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 46/85] 10.222.0.47 () {56 vars in 1101 bytes} [Thu Jul 21 08:16:10 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 08:16:10.922496 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-898005f2-8de7-4192-b397-45de9112f9e8 None admin] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:16:10.956655 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG dbcounter [-] [80245] Writing DB stats keystone:SELECT=100 {{(pid=80245) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:16:10.998724 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6acbedda-179e-4c99-be5e-6eceb2532b1b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:16:11.000238 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6acbedda-179e-4c99-be5e-6eceb2532b1b None admin] RBAC: Authorization granted {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:16:11.050850 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-898005f2-8de7-4192-b397-45de9112f9e8 None admin] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:16:11.092257 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0e7a16a-80b7-4f53-b4db-6b40cfffe35c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:16:11.093289 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0e7a16a-80b7-4f53-b4db-6b40cfffe35c None admin] REQUEST_METHOD: `GET` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:16:11.094427 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0e7a16a-80b7-4f53-b4db-6b40cfffe35c None admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:16:11.094718 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0e7a16a-80b7-4f53-b4db-6b40cfffe35c None admin] PATH_INFO: `/v3/projects/eab7e5ec54c94574ad68c89747ad5bcc` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:16:11.096078 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0e7a16a-80b7-4f53-b4db-6b40cfffe35c None admin] RBAC: Authorizing `identity:get_project(project_id=eab7e5ec54c94574ad68c89747ad5bcc)` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:16:11.097671 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0e7a16a-80b7-4f53-b4db-6b40cfffe35c None admin] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:16:11.100231 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 47/86] 10.222.0.47 () {56 vars in 1186 bytes} [Thu Jul 21 08:16:10 2022] GET /identity/v3/projects/eab7e5ec54c94574ad68c89747ad5bcc => generated 298 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 08:16:11.101922 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0e7a16a-80b7-4f53-b4db-6b40cfffe35c None admin] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:16:11.132133 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: [pid: 80245|app: 0|req: 40/87] 10.222.0.47 () {58 vars in 1282 bytes} [Thu Jul 21 08:16:10 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 401 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 21 08:16:11.133727 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6acbedda-179e-4c99-be5e-6eceb2532b1b None admin] Authenticating user token {{(pid=80245) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:16:11.140475 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG dbcounter [-] [80244] Writing DB stats keystone:SELECT=100 {{(pid=80244) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:16:11.229423 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG dbcounter [-] [80245] Writing DB stats keystone:SELECT=100 {{(pid=80245) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:16:11.239146 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0e7a16a-80b7-4f53-b4db-6b40cfffe35c None admin] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:16:11.262160 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6acbedda-179e-4c99-be5e-6eceb2532b1b None admin] Validating token access rules against request {{(pid=80245) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:16:11.282732 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6280d0cb-018a-4727-b942-10cae7fd098d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:16:11.283856 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6280d0cb-018a-4727-b942-10cae7fd098d None admin] REQUEST_METHOD: `GET` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:16:11.284297 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6280d0cb-018a-4727-b942-10cae7fd098d None admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:16:11.284691 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6280d0cb-018a-4727-b942-10cae7fd098d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:16:11.306449 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15d2e480-ffce-4181-b4fe-f701e07be60e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80245) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:16:11.307621 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15d2e480-ffce-4181-b4fe-f701e07be60e None admin] REQUEST_METHOD: `GET` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:16:11.308269 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15d2e480-ffce-4181-b4fe-f701e07be60e None admin] SCRIPT_NAME: `/identity` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:16:11.308623 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15d2e480-ffce-4181-b4fe-f701e07be60e None admin] PATH_INFO: `/v3/users/39bf6631ce574c758a674faedecf2254` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:16:11.309976 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15d2e480-ffce-4181-b4fe-f701e07be60e None admin] RBAC: Authorizing `identity:get_user(user_id=39bf6631ce574c758a674faedecf2254)` {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:16:11.312151 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15d2e480-ffce-4181-b4fe-f701e07be60e None admin] RBAC: Authorization granted {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:16:11.314859 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: [pid: 80245|app: 0|req: 41/88] 10.222.0.47 () {56 vars in 1177 bytes} [Thu Jul 21 08:16:11 2022] GET /identity/v3/users/39bf6631ce574c758a674faedecf2254 => generated 275 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 08:16:11.316110 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15d2e480-ffce-4181-b4fe-f701e07be60e None admin] Authenticating user token {{(pid=80245) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:16:11.370833 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6280d0cb-018a-4727-b942-10cae7fd098d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:16:11.372773 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6280d0cb-018a-4727-b942-10cae7fd098d None admin] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:16:11.413126 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG dbcounter [-] [80244] Writing DB stats keystone:SELECT=100 {{(pid=80244) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:16:11.445721 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15d2e480-ffce-4181-b4fe-f701e07be60e None admin] Validating token access rules against request {{(pid=80245) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:16:11.499002 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2438fab9-dabf-4839-a2b8-b19792e1f1be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80245) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:16:11.499963 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2438fab9-dabf-4839-a2b8-b19792e1f1be None admin] REQUEST_METHOD: `GET` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:16:11.500256 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2438fab9-dabf-4839-a2b8-b19792e1f1be None admin] SCRIPT_NAME: `/identity` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:16:11.500536 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2438fab9-dabf-4839-a2b8-b19792e1f1be None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:16:11.501824 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2438fab9-dabf-4839-a2b8-b19792e1f1be None admin] RBAC: Adding query filter params (role.id=e29870f6e9e2411a9db0d5bf45f32ca5, scope.project.id=eab7e5ec54c94574ad68c89747ad5bcc, user.id=2dce40b8d74c41ec94cc36e6ff05d412) {{(pid=80245) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 08:16:11.502415 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2438fab9-dabf-4839-a2b8-b19792e1f1be None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:16:11.504173 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2438fab9-dabf-4839-a2b8-b19792e1f1be None admin] RBAC: Authorization granted {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:16:11.512454 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: [pid: 80245|app: 0|req: 42/89] 10.222.0.47 () {56 vars in 1374 bytes} [Thu Jul 21 08:16:11 2022] GET /identity/v3/role_assignments?user.id=2dce40b8d74c41ec94cc36e6ff05d412&scope.project.id=eab7e5ec54c94574ad68c89747ad5bcc&role.id=e29870f6e9e2411a9db0d5bf45f32ca5 => generated 259 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 08:16:11.513014 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 48/90] 10.222.0.47 () {58 vars in 1282 bytes} [Thu Jul 21 08:16:11 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 412 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 21 08:16:11.513347 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2438fab9-dabf-4839-a2b8-b19792e1f1be None admin] Authenticating user token {{(pid=80245) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:16:11.514361 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6280d0cb-018a-4727-b942-10cae7fd098d None admin] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:16:11.524421 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG dbcounter [-] [80245] Writing DB stats keystone:SELECT=100 {{(pid=80245) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:16:11.684099 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6280d0cb-018a-4727-b942-10cae7fd098d None admin] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:16:11.697021 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2438fab9-dabf-4839-a2b8-b19792e1f1be None admin] Validating token access rules against request {{(pid=80245) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:16:11.732822 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ab5c17d-43b4-4186-ae63-36be7344c43c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80245) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:16:11.734056 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ab5c17d-43b4-4186-ae63-36be7344c43c None admin] REQUEST_METHOD: `GET` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:16:11.734538 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ab5c17d-43b4-4186-ae63-36be7344c43c None admin] SCRIPT_NAME: `/identity` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:16:11.735036 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ab5c17d-43b4-4186-ae63-36be7344c43c None admin] PATH_INFO: `/v3/projects/eab7e5ec54c94574ad68c89747ad5bcc` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:16:11.736512 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ab5c17d-43b4-4186-ae63-36be7344c43c None admin] RBAC: Authorizing `identity:get_project(project_id=eab7e5ec54c94574ad68c89747ad5bcc)` {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:16:11.738177 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ab5c17d-43b4-4186-ae63-36be7344c43c None admin] RBAC: Authorization granted {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:16:11.740466 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: [pid: 80245|app: 0|req: 43/91] 10.222.0.47 () {56 vars in 1186 bytes} [Thu Jul 21 08:16:11 2022] GET /identity/v3/projects/eab7e5ec54c94574ad68c89747ad5bcc => generated 298 bytes in 227 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 08:16:11.746809 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ab5c17d-43b4-4186-ae63-36be7344c43c None admin] Authenticating user token {{(pid=80245) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:16:11.757286 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG dbcounter [-] [80244] Writing DB stats keystone:SELECT=100 {{(pid=80244) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:16:11.775903 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c11ba13a-5350-4473-b8cf-5ecaef5bcece None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:16:11.776498 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c11ba13a-5350-4473-b8cf-5ecaef5bcece None admin] REQUEST_METHOD: `GET` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:16:11.776790 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c11ba13a-5350-4473-b8cf-5ecaef5bcece None admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:16:11.777071 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c11ba13a-5350-4473-b8cf-5ecaef5bcece None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:16:11.847703 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG dbcounter [-] [80245] Writing DB stats keystone:SELECT=100 {{(pid=80245) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:16:11.894470 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ab5c17d-43b4-4186-ae63-36be7344c43c None admin] Validating token access rules against request {{(pid=80245) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:16:11.922240 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c11ba13a-5350-4473-b8cf-5ecaef5bcece None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:16:11.924130 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c11ba13a-5350-4473-b8cf-5ecaef5bcece None admin] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:16:11.962428 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-300604ee-bfbd-4ad4-b6de-d8872f080ac4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80245) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:16:11.963760 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-300604ee-bfbd-4ad4-b6de-d8872f080ac4 None admin] REQUEST_METHOD: `GET` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:16:11.964327 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-300604ee-bfbd-4ad4-b6de-d8872f080ac4 None admin] SCRIPT_NAME: `/identity` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:16:11.964807 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-300604ee-bfbd-4ad4-b6de-d8872f080ac4 None admin] PATH_INFO: `/v3/projects/eab7e5ec54c94574ad68c89747ad5bcc` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:16:11.966225 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-300604ee-bfbd-4ad4-b6de-d8872f080ac4 None admin] RBAC: Authorizing `identity:get_project(project_id=eab7e5ec54c94574ad68c89747ad5bcc)` {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:16:11.969065 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-300604ee-bfbd-4ad4-b6de-d8872f080ac4 None admin] RBAC: Authorization granted {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:16:11.971932 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: [pid: 80245|app: 0|req: 44/92] 10.222.0.47 () {56 vars in 1186 bytes} [Thu Jul 21 08:16:11 2022] GET /identity/v3/projects/eab7e5ec54c94574ad68c89747ad5bcc => generated 298 bytes in 231 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 08:16:11.972927 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-300604ee-bfbd-4ad4-b6de-d8872f080ac4 None admin] Authenticating user token {{(pid=80245) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:16:12.110828 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 49/93] 10.222.0.47 () {58 vars in 1282 bytes} [Thu Jul 21 08:16:11 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 597 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 21 08:16:12.112250 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c11ba13a-5350-4473-b8cf-5ecaef5bcece None admin] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:16:12.152231 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG dbcounter [-] [80244] Writing DB stats keystone:SELECT=100 {{(pid=80244) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:16:12.168724 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-300604ee-bfbd-4ad4-b6de-d8872f080ac4 None admin] Validating token access rules against request {{(pid=80245) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:16:12.243215 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c11ba13a-5350-4473-b8cf-5ecaef5bcece None admin] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:16:12.258748 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97ede874-458a-4d0f-a185-8ef654f90cd2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80245) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:16:12.260174 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97ede874-458a-4d0f-a185-8ef654f90cd2 None admin] REQUEST_METHOD: `GET` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:16:12.260665 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97ede874-458a-4d0f-a185-8ef654f90cd2 None admin] SCRIPT_NAME: `/identity` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:16:12.261146 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97ede874-458a-4d0f-a185-8ef654f90cd2 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:16:12.262304 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97ede874-458a-4d0f-a185-8ef654f90cd2 None admin] RBAC: Adding query filter params (role.id=3a074738221e4601b5205715ba87d240, scope.project.id=eab7e5ec54c94574ad68c89747ad5bcc, user.id=39bf6631ce574c758a674faedecf2254) {{(pid=80245) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 08:16:12.262860 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97ede874-458a-4d0f-a185-8ef654f90cd2 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:16:12.265373 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97ede874-458a-4d0f-a185-8ef654f90cd2 None admin] RBAC: Authorization granted {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:16:12.273294 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: [pid: 80245|app: 0|req: 45/94] 10.222.0.47 () {56 vars in 1374 bytes} [Thu Jul 21 08:16:11 2022] GET /identity/v3/role_assignments?user.id=39bf6631ce574c758a674faedecf2254&scope.project.id=eab7e5ec54c94574ad68c89747ad5bcc&role.id=3a074738221e4601b5205715ba87d240 => generated 259 bytes in 301 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 08:16:12.274537 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97ede874-458a-4d0f-a185-8ef654f90cd2 None admin] Authenticating user token {{(pid=80245) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:16:12.286852 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG dbcounter [-] [80245] Writing DB stats keystone:SELECT=100 {{(pid=80245) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:16:12.332754 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4e2a9e6-ba2d-4339-98cd-d7ad0bd0bfba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:16:12.333447 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4e2a9e6-ba2d-4339-98cd-d7ad0bd0bfba None admin] REQUEST_METHOD: `GET` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:16:12.333735 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4e2a9e6-ba2d-4339-98cd-d7ad0bd0bfba None admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:16:12.334016 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4e2a9e6-ba2d-4339-98cd-d7ad0bd0bfba None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:16:12.335544 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4e2a9e6-ba2d-4339-98cd-d7ad0bd0bfba None admin] RBAC: Adding query filter params (role.id=99a638c0aac44b79a1f2858574724495, scope.project.id=eab7e5ec54c94574ad68c89747ad5bcc, user.id=39bf6631ce574c758a674faedecf2254) {{(pid=80244) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 08:16:12.335861 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4e2a9e6-ba2d-4339-98cd-d7ad0bd0bfba None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:16:12.337344 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4e2a9e6-ba2d-4339-98cd-d7ad0bd0bfba None admin] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:16:12.357226 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 50/95] 10.222.0.47 () {56 vars in 1374 bytes} [Thu Jul 21 08:16:12 2022] GET /identity/v3/role_assignments?user.id=39bf6631ce574c758a674faedecf2254&scope.project.id=eab7e5ec54c94574ad68c89747ad5bcc&role.id=99a638c0aac44b79a1f2858574724495 => generated 259 bytes in 245 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 08:16:12.398509 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97ede874-458a-4d0f-a185-8ef654f90cd2 None admin] Validating token access rules against request {{(pid=80245) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:16:12.442675 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa81afd4-533d-4d81-8a24-774f214a0c34 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80245) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:16:12.444419 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa81afd4-533d-4d81-8a24-774f214a0c34 None admin] REQUEST_METHOD: `GET` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:16:12.445002 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa81afd4-533d-4d81-8a24-774f214a0c34 None admin] SCRIPT_NAME: `/identity` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:16:12.445515 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa81afd4-533d-4d81-8a24-774f214a0c34 None admin] PATH_INFO: `/v3/projects/d77ff705f0ca44808214e0d3b6c4a1cf` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:16:12.447310 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa81afd4-533d-4d81-8a24-774f214a0c34 None admin] RBAC: Authorizing `identity:get_project(project_id=d77ff705f0ca44808214e0d3b6c4a1cf)` {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:16:12.449217 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa81afd4-533d-4d81-8a24-774f214a0c34 None admin] RBAC: Authorization granted {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:16:12.451521 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: [pid: 80245|app: 0|req: 46/96] 10.222.0.47 () {56 vars in 1186 bytes} [Thu Jul 21 08:16:12 2022] GET /identity/v3/projects/d77ff705f0ca44808214e0d3b6c4a1cf => generated 312 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 08:16:12.459465 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4e2a9e6-ba2d-4339-98cd-d7ad0bd0bfba None admin] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:16:12.605984 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG dbcounter [-] [80244] Writing DB stats keystone:SELECT=100 {{(pid=80244) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:16:12.615226 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4e2a9e6-ba2d-4339-98cd-d7ad0bd0bfba None admin] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:16:12.662997 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5b1ca1c-3778-49c3-b3b5-a00c92bb84e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:16:12.663614 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5b1ca1c-3778-49c3-b3b5-a00c92bb84e5 None admin] REQUEST_METHOD: `GET` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:16:12.663899 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5b1ca1c-3778-49c3-b3b5-a00c92bb84e5 None admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:16:12.664173 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5b1ca1c-3778-49c3-b3b5-a00c92bb84e5 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:16:12.665026 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5b1ca1c-3778-49c3-b3b5-a00c92bb84e5 None admin] RBAC: Adding query filter params (role.id=3a074738221e4601b5205715ba87d240, scope.project.id=d77ff705f0ca44808214e0d3b6c4a1cf, user.id=39bf6631ce574c758a674faedecf2254) {{(pid=80244) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 08:16:12.665338 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5b1ca1c-3778-49c3-b3b5-a00c92bb84e5 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:16:12.666844 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5b1ca1c-3778-49c3-b3b5-a00c92bb84e5 None admin] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:16:12.675510 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 51/97] 10.222.0.47 () {56 vars in 1374 bytes} [Thu Jul 21 08:16:12 2022] GET /identity/v3/role_assignments?user.id=39bf6631ce574c758a674faedecf2254&scope.project.id=d77ff705f0ca44808214e0d3b6c4a1cf&role.id=3a074738221e4601b5205715ba87d240 => generated 259 bytes in 218 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 08:16:13.332535 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29e5cad5-798e-4d4c-9dfb-d627b60c8bdd None None] REQUEST_METHOD: `GET` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:16:13.333572 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29e5cad5-798e-4d4c-9dfb-d627b60c8bdd None None] SCRIPT_NAME: `/identity` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:16:13.334102 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29e5cad5-798e-4d4c-9dfb-d627b60c8bdd None None] PATH_INFO: `/` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:16:13.336372 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: [pid: 80245|app: 0|req: 47/98] 10.222.0.47 () {54 vars in 926 bytes} [Thu Jul 21 08:16:13 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 08:16:13.342103 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9409f8ff-aa79-4a29-99bf-b5b3b97618ff None None] REQUEST_METHOD: `POST` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:16:13.342996 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9409f8ff-aa79-4a29-99bf-b5b3b97618ff None None] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:16:13.343509 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9409f8ff-aa79-4a29-99bf-b5b3b97618ff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:16:13.364326 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.auth.core [None req-9409f8ff-aa79-4a29-99bf-b5b3b97618ff None None] MFA Rules not processed for user `2dce40b8d74c41ec94cc36e6ff05d412`. Rule list: `[]` (Enabled: `True`). {{(pid=80244) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 08:16:13.365907 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.fernet_utils [None req-9409f8ff-aa79-4a29-99bf-b5b3b97618ff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80244) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 08:16:13.452718 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 52/99] 10.222.0.47 () {58 vars in 1023 bytes} [Thu Jul 21 08:16:13 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 21 08:16:13.509035 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29e5cad5-798e-4d4c-9dfb-d627b60c8bdd None None] Authenticating user token {{(pid=80245) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:16:13.608331 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59d72bc3-7b02-47f4-86f9-984550a3731d None None] REQUEST_METHOD: `GET` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:16:13.608636 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59d72bc3-7b02-47f4-86f9-984550a3731d None None] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:16:13.608824 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59d72bc3-7b02-47f4-86f9-984550a3731d None None] PATH_INFO: `/` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:16:13.610368 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 53/100] 10.222.0.47 () {54 vars in 926 bytes} [Thu Jul 21 08:16:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 21 08:16:13.617257 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG dbcounter [-] [80245] Writing DB stats keystone:SELECT=100 {{(pid=80245) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:16:13.619226 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8e71027-9eb6-431c-93bc-1f50a2920078 None None] REQUEST_METHOD: `POST` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:16:13.619544 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8e71027-9eb6-431c-93bc-1f50a2920078 None None] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:16:13.619827 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8e71027-9eb6-431c-93bc-1f50a2920078 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:16:13.656707 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.auth.core [None req-c8e71027-9eb6-431c-93bc-1f50a2920078 None None] MFA Rules not processed for user `2dce40b8d74c41ec94cc36e6ff05d412`. Rule list: `[]` (Enabled: `True`). {{(pid=80244) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 08:16:13.658220 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.fernet_utils [None req-c8e71027-9eb6-431c-93bc-1f50a2920078 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80244) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 08:16:13.672006 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29e5cad5-798e-4d4c-9dfb-d627b60c8bdd None None] Validating token access rules against request {{(pid=80245) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:16:13.752217 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f51285b-f152-44d2-8cea-be4c1178d14e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80245) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:16:13.753592 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f51285b-f152-44d2-8cea-be4c1178d14e None admin] REQUEST_METHOD: `GET` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:16:13.753592 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f51285b-f152-44d2-8cea-be4c1178d14e None admin] SCRIPT_NAME: `/identity` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:16:13.753592 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f51285b-f152-44d2-8cea-be4c1178d14e None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:16:13.785572 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 54/101] 10.222.0.47 () {58 vars in 1023 bytes} [Thu Jul 21 08:16:13 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 21 08:16:13.788471 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f51285b-f152-44d2-8cea-be4c1178d14e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 21 08:16:13.796337 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application [None req-2f51285b-f152-44d2-8cea-be4c1178d14e None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jul 21 08:16:13.796337 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 08:16:13.796337 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 08:16:13.796337 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 08:16:13.796337 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 08:16:13.796337 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 08:16:13.796337 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 08:16:13.796337 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 08:16:13.796337 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 08:16:13.796337 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 08:16:13.796337 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 08:16:13.796337 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 08:16:13.796337 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 21 08:16:13.796337 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 21 08:16:13.796337 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 21 08:16:13.796337 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 21 08:16:13.797143 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 21 08:16:13.797143 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 21 08:16:13.797143 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 21 08:16:13.797143 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 21 08:16:13.797143 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 08:16:13.797143 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 08:16:13.797143 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 08:16:13.797143 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 08:16:13.797143 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 08:16:13.797143 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 08:16:13.797143 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 08:16:13.797143 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application with Lock( Jul 21 08:16:13.797143 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 08:16:13.797143 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application return self._enter() Jul 21 08:16:13.797143 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 08:16:13.797143 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 08:16:13.797918 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 08:16:13.797918 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application return self.creator() Jul 21 08:16:13.797918 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 08:16:13.797918 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application created_value = creator( Jul 21 08:16:13.797918 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 21 08:16:13.797918 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 21 08:16:13.797918 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 21 08:16:13.797918 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 21 08:16:13.797918 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 21 08:16:13.797918 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 21 08:16:13.797918 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jul 21 08:16:13.797918 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application Jul 21 08:16:13.797918 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: [pid: 80245|app: 0|req: 48/102] 10.222.0.47 () {56 vars in 1099 bytes} [Thu Jul 21 08:16:13 2022] GET /identity/v3/roles/reader => generated 84 bytes in 289 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 08:16:13.805365 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8e71027-9eb6-431c-93bc-1f50a2920078 None None] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:16:13.847771 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG dbcounter [-] [80244] Writing DB stats keystone:SELECT=100 {{(pid=80244) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:16:13.868501 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f51285b-f152-44d2-8cea-be4c1178d14e None admin] Authenticating user token {{(pid=80245) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:16:13.946542 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8e71027-9eb6-431c-93bc-1f50a2920078 None None] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:16:14.034827 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-437bfbdd-4c32-4cde-a1ca-e520f411c8af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:16:14.035813 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-437bfbdd-4c32-4cde-a1ca-e520f411c8af None admin] REQUEST_METHOD: `GET` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:16:14.036250 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-437bfbdd-4c32-4cde-a1ca-e520f411c8af None admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:16:14.036665 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-437bfbdd-4c32-4cde-a1ca-e520f411c8af None admin] PATH_INFO: `/v3/roles` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:16:14.037553 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-437bfbdd-4c32-4cde-a1ca-e520f411c8af None admin] RBAC: Adding query filter params (name=reader) {{(pid=80244) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 08:16:14.038547 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-437bfbdd-4c32-4cde-a1ca-e520f411c8af None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:16:14.040119 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-437bfbdd-4c32-4cde-a1ca-e520f411c8af None admin] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:16:14.065346 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 55/103] 10.222.0.47 () {56 vars in 1101 bytes} [Thu Jul 21 08:16:13 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 260 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 08:16:14.071246 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-437bfbdd-4c32-4cde-a1ca-e520f411c8af None admin] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:16:14.078519 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f51285b-f152-44d2-8cea-be4c1178d14e None admin] Validating token access rules against request {{(pid=80245) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:16:14.135844 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6756f42-6857-4c5b-ab45-5241c3f40d36 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80245) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:16:14.136673 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6756f42-6857-4c5b-ab45-5241c3f40d36 None admin] REQUEST_METHOD: `GET` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:16:14.136965 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6756f42-6857-4c5b-ab45-5241c3f40d36 None admin] SCRIPT_NAME: `/identity` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:16:14.137244 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6756f42-6857-4c5b-ab45-5241c3f40d36 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:16:14.138532 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6756f42-6857-4c5b-ab45-5241c3f40d36 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:16:14.145918 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6756f42-6857-4c5b-ab45-5241c3f40d36 None admin] RBAC: Authorization granted {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:16:14.145918 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: [pid: 80245|app: 0|req: 49/104] 10.222.0.47 () {56 vars in 1108 bytes} [Thu Jul 21 08:16:13 2022] GET /identity/v3/domains/default => generated 199 bytes in 274 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 08:16:14.148460 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6756f42-6857-4c5b-ab45-5241c3f40d36 None admin] Authenticating user token {{(pid=80245) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:16:14.155628 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG dbcounter [-] [80245] Writing DB stats keystone:SELECT=100 {{(pid=80245) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:16:14.248933 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG dbcounter [-] [80244] Writing DB stats keystone:SELECT=100 {{(pid=80244) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:16:14.303080 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-437bfbdd-4c32-4cde-a1ca-e520f411c8af None admin] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:16:14.361624 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6756f42-6857-4c5b-ab45-5241c3f40d36 None admin] Validating token access rules against request {{(pid=80245) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:16:14.385198 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb39ca3d-08a7-419b-bcba-a162859638b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:16:14.386765 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb39ca3d-08a7-419b-bcba-a162859638b3 None admin] REQUEST_METHOD: `GET` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:16:14.387203 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb39ca3d-08a7-419b-bcba-a162859638b3 None admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:16:14.387647 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb39ca3d-08a7-419b-bcba-a162859638b3 None admin] PATH_INFO: `/v3/users/62835f20a4f74cb8a96c4979d095b168` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:16:14.417778 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb39ca3d-08a7-419b-bcba-a162859638b3 None admin] RBAC: Authorizing `identity:get_user(user_id=62835f20a4f74cb8a96c4979d095b168)` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:16:14.419555 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb39ca3d-08a7-419b-bcba-a162859638b3 None admin] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:16:14.423418 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 56/105] 10.222.0.47 () {56 vars in 1177 bytes} [Thu Jul 21 08:16:14 2022] GET /identity/v3/users/62835f20a4f74cb8a96c4979d095b168 => generated 289 bytes in 351 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 08:16:14.430406 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb39ca3d-08a7-419b-bcba-a162859638b3 None admin] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:16:14.441086 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39c0b4bb-22ef-4200-99b2-0e8da86f0149 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80245) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:16:14.441776 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39c0b4bb-22ef-4200-99b2-0e8da86f0149 None admin] REQUEST_METHOD: `POST` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:16:14.442066 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39c0b4bb-22ef-4200-99b2-0e8da86f0149 None admin] SCRIPT_NAME: `/identity` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:16:14.442348 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39c0b4bb-22ef-4200-99b2-0e8da86f0149 None admin] PATH_INFO: `/v3/projects` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:16:14.443071 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39c0b4bb-22ef-4200-99b2-0e8da86f0149 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:16:14.445110 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39c0b4bb-22ef-4200-99b2-0e8da86f0149 None admin] RBAC: Authorization granted {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:16:14.499888 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: [pid: 80245|app: 0|req: 50/106] 10.222.0.47 () {60 vars in 1141 bytes} [Thu Jul 21 08:16:14 2022] POST /identity/v3/projects => generated 302 bytes in 352 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 08:16:14.593132 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84596072-12ed-422f-941d-cdfb52eaf8dd None None] REQUEST_METHOD: `GET` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:16:14.593427 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84596072-12ed-422f-941d-cdfb52eaf8dd None None] SCRIPT_NAME: `/identity` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:16:14.593711 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84596072-12ed-422f-941d-cdfb52eaf8dd None None] PATH_INFO: `/` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:16:14.595653 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: [pid: 80245|app: 0|req: 51/107] 10.222.0.47 () {54 vars in 926 bytes} [Thu Jul 21 08:16:14 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 08:16:14.603760 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-329ec9e4-2c24-4f0b-9749-440200efbe7a None None] REQUEST_METHOD: `POST` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:16:14.604052 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-329ec9e4-2c24-4f0b-9749-440200efbe7a None None] SCRIPT_NAME: `/identity` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:16:14.604449 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-329ec9e4-2c24-4f0b-9749-440200efbe7a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:16:14.634189 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb39ca3d-08a7-419b-bcba-a162859638b3 None admin] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:16:14.634913 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.auth.core [None req-329ec9e4-2c24-4f0b-9749-440200efbe7a None None] MFA Rules not processed for user `2dce40b8d74c41ec94cc36e6ff05d412`. Rule list: `[]` (Enabled: `True`). {{(pid=80245) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 08:16:14.636265 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.fernet_utils [None req-329ec9e4-2c24-4f0b-9749-440200efbe7a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80245) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 08:16:14.729074 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG dbcounter [-] [80245] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80245) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:16:14.730041 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4baec35b-c003-411f-b680-c9dfc47d991b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:16:14.731310 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4baec35b-c003-411f-b680-c9dfc47d991b None admin] REQUEST_METHOD: `GET` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:16:14.731856 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4baec35b-c003-411f-b680-c9dfc47d991b None admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:16:14.732443 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4baec35b-c003-411f-b680-c9dfc47d991b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:16:14.763515 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: [pid: 80245|app: 0|req: 52/108] 10.222.0.47 () {58 vars in 1023 bytes} [Thu Jul 21 08:16:14 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 160 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 21 08:16:14.764319 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG dbcounter [-] [80244] Writing DB stats keystone:SELECT=100 {{(pid=80244) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:16:14.792091 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-329ec9e4-2c24-4f0b-9749-440200efbe7a None None] Authenticating user token {{(pid=80245) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:16:14.842333 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4baec35b-c003-411f-b680-c9dfc47d991b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:16:14.844170 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4baec35b-c003-411f-b680-c9dfc47d991b None admin] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:16:14.952813 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 57/109] 10.222.0.47 () {58 vars in 1282 bytes} [Thu Jul 21 08:16:14 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 523 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Jul 21 08:16:14.963129 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4baec35b-c003-411f-b680-c9dfc47d991b None admin] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:16:14.993619 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-329ec9e4-2c24-4f0b-9749-440200efbe7a None None] Validating token access rules against request {{(pid=80245) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:16:15.035591 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG dbcounter [-] [80244] Writing DB stats keystone:SELECT=100 {{(pid=80244) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:16:15.068521 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e45b6d00-501e-4399-9c5c-8d7896b8128e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80245) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:16:15.069650 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e45b6d00-501e-4399-9c5c-8d7896b8128e None admin] REQUEST_METHOD: `GET` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:16:15.070071 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e45b6d00-501e-4399-9c5c-8d7896b8128e None admin] SCRIPT_NAME: `/identity` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:16:15.070485 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e45b6d00-501e-4399-9c5c-8d7896b8128e None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:16:15.091789 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e45b6d00-501e-4399-9c5c-8d7896b8128e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 21 08:16:15.106159 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application [None req-e45b6d00-501e-4399-9c5c-8d7896b8128e None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 21 08:16:15.106159 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 08:16:15.106159 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 08:16:15.106159 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 08:16:15.106159 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 08:16:15.106159 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 08:16:15.106159 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 08:16:15.106159 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 08:16:15.106159 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 08:16:15.106159 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 08:16:15.106159 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 08:16:15.106159 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 08:16:15.106159 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 21 08:16:15.106159 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 21 08:16:15.106159 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 21 08:16:15.106159 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 21 08:16:15.107201 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 21 08:16:15.107201 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 21 08:16:15.107201 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 21 08:16:15.107201 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 21 08:16:15.107201 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 08:16:15.107201 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 08:16:15.107201 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 08:16:15.107201 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 08:16:15.107201 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 08:16:15.107201 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 08:16:15.107201 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 08:16:15.107201 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application with Lock( Jul 21 08:16:15.107201 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 08:16:15.107201 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application return self._enter() Jul 21 08:16:15.107201 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 08:16:15.107201 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 08:16:15.108098 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 08:16:15.108098 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application return self.creator() Jul 21 08:16:15.108098 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 08:16:15.108098 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application created_value = creator( Jul 21 08:16:15.108098 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 21 08:16:15.108098 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 21 08:16:15.108098 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 21 08:16:15.108098 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 21 08:16:15.108098 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 21 08:16:15.108098 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 21 08:16:15.108098 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 21 08:16:15.108098 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application Jul 21 08:16:15.115111 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: [pid: 80245|app: 0|req: 53/110] 10.222.0.47 () {56 vars in 1096 bytes} [Thu Jul 21 08:16:14 2022] GET /identity/v3/roles/admin => generated 83 bytes in 324 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 21 08:16:15.124388 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e45b6d00-501e-4399-9c5c-8d7896b8128e None admin] Authenticating user token {{(pid=80245) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:16:15.205556 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4baec35b-c003-411f-b680-c9dfc47d991b None admin] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:16:15.210342 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG dbcounter [-] [80245] Writing DB stats keystone:SELECT=100 {{(pid=80245) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:16:15.253872 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9faf111f-1e26-44bd-a77a-dde8bdef95d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:16:15.259345 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9faf111f-1e26-44bd-a77a-dde8bdef95d5 None admin] REQUEST_METHOD: `GET` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:16:15.259345 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9faf111f-1e26-44bd-a77a-dde8bdef95d5 None admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:16:15.259345 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9faf111f-1e26-44bd-a77a-dde8bdef95d5 None admin] PATH_INFO: `/v3/projects/eab7e5ec54c94574ad68c89747ad5bcc` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:16:15.259345 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9faf111f-1e26-44bd-a77a-dde8bdef95d5 None admin] RBAC: Authorizing `identity:get_project(project_id=eab7e5ec54c94574ad68c89747ad5bcc)` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:16:15.259345 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9faf111f-1e26-44bd-a77a-dde8bdef95d5 None admin] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:16:15.261357 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 58/111] 10.222.0.47 () {56 vars in 1186 bytes} [Thu Jul 21 08:16:14 2022] GET /identity/v3/projects/eab7e5ec54c94574ad68c89747ad5bcc => generated 298 bytes in 299 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 08:16:15.262873 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1d00494-376a-4a60-9927-5b93686cb6bb None None] REQUEST_METHOD: `GET` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:16:15.263285 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1d00494-376a-4a60-9927-5b93686cb6bb None None] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:16:15.263617 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1d00494-376a-4a60-9927-5b93686cb6bb None None] PATH_INFO: `/` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:16:15.265148 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 59/112] 10.222.0.47 () {54 vars in 926 bytes} [Thu Jul 21 08:16:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 08:16:15.271255 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1d00494-376a-4a60-9927-5b93686cb6bb None None] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:16:15.336632 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e45b6d00-501e-4399-9c5c-8d7896b8128e None admin] Validating token access rules against request {{(pid=80245) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:16:15.415987 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8d95a51-9fcd-4a5a-b36a-055c6d3e124f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80245) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:16:15.416761 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8d95a51-9fcd-4a5a-b36a-055c6d3e124f None admin] REQUEST_METHOD: `GET` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:16:15.416965 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8d95a51-9fcd-4a5a-b36a-055c6d3e124f None admin] SCRIPT_NAME: `/identity` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:16:15.417248 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8d95a51-9fcd-4a5a-b36a-055c6d3e124f None admin] PATH_INFO: `/v3/roles` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:16:15.418012 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8d95a51-9fcd-4a5a-b36a-055c6d3e124f None admin] RBAC: Adding query filter params (name=admin) {{(pid=80245) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 08:16:15.418324 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8d95a51-9fcd-4a5a-b36a-055c6d3e124f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:16:15.419769 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8d95a51-9fcd-4a5a-b36a-055c6d3e124f None admin] RBAC: Authorization granted {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:16:15.442781 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: [pid: 80245|app: 0|req: 54/113] 10.222.0.47 () {56 vars in 1099 bytes} [Thu Jul 21 08:16:15 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 319 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 08:16:15.447406 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-931d4c4c-2fb5-46ca-a563-82210eb94463 None None] REQUEST_METHOD: `POST` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:16:15.448768 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-931d4c4c-2fb5-46ca-a563-82210eb94463 None None] SCRIPT_NAME: `/identity` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:16:15.449803 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-931d4c4c-2fb5-46ca-a563-82210eb94463 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:16:15.465409 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1d00494-376a-4a60-9927-5b93686cb6bb None None] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:16:15.472996 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG dbcounter [-] [80244] Writing DB stats keystone:SELECT=100 {{(pid=80244) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:16:15.488724 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.auth.core [None req-931d4c4c-2fb5-46ca-a563-82210eb94463 None None] MFA Rules not processed for user `2dce40b8d74c41ec94cc36e6ff05d412`. Rule list: `[]` (Enabled: `True`). {{(pid=80245) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 08:16:15.490203 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.fernet_utils [None req-931d4c4c-2fb5-46ca-a563-82210eb94463 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80245) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 08:16:15.525117 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da15b3e7-ee44-4c0d-8f9c-8bf2b8aab9e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:16:15.525718 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da15b3e7-ee44-4c0d-8f9c-8bf2b8aab9e6 None admin] REQUEST_METHOD: `GET` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:16:15.526005 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da15b3e7-ee44-4c0d-8f9c-8bf2b8aab9e6 None admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:16:15.526286 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da15b3e7-ee44-4c0d-8f9c-8bf2b8aab9e6 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:16:15.527143 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da15b3e7-ee44-4c0d-8f9c-8bf2b8aab9e6 None admin] RBAC: Adding query filter params (role.id=9e5171884e9f401ab2337b74db49546e, scope.project.id=eab7e5ec54c94574ad68c89747ad5bcc, user.id=62835f20a4f74cb8a96c4979d095b168) {{(pid=80244) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 08:16:15.527488 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da15b3e7-ee44-4c0d-8f9c-8bf2b8aab9e6 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:16:15.529018 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da15b3e7-ee44-4c0d-8f9c-8bf2b8aab9e6 None admin] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:16:15.535092 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 60/114] 10.222.0.47 () {56 vars in 1374 bytes} [Thu Jul 21 08:16:15 2022] GET /identity/v3/role_assignments?user.id=62835f20a4f74cb8a96c4979d095b168&scope.project.id=eab7e5ec54c94574ad68c89747ad5bcc&role.id=9e5171884e9f401ab2337b74db49546e => generated 259 bytes in 265 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 08:16:15.536138 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da15b3e7-ee44-4c0d-8f9c-8bf2b8aab9e6 None admin] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:16:15.572240 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: [pid: 80245|app: 0|req: 55/115] 10.222.0.47 () {58 vars in 1023 bytes} [Thu Jul 21 08:16:15 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 21 08:16:15.573796 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70687a6f-4271-49b2-b627-08e5810238bc None None] REQUEST_METHOD: `GET` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:16:15.574094 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70687a6f-4271-49b2-b627-08e5810238bc None None] SCRIPT_NAME: `/identity` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:16:15.574378 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70687a6f-4271-49b2-b627-08e5810238bc None None] PATH_INFO: `/` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:16:15.575821 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: [pid: 80245|app: 0|req: 56/116] 10.222.0.47 () {54 vars in 926 bytes} [Thu Jul 21 08:16:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 08:16:15.585692 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cf8702d-7c58-4932-9e12-fc7250f42648 None None] REQUEST_METHOD: `POST` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:16:15.586008 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cf8702d-7c58-4932-9e12-fc7250f42648 None None] SCRIPT_NAME: `/identity` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:16:15.586252 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cf8702d-7c58-4932-9e12-fc7250f42648 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:16:15.611729 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.auth.core [None req-0cf8702d-7c58-4932-9e12-fc7250f42648 None None] MFA Rules not processed for user `2dce40b8d74c41ec94cc36e6ff05d412`. Rule list: `[]` (Enabled: `True`). {{(pid=80245) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 08:16:15.612967 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.fernet_utils [None req-0cf8702d-7c58-4932-9e12-fc7250f42648 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80245) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 08:16:15.649548 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG dbcounter [-] [80245] Writing DB stats keystone:SELECT=100 {{(pid=80245) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:16:15.665032 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da15b3e7-ee44-4c0d-8f9c-8bf2b8aab9e6 None admin] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:16:15.692437 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: [pid: 80245|app: 0|req: 57/117] 10.222.0.47 () {58 vars in 1023 bytes} [Thu Jul 21 08:16:15 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 21 08:16:15.693890 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd0c8e86-0644-4c3e-ba45-e2d80c8d062b None None] REQUEST_METHOD: `GET` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:16:15.694678 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd0c8e86-0644-4c3e-ba45-e2d80c8d062b None None] SCRIPT_NAME: `/identity` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:16:15.694964 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd0c8e86-0644-4c3e-ba45-e2d80c8d062b None None] PATH_INFO: `/` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:16:15.704678 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: [pid: 80245|app: 0|req: 58/118] 10.222.0.47 () {54 vars in 926 bytes} [Thu Jul 21 08:16:15 2022] GET /identity => generated 270 bytes in 12 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 08:16:15.707376 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd0c8e86-0644-4c3e-ba45-e2d80c8d062b None None] Authenticating user token {{(pid=80245) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:16:15.711011 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f68062a-c15d-45aa-afa1-f30ad63a0e30 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:16:15.711970 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f68062a-c15d-45aa-afa1-f30ad63a0e30 None admin] REQUEST_METHOD: `GET` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:16:15.712258 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f68062a-c15d-45aa-afa1-f30ad63a0e30 None admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:16:15.712538 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f68062a-c15d-45aa-afa1-f30ad63a0e30 None admin] PATH_INFO: `/v3/users/2dce40b8d74c41ec94cc36e6ff05d412` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:16:15.713911 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f68062a-c15d-45aa-afa1-f30ad63a0e30 None admin] RBAC: Authorizing `identity:get_user(user_id=2dce40b8d74c41ec94cc36e6ff05d412)` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:16:15.715450 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f68062a-c15d-45aa-afa1-f30ad63a0e30 None admin] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:16:15.717341 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 61/119] 10.222.0.47 () {56 vars in 1177 bytes} [Thu Jul 21 08:16:15 2022] GET /identity/v3/users/2dce40b8d74c41ec94cc36e6ff05d412 => generated 247 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 08:16:15.718590 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfd3c3dd-69d5-423f-ba14-986ee53bda0d None None] REQUEST_METHOD: `POST` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:16:15.718884 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfd3c3dd-69d5-423f-ba14-986ee53bda0d None None] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:16:15.719165 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfd3c3dd-69d5-423f-ba14-986ee53bda0d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:16:15.737984 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.auth.core [None req-bfd3c3dd-69d5-423f-ba14-986ee53bda0d None None] MFA Rules not processed for user `2dce40b8d74c41ec94cc36e6ff05d412`. Rule list: `[]` (Enabled: `True`). {{(pid=80244) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 08:16:15.739296 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.fernet_utils [None req-bfd3c3dd-69d5-423f-ba14-986ee53bda0d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80244) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 08:16:15.800158 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG dbcounter [-] [80244] Writing DB stats keystone:SELECT=100 {{(pid=80244) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:16:15.834489 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 62/120] 10.222.0.47 () {58 vars in 1023 bytes} [Thu Jul 21 08:16:15 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 21 08:16:15.843436 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfd3c3dd-69d5-423f-ba14-986ee53bda0d None None] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:16:15.871869 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd0c8e86-0644-4c3e-ba45-e2d80c8d062b None None] Validating token access rules against request {{(pid=80245) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:16:15.911974 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce539d65-d23d-4dd2-8456-0c26961d7aa6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80245) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:16:15.913399 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce539d65-d23d-4dd2-8456-0c26961d7aa6 None admin] REQUEST_METHOD: `GET` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:16:15.913692 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce539d65-d23d-4dd2-8456-0c26961d7aa6 None admin] SCRIPT_NAME: `/identity` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:16:15.913975 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce539d65-d23d-4dd2-8456-0c26961d7aa6 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:16:15.925880 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce539d65-d23d-4dd2-8456-0c26961d7aa6 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 21 08:16:15.935206 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application [None req-ce539d65-d23d-4dd2-8456-0c26961d7aa6 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 21 08:16:15.935206 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 08:16:15.935206 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 08:16:15.935206 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 08:16:15.935206 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 08:16:15.935206 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 08:16:15.935206 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 08:16:15.935206 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 08:16:15.935206 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 08:16:15.935206 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 08:16:15.935206 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 08:16:15.935206 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 08:16:15.935206 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 21 08:16:15.935206 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 21 08:16:15.935206 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 21 08:16:15.935206 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 21 08:16:15.936333 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 21 08:16:15.936333 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 21 08:16:15.936333 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 21 08:16:15.936333 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 21 08:16:15.936333 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 08:16:15.936333 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 08:16:15.936333 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 08:16:15.936333 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 08:16:15.936333 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 08:16:15.936333 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 08:16:15.936333 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 08:16:15.936333 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application with Lock( Jul 21 08:16:15.936333 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 08:16:15.936333 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application return self._enter() Jul 21 08:16:15.936333 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 08:16:15.936333 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 08:16:15.937299 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 08:16:15.937299 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application return self.creator() Jul 21 08:16:15.937299 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 08:16:15.937299 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application created_value = creator( Jul 21 08:16:15.937299 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 21 08:16:15.937299 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 21 08:16:15.937299 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 21 08:16:15.937299 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 21 08:16:15.937299 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 21 08:16:15.937299 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 21 08:16:15.937299 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 21 08:16:15.937299 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application Jul 21 08:16:15.938702 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: [pid: 80245|app: 0|req: 59/121] 10.222.0.47 () {56 vars in 1099 bytes} [Thu Jul 21 08:16:15 2022] GET /identity/v3/roles/member => generated 84 bytes in 233 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 08:16:15.939571 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce539d65-d23d-4dd2-8456-0c26961d7aa6 None admin] Authenticating user token {{(pid=80245) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:16:15.970480 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG dbcounter [-] [80245] Writing DB stats keystone:SELECT=100 {{(pid=80245) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:16:15.970661 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfd3c3dd-69d5-423f-ba14-986ee53bda0d None None] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:16:16.009090 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c9b5375-deee-4ff2-ada8-758d55739acf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:16:16.010108 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c9b5375-deee-4ff2-ada8-758d55739acf None admin] REQUEST_METHOD: `GET` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:16:16.010397 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c9b5375-deee-4ff2-ada8-758d55739acf None admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:16:16.010680 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c9b5375-deee-4ff2-ada8-758d55739acf None admin] PATH_INFO: `/v3/projects/eab7e5ec54c94574ad68c89747ad5bcc` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:16:16.012586 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c9b5375-deee-4ff2-ada8-758d55739acf None admin] RBAC: Authorizing `identity:get_project(project_id=eab7e5ec54c94574ad68c89747ad5bcc)` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:16:16.014333 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c9b5375-deee-4ff2-ada8-758d55739acf None admin] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:16:16.016486 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 63/122] 10.222.0.47 () {56 vars in 1186 bytes} [Thu Jul 21 08:16:15 2022] GET /identity/v3/projects/eab7e5ec54c94574ad68c89747ad5bcc => generated 298 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 08:16:16.017854 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c9b5375-deee-4ff2-ada8-758d55739acf None admin] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:16:16.055407 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce539d65-d23d-4dd2-8456-0c26961d7aa6 None admin] Validating token access rules against request {{(pid=80245) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:16:16.062246 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG dbcounter [-] [80244] Writing DB stats keystone:SELECT=100 {{(pid=80244) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:16:16.097280 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aac8aa4-9b1f-4420-a794-9a78a4b14847 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80245) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:16:16.098493 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aac8aa4-9b1f-4420-a794-9a78a4b14847 None admin] REQUEST_METHOD: `GET` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:16:16.099042 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aac8aa4-9b1f-4420-a794-9a78a4b14847 None admin] SCRIPT_NAME: `/identity` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:16:16.099559 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aac8aa4-9b1f-4420-a794-9a78a4b14847 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:16:16.111216 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5aac8aa4-9b1f-4420-a794-9a78a4b14847 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 21 08:16:16.120940 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application [None req-5aac8aa4-9b1f-4420-a794-9a78a4b14847 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 21 08:16:16.120940 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 08:16:16.120940 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 08:16:16.120940 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 08:16:16.120940 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 08:16:16.120940 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 08:16:16.120940 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 08:16:16.120940 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 08:16:16.120940 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 08:16:16.120940 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 08:16:16.120940 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 08:16:16.120940 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 08:16:16.120940 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 21 08:16:16.120940 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 21 08:16:16.120940 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 21 08:16:16.120940 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 21 08:16:16.121793 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 21 08:16:16.121793 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 21 08:16:16.121793 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 21 08:16:16.121793 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 21 08:16:16.121793 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 08:16:16.121793 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 08:16:16.121793 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 08:16:16.121793 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 08:16:16.121793 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 08:16:16.121793 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 08:16:16.121793 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 08:16:16.121793 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application with Lock( Jul 21 08:16:16.121793 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 08:16:16.121793 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application return self._enter() Jul 21 08:16:16.121793 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 08:16:16.121793 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 08:16:16.122629 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 08:16:16.122629 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application return self.creator() Jul 21 08:16:16.122629 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 08:16:16.122629 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application created_value = creator( Jul 21 08:16:16.122629 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 21 08:16:16.122629 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 21 08:16:16.122629 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 21 08:16:16.122629 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 21 08:16:16.122629 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 21 08:16:16.122629 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 21 08:16:16.122629 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 21 08:16:16.122629 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application Jul 21 08:16:16.125132 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: [pid: 80245|app: 0|req: 60/123] 10.222.0.47 () {56 vars in 1114 bytes} [Thu Jul 21 08:16:15 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 08:16:16.126150 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aac8aa4-9b1f-4420-a794-9a78a4b14847 None admin] Authenticating user token {{(pid=80245) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:16:16.141083 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c9b5375-deee-4ff2-ada8-758d55739acf None admin] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:16:16.182845 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5ed9765-a770-40c5-8b5c-c845d426e4c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:16:16.183809 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5ed9765-a770-40c5-8b5c-c845d426e4c5 None admin] REQUEST_METHOD: `GET` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:16:16.184080 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5ed9765-a770-40c5-8b5c-c845d426e4c5 None admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:16:16.184367 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5ed9765-a770-40c5-8b5c-c845d426e4c5 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:16:16.202802 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5ed9765-a770-40c5-8b5c-c845d426e4c5 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 21 08:16:16.213459 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application [None req-b5ed9765-a770-40c5-8b5c-c845d426e4c5 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 21 08:16:16.213459 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 08:16:16.213459 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 08:16:16.213459 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 08:16:16.213459 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 08:16:16.213459 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 08:16:16.213459 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 08:16:16.213459 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 08:16:16.213459 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 08:16:16.213459 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 08:16:16.213459 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 08:16:16.213459 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 08:16:16.213459 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 21 08:16:16.213459 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 21 08:16:16.213459 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 21 08:16:16.213459 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 21 08:16:16.214371 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 21 08:16:16.214371 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 21 08:16:16.214371 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 21 08:16:16.214371 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 21 08:16:16.214371 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 08:16:16.214371 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 08:16:16.214371 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 08:16:16.214371 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 08:16:16.214371 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 08:16:16.214371 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 08:16:16.214371 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 08:16:16.214371 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application with Lock( Jul 21 08:16:16.214371 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 08:16:16.214371 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self._enter() Jul 21 08:16:16.214371 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 08:16:16.214371 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 08:16:16.215123 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 08:16:16.215123 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self.creator() Jul 21 08:16:16.215123 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 08:16:16.215123 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application created_value = creator( Jul 21 08:16:16.215123 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 21 08:16:16.215123 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 21 08:16:16.215123 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 21 08:16:16.215123 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 21 08:16:16.215123 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 21 08:16:16.215123 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 21 08:16:16.215123 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 21 08:16:16.215123 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application Jul 21 08:16:16.215123 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 64/124] 10.222.0.47 () {56 vars in 1099 bytes} [Thu Jul 21 08:16:16 2022] GET /identity/v3/roles/member => generated 84 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 08:16:16.215766 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5ed9765-a770-40c5-8b5c-c845d426e4c5 None admin] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:16:16.236208 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG dbcounter [-] [80245] Writing DB stats keystone:SELECT=100 {{(pid=80245) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:16:16.260510 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aac8aa4-9b1f-4420-a794-9a78a4b14847 None admin] Validating token access rules against request {{(pid=80245) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:16:16.308049 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28acb7a2-e928-4cd8-9e89-fbe3882b09d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80245) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:16:16.309391 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28acb7a2-e928-4cd8-9e89-fbe3882b09d2 None admin] REQUEST_METHOD: `GET` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:16:16.310080 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28acb7a2-e928-4cd8-9e89-fbe3882b09d2 None admin] SCRIPT_NAME: `/identity` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:16:16.310502 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28acb7a2-e928-4cd8-9e89-fbe3882b09d2 None admin] PATH_INFO: `/v3/roles` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:16:16.311667 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28acb7a2-e928-4cd8-9e89-fbe3882b09d2 None admin] RBAC: Adding query filter params (name=member) {{(pid=80245) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 08:16:16.312490 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28acb7a2-e928-4cd8-9e89-fbe3882b09d2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:16:16.314064 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28acb7a2-e928-4cd8-9e89-fbe3882b09d2 None admin] RBAC: Authorization granted {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:16:16.323394 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG dbcounter [-] [80244] Writing DB stats keystone:SELECT=100 {{(pid=80244) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:16:16.325380 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: [pid: 80245|app: 0|req: 61/125] 10.222.0.47 () {56 vars in 1101 bytes} [Thu Jul 21 08:16:16 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 08:16:16.331136 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28acb7a2-e928-4cd8-9e89-fbe3882b09d2 None admin] Authenticating user token {{(pid=80245) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:16:16.332108 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5ed9765-a770-40c5-8b5c-c845d426e4c5 None admin] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:16:16.380172 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e16c187-8e09-48e3-8cf1-ec316d204509 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:16:16.381655 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e16c187-8e09-48e3-8cf1-ec316d204509 None admin] REQUEST_METHOD: `PUT` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:16:16.382083 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e16c187-8e09-48e3-8cf1-ec316d204509 None admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:16:16.382792 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e16c187-8e09-48e3-8cf1-ec316d204509 None admin] PATH_INFO: `/v3/projects/eab7e5ec54c94574ad68c89747ad5bcc/users/2dce40b8d74c41ec94cc36e6ff05d412/roles/e29870f6e9e2411a9db0d5bf45f32ca5` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:16:16.386015 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e16c187-8e09-48e3-8cf1-ec316d204509 None admin] RBAC: Authorizing `identity:create_grant(project_id=eab7e5ec54c94574ad68c89747ad5bcc, user_id=2dce40b8d74c41ec94cc36e6ff05d412, role_id=e29870f6e9e2411a9db0d5bf45f32ca5)` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:16:16.388392 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e16c187-8e09-48e3-8cf1-ec316d204509 None admin] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:16:16.406118 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 65/126] 10.222.0.47 () {58 vars in 1439 bytes} [Thu Jul 21 08:16:16 2022] PUT /identity/v3/projects/eab7e5ec54c94574ad68c89747ad5bcc/users/2dce40b8d74c41ec94cc36e6ff05d412/roles/e29870f6e9e2411a9db0d5bf45f32ca5 => generated 0 bytes in 191 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 08:16:16.407732 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e16c187-8e09-48e3-8cf1-ec316d204509 None admin] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:16:16.449289 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28acb7a2-e928-4cd8-9e89-fbe3882b09d2 None admin] Validating token access rules against request {{(pid=80245) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:16:16.493574 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-229e11b5-371c-4bdf-92cc-095549c3f6a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80245) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:16:16.494291 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-229e11b5-371c-4bdf-92cc-095549c3f6a6 None admin] REQUEST_METHOD: `GET` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:16:16.494764 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-229e11b5-371c-4bdf-92cc-095549c3f6a6 None admin] SCRIPT_NAME: `/identity` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:16:16.495024 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-229e11b5-371c-4bdf-92cc-095549c3f6a6 None admin] PATH_INFO: `/v3/roles` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:16:16.495843 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-229e11b5-371c-4bdf-92cc-095549c3f6a6 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80245) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 08:16:16.496488 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-229e11b5-371c-4bdf-92cc-095549c3f6a6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:16:16.497873 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-229e11b5-371c-4bdf-92cc-095549c3f6a6 None admin] RBAC: Authorization granted {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:16:16.510152 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: [pid: 80245|app: 0|req: 62/127] 10.222.0.47 () {56 vars in 1111 bytes} [Thu Jul 21 08:16:16 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 08:16:16.510152 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-229e11b5-371c-4bdf-92cc-095549c3f6a6 None admin] Authenticating user token {{(pid=80245) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:16:16.524594 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG dbcounter [-] [80245] Writing DB stats keystone:SELECT=100 {{(pid=80245) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:16:16.555012 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e16c187-8e09-48e3-8cf1-ec316d204509 None admin] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:16:16.634694 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-229e11b5-371c-4bdf-92cc-095549c3f6a6 None admin] Validating token access rules against request {{(pid=80245) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:16:16.637723 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6744a183-cb9c-4bdb-8d66-201e3742d273 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:16:16.638470 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6744a183-cb9c-4bdb-8d66-201e3742d273 None admin] REQUEST_METHOD: `GET` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:16:16.638770 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6744a183-cb9c-4bdb-8d66-201e3742d273 None admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:16:16.639054 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6744a183-cb9c-4bdb-8d66-201e3742d273 None admin] PATH_INFO: `/v3/roles` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:16:16.639914 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6744a183-cb9c-4bdb-8d66-201e3742d273 None admin] RBAC: Adding query filter params (name=member) {{(pid=80244) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 08:16:16.640234 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6744a183-cb9c-4bdb-8d66-201e3742d273 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:16:16.641671 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6744a183-cb9c-4bdb-8d66-201e3742d273 None admin] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:16:16.656453 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 66/128] 10.222.0.47 () {56 vars in 1101 bytes} [Thu Jul 21 08:16:16 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 249 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 08:16:16.657389 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6744a183-cb9c-4bdb-8d66-201e3742d273 None admin] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:16:16.677862 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-327b8cd4-49d7-41e9-9832-b5e8f6620847 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80245) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:16:16.679451 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-327b8cd4-49d7-41e9-9832-b5e8f6620847 None admin] REQUEST_METHOD: `GET` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:16:16.679734 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-327b8cd4-49d7-41e9-9832-b5e8f6620847 None admin] SCRIPT_NAME: `/identity` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:16:16.680011 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-327b8cd4-49d7-41e9-9832-b5e8f6620847 None admin] PATH_INFO: `/v3/users/39bf6631ce574c758a674faedecf2254` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:16:16.680795 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG dbcounter [-] [80244] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80244) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:16:16.682397 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-327b8cd4-49d7-41e9-9832-b5e8f6620847 None admin] RBAC: Authorizing `identity:get_user(user_id=39bf6631ce574c758a674faedecf2254)` {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:16:16.684245 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-327b8cd4-49d7-41e9-9832-b5e8f6620847 None admin] RBAC: Authorization granted {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:16:16.686934 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: [pid: 80245|app: 0|req: 63/129] 10.222.0.47 () {56 vars in 1177 bytes} [Thu Jul 21 08:16:16 2022] GET /identity/v3/users/39bf6631ce574c758a674faedecf2254 => generated 275 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 08:16:16.687812 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-327b8cd4-49d7-41e9-9832-b5e8f6620847 None admin] Authenticating user token {{(pid=80245) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:16:16.775348 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6744a183-cb9c-4bdb-8d66-201e3742d273 None admin] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:16:16.814169 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c70fdefc-620f-4bc4-91fa-b83945fba6b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:16:16.815452 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c70fdefc-620f-4bc4-91fa-b83945fba6b1 None admin] REQUEST_METHOD: `GET` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:16:16.817065 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c70fdefc-620f-4bc4-91fa-b83945fba6b1 None admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:16:16.817548 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c70fdefc-620f-4bc4-91fa-b83945fba6b1 None admin] PATH_INFO: `/v3/users/39bf6631ce574c758a674faedecf2254` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:16:16.820099 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c70fdefc-620f-4bc4-91fa-b83945fba6b1 None admin] RBAC: Authorizing `identity:get_user(user_id=39bf6631ce574c758a674faedecf2254)` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:16:16.822372 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c70fdefc-620f-4bc4-91fa-b83945fba6b1 None admin] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:16:16.824461 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 67/130] 10.222.0.47 () {56 vars in 1177 bytes} [Thu Jul 21 08:16:16 2022] GET /identity/v3/users/39bf6631ce574c758a674faedecf2254 => generated 275 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 08:16:16.830222 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c70fdefc-620f-4bc4-91fa-b83945fba6b1 None admin] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:16:16.859957 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG dbcounter [-] [80245] Writing DB stats keystone:SELECT=100 {{(pid=80245) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:16:16.906285 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-327b8cd4-49d7-41e9-9832-b5e8f6620847 None admin] Validating token access rules against request {{(pid=80245) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:16:16.976977 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1af7b7b-a532-4115-b45c-1d154e13a9f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80245) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:16:16.978281 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1af7b7b-a532-4115-b45c-1d154e13a9f4 None admin] REQUEST_METHOD: `GET` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:16:16.979058 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG dbcounter [-] [80244] Writing DB stats keystone:SELECT=100 {{(pid=80244) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:16:16.979940 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1af7b7b-a532-4115-b45c-1d154e13a9f4 None admin] SCRIPT_NAME: `/identity` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:16:16.980435 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1af7b7b-a532-4115-b45c-1d154e13a9f4 None admin] PATH_INFO: `/v3/users/39bf6631ce574c758a674faedecf2254` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:16:16.983033 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1af7b7b-a532-4115-b45c-1d154e13a9f4 None admin] RBAC: Authorizing `identity:get_user(user_id=39bf6631ce574c758a674faedecf2254)` {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:16:16.985452 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1af7b7b-a532-4115-b45c-1d154e13a9f4 None admin] RBAC: Authorization granted {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:16:16.987372 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: [pid: 80245|app: 0|req: 64/131] 10.222.0.47 () {56 vars in 1177 bytes} [Thu Jul 21 08:16:16 2022] GET /identity/v3/users/39bf6631ce574c758a674faedecf2254 => generated 275 bytes in 300 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 08:16:16.991523 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1af7b7b-a532-4115-b45c-1d154e13a9f4 None admin] Authenticating user token {{(pid=80245) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:16:17.017307 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c70fdefc-620f-4bc4-91fa-b83945fba6b1 None admin] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:16:17.094981 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f0d1583-f7e0-4ccb-b8c3-419dd63f9248 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:16:17.096441 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f0d1583-f7e0-4ccb-b8c3-419dd63f9248 None admin] REQUEST_METHOD: `GET` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:16:17.096935 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f0d1583-f7e0-4ccb-b8c3-419dd63f9248 None admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:16:17.097404 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f0d1583-f7e0-4ccb-b8c3-419dd63f9248 None admin] PATH_INFO: `/v3/projects/eab7e5ec54c94574ad68c89747ad5bcc` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:16:17.103001 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f0d1583-f7e0-4ccb-b8c3-419dd63f9248 None admin] RBAC: Authorizing `identity:get_project(project_id=eab7e5ec54c94574ad68c89747ad5bcc)` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:16:17.104754 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f0d1583-f7e0-4ccb-b8c3-419dd63f9248 None admin] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:16:17.106758 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 68/132] 10.222.0.47 () {56 vars in 1186 bytes} [Thu Jul 21 08:16:16 2022] GET /identity/v3/projects/eab7e5ec54c94574ad68c89747ad5bcc => generated 298 bytes in 278 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 08:16:17.107869 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f0d1583-f7e0-4ccb-b8c3-419dd63f9248 None admin] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:16:17.124584 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1af7b7b-a532-4115-b45c-1d154e13a9f4 None admin] Validating token access rules against request {{(pid=80245) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:16:17.167163 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-693542e7-6d8d-4dd6-8341-9ed6ac2ccb85 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80245) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:16:17.168029 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-693542e7-6d8d-4dd6-8341-9ed6ac2ccb85 None admin] REQUEST_METHOD: `GET` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:16:17.169440 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-693542e7-6d8d-4dd6-8341-9ed6ac2ccb85 None admin] SCRIPT_NAME: `/identity` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:16:17.169925 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-693542e7-6d8d-4dd6-8341-9ed6ac2ccb85 None admin] PATH_INFO: `/v3/projects/eab7e5ec54c94574ad68c89747ad5bcc` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:16:17.172735 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-693542e7-6d8d-4dd6-8341-9ed6ac2ccb85 None admin] RBAC: Authorizing `identity:get_project(project_id=eab7e5ec54c94574ad68c89747ad5bcc)` {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:16:17.174209 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-693542e7-6d8d-4dd6-8341-9ed6ac2ccb85 None admin] RBAC: Authorization granted {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:16:17.176230 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: [pid: 80245|app: 0|req: 65/133] 10.222.0.47 () {56 vars in 1186 bytes} [Thu Jul 21 08:16:16 2022] GET /identity/v3/projects/eab7e5ec54c94574ad68c89747ad5bcc => generated 298 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 08:16:17.182074 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-693542e7-6d8d-4dd6-8341-9ed6ac2ccb85 None admin] Authenticating user token {{(pid=80245) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:16:17.201695 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG dbcounter [-] [80245] Writing DB stats keystone:SELECT=100 {{(pid=80245) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:16:17.293804 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-693542e7-6d8d-4dd6-8341-9ed6ac2ccb85 None admin] Validating token access rules against request {{(pid=80245) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:16:17.317715 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f0d1583-f7e0-4ccb-b8c3-419dd63f9248 None admin] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:16:17.329698 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de3a08af-efb9-4343-a83e-8b4df2aaf18f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80245) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:16:17.330604 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de3a08af-efb9-4343-a83e-8b4df2aaf18f None admin] REQUEST_METHOD: `PUT` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:16:17.330891 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de3a08af-efb9-4343-a83e-8b4df2aaf18f None admin] SCRIPT_NAME: `/identity` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:16:17.331173 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de3a08af-efb9-4343-a83e-8b4df2aaf18f None admin] PATH_INFO: `/v3/projects/eab7e5ec54c94574ad68c89747ad5bcc/users/39bf6631ce574c758a674faedecf2254/roles/3a074738221e4601b5205715ba87d240` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:16:17.333207 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de3a08af-efb9-4343-a83e-8b4df2aaf18f None admin] RBAC: Authorizing `identity:create_grant(project_id=eab7e5ec54c94574ad68c89747ad5bcc, user_id=39bf6631ce574c758a674faedecf2254, role_id=3a074738221e4601b5205715ba87d240)` {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:16:17.335239 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de3a08af-efb9-4343-a83e-8b4df2aaf18f None admin] RBAC: Authorization granted {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:16:17.352579 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: [pid: 80245|app: 0|req: 66/134] 10.222.0.47 () {58 vars in 1439 bytes} [Thu Jul 21 08:16:17 2022] PUT /identity/v3/projects/eab7e5ec54c94574ad68c89747ad5bcc/users/39bf6631ce574c758a674faedecf2254/roles/3a074738221e4601b5205715ba87d240 => generated 0 bytes in 176 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 08:16:17.353574 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de3a08af-efb9-4343-a83e-8b4df2aaf18f None admin] Authenticating user token {{(pid=80245) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:16:17.362073 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf3d089b-c61c-484b-8f92-e1540f192043 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:16:17.363517 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf3d089b-c61c-484b-8f92-e1540f192043 None admin] REQUEST_METHOD: `GET` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:16:17.364761 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf3d089b-c61c-484b-8f92-e1540f192043 None admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:16:17.365401 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf3d089b-c61c-484b-8f92-e1540f192043 None admin] PATH_INFO: `/v3/projects/d77ff705f0ca44808214e0d3b6c4a1cf` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:16:17.367741 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf3d089b-c61c-484b-8f92-e1540f192043 None admin] RBAC: Authorizing `identity:get_project(project_id=d77ff705f0ca44808214e0d3b6c4a1cf)` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:16:17.370415 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf3d089b-c61c-484b-8f92-e1540f192043 None admin] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:16:17.373543 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 69/135] 10.222.0.47 () {56 vars in 1186 bytes} [Thu Jul 21 08:16:17 2022] GET /identity/v3/projects/d77ff705f0ca44808214e0d3b6c4a1cf => generated 312 bytes in 266 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 08:16:17.375521 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24680bf4-04f4-4325-9b8b-73a2713c7ef2 None None] REQUEST_METHOD: `GET` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:16:17.376430 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24680bf4-04f4-4325-9b8b-73a2713c7ef2 None None] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:16:17.377304 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24680bf4-04f4-4325-9b8b-73a2713c7ef2 None None] PATH_INFO: `/` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:16:17.379054 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 70/136] 10.222.0.47 () {54 vars in 926 bytes} [Thu Jul 21 08:16:17 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 08:16:17.380157 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24680bf4-04f4-4325-9b8b-73a2713c7ef2 None None] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:16:17.430940 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG dbcounter [-] [80244] Writing DB stats keystone:SELECT=100 {{(pid=80244) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:16:17.450491 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG dbcounter [-] [80245] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80245) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:16:17.481972 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de3a08af-efb9-4343-a83e-8b4df2aaf18f None admin] Validating token access rules against request {{(pid=80245) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:16:17.528572 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6de73fa-c36b-48f4-b94e-5254f4b084a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80245) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:16:17.529545 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6de73fa-c36b-48f4-b94e-5254f4b084a3 None admin] REQUEST_METHOD: `PUT` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:16:17.529840 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6de73fa-c36b-48f4-b94e-5254f4b084a3 None admin] SCRIPT_NAME: `/identity` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:16:17.530123 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6de73fa-c36b-48f4-b94e-5254f4b084a3 None admin] PATH_INFO: `/v3/projects/eab7e5ec54c94574ad68c89747ad5bcc/users/39bf6631ce574c758a674faedecf2254/roles/99a638c0aac44b79a1f2858574724495` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:16:17.535931 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6de73fa-c36b-48f4-b94e-5254f4b084a3 None admin] RBAC: Authorizing `identity:create_grant(project_id=eab7e5ec54c94574ad68c89747ad5bcc, user_id=39bf6631ce574c758a674faedecf2254, role_id=99a638c0aac44b79a1f2858574724495)` {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:16:17.537574 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6de73fa-c36b-48f4-b94e-5254f4b084a3 None admin] RBAC: Authorization granted {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:16:17.558896 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: [pid: 80245|app: 0|req: 67/137] 10.222.0.47 () {58 vars in 1439 bytes} [Thu Jul 21 08:16:17 2022] PUT /identity/v3/projects/eab7e5ec54c94574ad68c89747ad5bcc/users/39bf6631ce574c758a674faedecf2254/roles/99a638c0aac44b79a1f2858574724495 => generated 0 bytes in 206 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 08:16:17.560405 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8be13e4-96f5-4d8f-b0a3-2edb96f98ec9 None None] REQUEST_METHOD: `POST` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:16:17.560698 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8be13e4-96f5-4d8f-b0a3-2edb96f98ec9 None None] SCRIPT_NAME: `/identity` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:16:17.561005 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8be13e4-96f5-4d8f-b0a3-2edb96f98ec9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:16:17.597424 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24680bf4-04f4-4325-9b8b-73a2713c7ef2 None None] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:16:17.612637 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.auth.core [None req-b8be13e4-96f5-4d8f-b0a3-2edb96f98ec9 None None] MFA Rules not processed for user `2dce40b8d74c41ec94cc36e6ff05d412`. Rule list: `[]` (Enabled: `True`). {{(pid=80245) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 08:16:17.613865 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.fernet_utils [None req-b8be13e4-96f5-4d8f-b0a3-2edb96f98ec9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80245) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 08:16:17.650005 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90ae91fe-f2e9-4fc5-b21d-f63caaed52e7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:16:17.652936 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90ae91fe-f2e9-4fc5-b21d-f63caaed52e7 None admin] REQUEST_METHOD: `PUT` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:16:17.653551 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90ae91fe-f2e9-4fc5-b21d-f63caaed52e7 None admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:16:17.654596 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90ae91fe-f2e9-4fc5-b21d-f63caaed52e7 None admin] PATH_INFO: `/v3/projects/d77ff705f0ca44808214e0d3b6c4a1cf/users/39bf6631ce574c758a674faedecf2254/roles/3a074738221e4601b5205715ba87d240` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:16:17.657398 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90ae91fe-f2e9-4fc5-b21d-f63caaed52e7 None admin] RBAC: Authorizing `identity:create_grant(project_id=d77ff705f0ca44808214e0d3b6c4a1cf, user_id=39bf6631ce574c758a674faedecf2254, role_id=3a074738221e4601b5205715ba87d240)` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:16:17.659302 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90ae91fe-f2e9-4fc5-b21d-f63caaed52e7 None admin] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:16:17.680219 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 71/138] 10.222.0.47 () {58 vars in 1439 bytes} [Thu Jul 21 08:16:17 2022] PUT /identity/v3/projects/d77ff705f0ca44808214e0d3b6c4a1cf/users/39bf6631ce574c758a674faedecf2254/roles/3a074738221e4601b5205715ba87d240 => generated 0 bytes in 300 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 08:16:17.756682 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: [pid: 80245|app: 0|req: 68/139] 10.222.0.47 () {58 vars in 1023 bytes} [Thu Jul 21 08:16:17 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 197 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 21 08:16:17.784051 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90ae91fe-f2e9-4fc5-b21d-f63caaed52e7 None admin] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:16:17.890417 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG dbcounter [-] [80244] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80244) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:16:17.913404 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90ae91fe-f2e9-4fc5-b21d-f63caaed52e7 None admin] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:16:17.953719 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f653aa74-7f59-40a8-a1b2-4ec4c636086f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:16:17.954532 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f653aa74-7f59-40a8-a1b2-4ec4c636086f None admin] REQUEST_METHOD: `GET` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:16:17.954872 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f653aa74-7f59-40a8-a1b2-4ec4c636086f None admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:16:17.955208 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f653aa74-7f59-40a8-a1b2-4ec4c636086f None admin] PATH_INFO: `/v3/domains/default` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:16:17.956512 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f653aa74-7f59-40a8-a1b2-4ec4c636086f None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:16:17.958046 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f653aa74-7f59-40a8-a1b2-4ec4c636086f None admin] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:16:17.960436 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 72/140] 10.222.0.47 () {56 vars in 1108 bytes} [Thu Jul 21 08:16:17 2022] GET /identity/v3/domains/default => generated 199 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 08:16:17.966250 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8be13e4-96f5-4d8f-b0a3-2edb96f98ec9 None None] Authenticating user token {{(pid=80245) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:16:18.033502 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG dbcounter [-] [80245] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80245) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:16:18.084271 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8be13e4-96f5-4d8f-b0a3-2edb96f98ec9 None None] Validating token access rules against request {{(pid=80245) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:16:18.128333 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f529643c-42fd-45bc-b414-fb382c6c44b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80245) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:16:18.129034 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f529643c-42fd-45bc-b414-fb382c6c44b8 None admin] REQUEST_METHOD: `POST` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:16:18.129324 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f529643c-42fd-45bc-b414-fb382c6c44b8 None admin] SCRIPT_NAME: `/identity` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:16:18.129729 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f529643c-42fd-45bc-b414-fb382c6c44b8 None admin] PATH_INFO: `/v3/users` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:16:18.130416 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f529643c-42fd-45bc-b414-fb382c6c44b8 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:16:18.132466 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f529643c-42fd-45bc-b414-fb382c6c44b8 None admin] RBAC: Authorization granted {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:16:18.159067 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: [pid: 80245|app: 0|req: 69/141] 10.222.0.47 () {60 vars in 1132 bytes} [Thu Jul 21 08:16:17 2022] POST /identity/v3/users => generated 283 bytes in 193 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 08:16:18.199902 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a85c8dd7-42a1-4548-a8cf-2ebaa66eda03 None None] REQUEST_METHOD: `GET` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:16:18.200215 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a85c8dd7-42a1-4548-a8cf-2ebaa66eda03 None None] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:16:18.200486 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a85c8dd7-42a1-4548-a8cf-2ebaa66eda03 None None] PATH_INFO: `/` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:16:18.201877 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 73/142] 10.222.0.47 () {54 vars in 926 bytes} [Thu Jul 21 08:16:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 08:16:18.207533 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb961c0a-efb3-4dd9-8e52-95f3265abc91 None None] REQUEST_METHOD: `POST` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:16:18.207881 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb961c0a-efb3-4dd9-8e52-95f3265abc91 None None] SCRIPT_NAME: `/identity` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:16:18.208170 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb961c0a-efb3-4dd9-8e52-95f3265abc91 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:16:18.230117 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.auth.core [None req-bb961c0a-efb3-4dd9-8e52-95f3265abc91 None None] MFA Rules not processed for user `2dce40b8d74c41ec94cc36e6ff05d412`. Rule list: `[]` (Enabled: `True`). {{(pid=80245) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 08:16:18.231389 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.fernet_utils [None req-bb961c0a-efb3-4dd9-8e52-95f3265abc91 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80245) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 08:16:18.322358 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: [pid: 80245|app: 0|req: 70/143] 10.222.0.47 () {58 vars in 1023 bytes} [Thu Jul 21 08:16:18 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 21 08:16:18.351589 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a85c8dd7-42a1-4548-a8cf-2ebaa66eda03 None None] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:16:18.500075 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a85c8dd7-42a1-4548-a8cf-2ebaa66eda03 None None] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:16:18.551580 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-907d0819-c3c0-42f6-a4be-be7dfcd83930 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:16:18.552492 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-907d0819-c3c0-42f6-a4be-be7dfcd83930 None admin] REQUEST_METHOD: `GET` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:16:18.552781 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-907d0819-c3c0-42f6-a4be-be7dfcd83930 None admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:16:18.553058 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-907d0819-c3c0-42f6-a4be-be7dfcd83930 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:16:18.564867 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-907d0819-c3c0-42f6-a4be-be7dfcd83930 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 21 08:16:18.573661 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application [None req-907d0819-c3c0-42f6-a4be-be7dfcd83930 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jul 21 08:16:18.573661 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 08:16:18.573661 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 08:16:18.573661 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 08:16:18.573661 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 08:16:18.573661 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 08:16:18.573661 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 08:16:18.573661 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 08:16:18.573661 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 08:16:18.573661 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 08:16:18.573661 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 08:16:18.573661 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 08:16:18.573661 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 21 08:16:18.573661 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 21 08:16:18.573661 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 21 08:16:18.573661 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 21 08:16:18.574492 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 21 08:16:18.574492 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 21 08:16:18.574492 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 21 08:16:18.574492 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 21 08:16:18.574492 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 08:16:18.574492 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 08:16:18.574492 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 08:16:18.574492 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 08:16:18.574492 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 08:16:18.574492 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 08:16:18.574492 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 08:16:18.574492 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application with Lock( Jul 21 08:16:18.574492 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 08:16:18.574492 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self._enter() Jul 21 08:16:18.574492 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 08:16:18.574492 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 08:16:18.575238 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 08:16:18.575238 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self.creator() Jul 21 08:16:18.575238 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 08:16:18.575238 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application created_value = creator( Jul 21 08:16:18.575238 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 21 08:16:18.575238 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 21 08:16:18.575238 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 21 08:16:18.575238 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 21 08:16:18.575238 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 21 08:16:18.575238 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 21 08:16:18.575238 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jul 21 08:16:18.575238 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application Jul 21 08:16:18.575238 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 74/144] 10.222.0.47 () {56 vars in 1099 bytes} [Thu Jul 21 08:16:18 2022] GET /identity/v3/roles/reader => generated 84 bytes in 224 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 08:16:18.581020 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb961c0a-efb3-4dd9-8e52-95f3265abc91 None None] Authenticating user token {{(pid=80245) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:16:18.637775 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG dbcounter [-] [80245] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80245) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:16:18.694174 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb961c0a-efb3-4dd9-8e52-95f3265abc91 None None] Validating token access rules against request {{(pid=80245) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:16:18.729693 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30aa24e7-32fe-4b41-bd9b-c31c8de26ef5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80245) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:16:18.730447 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30aa24e7-32fe-4b41-bd9b-c31c8de26ef5 None admin] REQUEST_METHOD: `GET` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:16:18.730739 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30aa24e7-32fe-4b41-bd9b-c31c8de26ef5 None admin] SCRIPT_NAME: `/identity` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:16:18.731019 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30aa24e7-32fe-4b41-bd9b-c31c8de26ef5 None admin] PATH_INFO: `/v3/roles` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:16:18.731809 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30aa24e7-32fe-4b41-bd9b-c31c8de26ef5 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80245) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 08:16:18.732119 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30aa24e7-32fe-4b41-bd9b-c31c8de26ef5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:16:18.733510 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30aa24e7-32fe-4b41-bd9b-c31c8de26ef5 None admin] RBAC: Authorization granted {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:16:18.742243 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: [pid: 80245|app: 0|req: 71/145] 10.222.0.47 () {56 vars in 1101 bytes} [Thu Jul 21 08:16:18 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 08:16:18.748100 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-907d0819-c3c0-42f6-a4be-be7dfcd83930 None admin] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:16:18.760626 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG dbcounter [-] [80244] Writing DB stats keystone:SELECT=100 {{(pid=80244) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:16:18.863547 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-907d0819-c3c0-42f6-a4be-be7dfcd83930 None admin] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:16:18.907761 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42c37106-8959-4247-811b-802f1fcc23b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:16:18.908694 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42c37106-8959-4247-811b-802f1fcc23b2 None admin] REQUEST_METHOD: `GET` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:16:18.908984 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42c37106-8959-4247-811b-802f1fcc23b2 None admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:16:18.909260 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42c37106-8959-4247-811b-802f1fcc23b2 None admin] PATH_INFO: `/v3/users/62835f20a4f74cb8a96c4979d095b168` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:16:18.916109 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42c37106-8959-4247-811b-802f1fcc23b2 None admin] RBAC: Authorizing `identity:get_user(user_id=62835f20a4f74cb8a96c4979d095b168)` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:16:18.917625 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42c37106-8959-4247-811b-802f1fcc23b2 None admin] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:16:18.919373 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 75/146] 10.222.0.47 () {56 vars in 1177 bytes} [Thu Jul 21 08:16:18 2022] GET /identity/v3/users/62835f20a4f74cb8a96c4979d095b168 => generated 289 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 08:16:18.925203 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30aa24e7-32fe-4b41-bd9b-c31c8de26ef5 None admin] Authenticating user token {{(pid=80245) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:16:19.051507 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30aa24e7-32fe-4b41-bd9b-c31c8de26ef5 None admin] Validating token access rules against request {{(pid=80245) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:16:19.065136 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG dbcounter [-] [80245] Writing DB stats keystone:SELECT=100 {{(pid=80245) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:16:19.088636 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b129aed8-f762-48cd-acdd-18d92835ee3a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80245) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:16:19.089517 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b129aed8-f762-48cd-acdd-18d92835ee3a None admin] REQUEST_METHOD: `GET` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:16:19.089797 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b129aed8-f762-48cd-acdd-18d92835ee3a None admin] SCRIPT_NAME: `/identity` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:16:19.090072 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b129aed8-f762-48cd-acdd-18d92835ee3a None admin] PATH_INFO: `/v3/projects/eab7e5ec54c94574ad68c89747ad5bcc` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:16:19.091277 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b129aed8-f762-48cd-acdd-18d92835ee3a None admin] RBAC: Authorizing `identity:get_project(project_id=eab7e5ec54c94574ad68c89747ad5bcc)` {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:16:19.092758 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b129aed8-f762-48cd-acdd-18d92835ee3a None admin] RBAC: Authorization granted {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:16:19.094629 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: [pid: 80245|app: 0|req: 72/147] 10.222.0.47 () {56 vars in 1186 bytes} [Thu Jul 21 08:16:18 2022] GET /identity/v3/projects/eab7e5ec54c94574ad68c89747ad5bcc => generated 298 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 08:16:19.100934 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42c37106-8959-4247-811b-802f1fcc23b2 None admin] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:16:19.202348 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG dbcounter [-] [80244] Writing DB stats keystone:SELECT=100 {{(pid=80244) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:16:19.228278 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42c37106-8959-4247-811b-802f1fcc23b2 None admin] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:16:19.264874 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-348a34c1-a200-4347-b73e-86eeb548821d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:16:19.265833 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-348a34c1-a200-4347-b73e-86eeb548821d None admin] REQUEST_METHOD: `PUT` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:16:19.266121 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-348a34c1-a200-4347-b73e-86eeb548821d None admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:16:19.266397 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-348a34c1-a200-4347-b73e-86eeb548821d None admin] PATH_INFO: `/v3/projects/eab7e5ec54c94574ad68c89747ad5bcc/users/62835f20a4f74cb8a96c4979d095b168/roles/9e5171884e9f401ab2337b74db49546e` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:16:19.268477 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-348a34c1-a200-4347-b73e-86eeb548821d None admin] RBAC: Authorizing `identity:create_grant(project_id=eab7e5ec54c94574ad68c89747ad5bcc, user_id=62835f20a4f74cb8a96c4979d095b168, role_id=9e5171884e9f401ab2337b74db49546e)` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:16:19.269932 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-348a34c1-a200-4347-b73e-86eeb548821d None admin] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:16:19.289361 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 76/148] 10.222.0.47 () {58 vars in 1439 bytes} [Thu Jul 21 08:16:19 2022] PUT /identity/v3/projects/eab7e5ec54c94574ad68c89747ad5bcc/users/62835f20a4f74cb8a96c4979d095b168/roles/9e5171884e9f401ab2337b74db49546e => generated 0 bytes in 189 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 08:16:19.562870 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1815346b-21a2-4c3b-8a19-245295458be2 None None] REQUEST_METHOD: `GET` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:16:19.563217 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1815346b-21a2-4c3b-8a19-245295458be2 None None] SCRIPT_NAME: `/identity` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:16:19.563443 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1815346b-21a2-4c3b-8a19-245295458be2 None None] PATH_INFO: `/` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:16:19.565189 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: [pid: 80245|app: 0|req: 73/149] 10.222.0.47 () {54 vars in 926 bytes} [Thu Jul 21 08:16:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 08:16:19.575169 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6df92e9c-ee46-4f24-b20c-878cd30a7a61 None None] REQUEST_METHOD: `POST` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:16:19.575500 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6df92e9c-ee46-4f24-b20c-878cd30a7a61 None None] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:16:19.575771 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6df92e9c-ee46-4f24-b20c-878cd30a7a61 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:16:19.604623 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.auth.core [None req-6df92e9c-ee46-4f24-b20c-878cd30a7a61 None None] MFA Rules not processed for user `2dce40b8d74c41ec94cc36e6ff05d412`. Rule list: `[]` (Enabled: `True`). {{(pid=80244) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 08:16:19.605884 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.fernet_utils [None req-6df92e9c-ee46-4f24-b20c-878cd30a7a61 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80244) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 08:16:19.689195 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 77/150] 10.222.0.47 () {58 vars in 1023 bytes} [Thu Jul 21 08:16:19 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 21 08:16:19.714223 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1815346b-21a2-4c3b-8a19-245295458be2 None None] Authenticating user token {{(pid=80245) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:16:19.831894 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1815346b-21a2-4c3b-8a19-245295458be2 None None] Validating token access rules against request {{(pid=80245) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:16:19.888253 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0adfae27-552f-4f96-ab72-72f06c8e1ba9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80245) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:16:19.889145 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0adfae27-552f-4f96-ab72-72f06c8e1ba9 None admin] REQUEST_METHOD: `GET` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:16:19.889439 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0adfae27-552f-4f96-ab72-72f06c8e1ba9 None admin] SCRIPT_NAME: `/identity` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:16:19.889719 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0adfae27-552f-4f96-ab72-72f06c8e1ba9 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:16:19.906030 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0adfae27-552f-4f96-ab72-72f06c8e1ba9 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 21 08:16:19.914598 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application [None req-0adfae27-552f-4f96-ab72-72f06c8e1ba9 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 21 08:16:19.914598 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 08:16:19.914598 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 08:16:19.914598 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 08:16:19.914598 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 08:16:19.914598 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 08:16:19.914598 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 08:16:19.914598 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 08:16:19.914598 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 08:16:19.914598 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 08:16:19.914598 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 08:16:19.914598 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 08:16:19.914598 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 21 08:16:19.914598 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 21 08:16:19.914598 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 21 08:16:19.914598 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 21 08:16:19.915531 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 21 08:16:19.915531 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 21 08:16:19.915531 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 21 08:16:19.915531 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 21 08:16:19.915531 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 08:16:19.915531 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 08:16:19.915531 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 08:16:19.915531 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 08:16:19.915531 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 08:16:19.915531 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 08:16:19.915531 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 08:16:19.915531 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application with Lock( Jul 21 08:16:19.915531 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 08:16:19.915531 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application return self._enter() Jul 21 08:16:19.915531 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 08:16:19.915531 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 08:16:19.916322 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 08:16:19.916322 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application return self.creator() Jul 21 08:16:19.916322 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 08:16:19.916322 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application created_value = creator( Jul 21 08:16:19.916322 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 21 08:16:19.916322 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 21 08:16:19.916322 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 21 08:16:19.916322 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 21 08:16:19.916322 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 21 08:16:19.916322 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 21 08:16:19.916322 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 21 08:16:19.916322 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application Jul 21 08:16:19.919451 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: [pid: 80245|app: 0|req: 74/151] 10.222.0.47 () {56 vars in 1096 bytes} [Thu Jul 21 08:16:19 2022] GET /identity/v3/roles/admin => generated 83 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 08:16:19.922251 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6df92e9c-ee46-4f24-b20c-878cd30a7a61 None None] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:16:19.995793 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG dbcounter [-] [80244] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80244) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:16:20.033554 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6df92e9c-ee46-4f24-b20c-878cd30a7a61 None None] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:16:20.069649 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db808614-04a0-426f-bf05-115b22142f20 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:16:20.070330 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db808614-04a0-426f-bf05-115b22142f20 None admin] REQUEST_METHOD: `GET` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:16:20.070613 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db808614-04a0-426f-bf05-115b22142f20 None admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:16:20.070891 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db808614-04a0-426f-bf05-115b22142f20 None admin] PATH_INFO: `/v3/roles` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:16:20.071697 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db808614-04a0-426f-bf05-115b22142f20 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80244) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 08:16:20.072003 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db808614-04a0-426f-bf05-115b22142f20 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:16:20.073412 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db808614-04a0-426f-bf05-115b22142f20 None admin] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:16:20.082528 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 78/152] 10.222.0.47 () {56 vars in 1099 bytes} [Thu Jul 21 08:16:19 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 08:16:20.088818 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0adfae27-552f-4f96-ab72-72f06c8e1ba9 None admin] Authenticating user token {{(pid=80245) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:16:20.134556 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG dbcounter [-] [80245] Writing DB stats keystone:SELECT=100 {{(pid=80245) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:16:20.248356 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0adfae27-552f-4f96-ab72-72f06c8e1ba9 None admin] Validating token access rules against request {{(pid=80245) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:16:20.313003 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9568bef-e661-4cb6-89b2-59a77328f67a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80245) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:16:20.314199 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9568bef-e661-4cb6-89b2-59a77328f67a None admin] REQUEST_METHOD: `GET` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:16:20.314664 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9568bef-e661-4cb6-89b2-59a77328f67a None admin] SCRIPT_NAME: `/identity` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:16:20.315116 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9568bef-e661-4cb6-89b2-59a77328f67a None admin] PATH_INFO: `/v3/users/2dce40b8d74c41ec94cc36e6ff05d412` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:16:20.320061 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9568bef-e661-4cb6-89b2-59a77328f67a None admin] RBAC: Authorizing `identity:get_user(user_id=2dce40b8d74c41ec94cc36e6ff05d412)` {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:16:20.321736 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9568bef-e661-4cb6-89b2-59a77328f67a None admin] RBAC: Authorization granted {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:16:20.324001 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: [pid: 80245|app: 0|req: 75/153] 10.222.0.47 () {56 vars in 1177 bytes} [Thu Jul 21 08:16:20 2022] GET /identity/v3/users/2dce40b8d74c41ec94cc36e6ff05d412 => generated 247 bytes in 235 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 08:16:20.329923 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db808614-04a0-426f-bf05-115b22142f20 None admin] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:16:20.435696 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db808614-04a0-426f-bf05-115b22142f20 None admin] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:16:20.463960 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG dbcounter [-] [80244] Writing DB stats keystone:SELECT=100 {{(pid=80244) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:16:20.472206 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b419043-5660-4aa9-bf43-8217947fab03 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:16:20.473186 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b419043-5660-4aa9-bf43-8217947fab03 None admin] REQUEST_METHOD: `GET` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:16:20.473668 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b419043-5660-4aa9-bf43-8217947fab03 None admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:16:20.474141 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b419043-5660-4aa9-bf43-8217947fab03 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:16:20.561023 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b419043-5660-4aa9-bf43-8217947fab03 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:16:20.562836 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b419043-5660-4aa9-bf43-8217947fab03 None admin] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:16:20.671659 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6d92864-b044-4f20-9d74-6d343b07e5c2 None None] REQUEST_METHOD: `GET` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:16:20.672692 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6d92864-b044-4f20-9d74-6d343b07e5c2 None None] SCRIPT_NAME: `/identity` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:16:20.673189 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6d92864-b044-4f20-9d74-6d343b07e5c2 None None] PATH_INFO: `/` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:16:20.676971 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: [pid: 80245|app: 0|req: 76/154] 10.222.0.47 () {54 vars in 926 bytes} [Thu Jul 21 08:16:20 2022] GET /identity => generated 270 bytes in 10 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 08:16:20.691663 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef142899-9f2e-4ba3-a3c9-e6eea2b7f2da None None] REQUEST_METHOD: `POST` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:16:20.692269 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef142899-9f2e-4ba3-a3c9-e6eea2b7f2da None None] SCRIPT_NAME: `/identity` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:16:20.692748 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef142899-9f2e-4ba3-a3c9-e6eea2b7f2da None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:16:20.722466 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 79/155] 10.222.0.47 () {58 vars in 1282 bytes} [Thu Jul 21 08:16:20 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 393 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 21 08:16:20.736667 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.auth.core [None req-ef142899-9f2e-4ba3-a3c9-e6eea2b7f2da None None] MFA Rules not processed for user `2dce40b8d74c41ec94cc36e6ff05d412`. Rule list: `[]` (Enabled: `True`). {{(pid=80245) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 08:16:20.738520 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.fernet_utils [None req-ef142899-9f2e-4ba3-a3c9-e6eea2b7f2da None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80245) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 08:16:20.740353 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b419043-5660-4aa9-bf43-8217947fab03 None admin] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:16:20.813362 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG dbcounter [-] [80244] Writing DB stats keystone:SELECT=100 {{(pid=80244) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:16:20.922113 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b419043-5660-4aa9-bf43-8217947fab03 None admin] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:16:20.930204 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: [pid: 80245|app: 0|req: 77/156] 10.222.0.47 () {58 vars in 1023 bytes} [Thu Jul 21 08:16:20 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 240 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 21 08:16:20.978074 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbc79474-0457-4a96-be90-a609a68f4408 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:16:20.979014 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbc79474-0457-4a96-be90-a609a68f4408 None admin] REQUEST_METHOD: `GET` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:16:20.979425 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbc79474-0457-4a96-be90-a609a68f4408 None admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:16:20.979702 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbc79474-0457-4a96-be90-a609a68f4408 None admin] PATH_INFO: `/v3/projects/eab7e5ec54c94574ad68c89747ad5bcc` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:16:20.984085 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbc79474-0457-4a96-be90-a609a68f4408 None admin] RBAC: Authorizing `identity:get_project(project_id=eab7e5ec54c94574ad68c89747ad5bcc)` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:16:20.985556 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbc79474-0457-4a96-be90-a609a68f4408 None admin] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:16:20.987555 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 80/157] 10.222.0.47 () {56 vars in 1186 bytes} [Thu Jul 21 08:16:20 2022] GET /identity/v3/projects/eab7e5ec54c94574ad68c89747ad5bcc => generated 298 bytes in 247 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 08:16:20.994631 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef142899-9f2e-4ba3-a3c9-e6eea2b7f2da None None] Authenticating user token {{(pid=80245) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:16:20.994882 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbc79474-0457-4a96-be90-a609a68f4408 None admin] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:16:21.078040 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG dbcounter [-] [80245] Writing DB stats keystone:SELECT=100 {{(pid=80245) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:16:21.132080 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG dbcounter [-] [80244] Writing DB stats keystone:SELECT=100 {{(pid=80244) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:16:21.139217 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbc79474-0457-4a96-be90-a609a68f4408 None admin] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:16:21.192672 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0071a66-7876-4678-aff2-147cd10355e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:16:21.193492 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0071a66-7876-4678-aff2-147cd10355e4 None admin] REQUEST_METHOD: `GET` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:16:21.193906 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0071a66-7876-4678-aff2-147cd10355e4 None admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:16:21.194212 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0071a66-7876-4678-aff2-147cd10355e4 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:16:21.207656 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef142899-9f2e-4ba3-a3c9-e6eea2b7f2da None None] Validating token access rules against request {{(pid=80245) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:16:21.212580 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0071a66-7876-4678-aff2-147cd10355e4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 21 08:16:21.229180 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application [None req-f0071a66-7876-4678-aff2-147cd10355e4 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 21 08:16:21.229180 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 08:16:21.229180 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 08:16:21.229180 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 08:16:21.229180 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 08:16:21.229180 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 08:16:21.229180 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 08:16:21.229180 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 08:16:21.229180 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 08:16:21.229180 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 08:16:21.229180 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 08:16:21.229180 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 08:16:21.229180 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 21 08:16:21.229180 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 21 08:16:21.229180 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 21 08:16:21.229180 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 21 08:16:21.230136 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 21 08:16:21.230136 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 21 08:16:21.230136 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 21 08:16:21.230136 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 21 08:16:21.230136 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 08:16:21.230136 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 08:16:21.230136 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 08:16:21.230136 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 08:16:21.230136 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 08:16:21.230136 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 08:16:21.230136 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 08:16:21.230136 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application with Lock( Jul 21 08:16:21.230136 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 08:16:21.230136 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self._enter() Jul 21 08:16:21.230136 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 08:16:21.230136 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 08:16:21.230882 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 08:16:21.230882 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self.creator() Jul 21 08:16:21.230882 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 08:16:21.230882 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application created_value = creator( Jul 21 08:16:21.230882 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 21 08:16:21.230882 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 21 08:16:21.230882 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 21 08:16:21.230882 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 21 08:16:21.230882 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 21 08:16:21.230882 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 21 08:16:21.230882 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 21 08:16:21.230882 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application Jul 21 08:16:21.231506 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 81/158] 10.222.0.47 () {56 vars in 1099 bytes} [Thu Jul 21 08:16:20 2022] GET /identity/v3/roles/member => generated 84 bytes in 236 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 21 08:16:21.237285 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0071a66-7876-4678-aff2-147cd10355e4 None admin] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:16:21.270896 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fcc779f-499e-4dbe-a2b7-8e13eebb327b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80245) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:16:21.272011 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fcc779f-499e-4dbe-a2b7-8e13eebb327b None admin] REQUEST_METHOD: `GET` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:16:21.272499 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fcc779f-499e-4dbe-a2b7-8e13eebb327b None admin] SCRIPT_NAME: `/identity` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:16:21.272979 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fcc779f-499e-4dbe-a2b7-8e13eebb327b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:16:21.274031 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fcc779f-499e-4dbe-a2b7-8e13eebb327b None admin] RBAC: Adding query filter params (role.id=e29870f6e9e2411a9db0d5bf45f32ca5, scope.project.id=eab7e5ec54c94574ad68c89747ad5bcc, user.id=2dce40b8d74c41ec94cc36e6ff05d412) {{(pid=80245) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 08:16:21.274563 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fcc779f-499e-4dbe-a2b7-8e13eebb327b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:16:21.276315 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fcc779f-499e-4dbe-a2b7-8e13eebb327b None admin] RBAC: Authorization granted {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:16:21.289101 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: [pid: 80245|app: 0|req: 78/159] 10.222.0.47 () {56 vars in 1374 bytes} [Thu Jul 21 08:16:20 2022] GET /identity/v3/role_assignments?user.id=2dce40b8d74c41ec94cc36e6ff05d412&scope.project.id=eab7e5ec54c94574ad68c89747ad5bcc&role.id=e29870f6e9e2411a9db0d5bf45f32ca5 => generated 608 bytes in 295 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 08:16:21.387854 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0071a66-7876-4678-aff2-147cd10355e4 None admin] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:16:21.430862 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13da1fe1-9f5e-4179-8017-fc5e60ee20e8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:16:21.431581 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13da1fe1-9f5e-4179-8017-fc5e60ee20e8 None admin] REQUEST_METHOD: `GET` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:16:21.431864 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13da1fe1-9f5e-4179-8017-fc5e60ee20e8 None admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:16:21.432138 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13da1fe1-9f5e-4179-8017-fc5e60ee20e8 None admin] PATH_INFO: `/v3/roles` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:16:21.432898 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13da1fe1-9f5e-4179-8017-fc5e60ee20e8 None admin] RBAC: Adding query filter params (name=member) {{(pid=80244) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 08:16:21.433200 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13da1fe1-9f5e-4179-8017-fc5e60ee20e8 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:16:21.434621 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13da1fe1-9f5e-4179-8017-fc5e60ee20e8 None admin] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:16:21.444435 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 82/160] 10.222.0.47 () {56 vars in 1101 bytes} [Thu Jul 21 08:16:21 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 21 08:16:21.449843 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fcc779f-499e-4dbe-a2b7-8e13eebb327b None admin] Authenticating user token {{(pid=80245) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:16:21.595708 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fcc779f-499e-4dbe-a2b7-8e13eebb327b None admin] Validating token access rules against request {{(pid=80245) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:16:21.597261 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG dbcounter [-] [80245] Writing DB stats keystone:SELECT=100 {{(pid=80245) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:16:21.642825 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24789fed-7c61-45f0-ba92-2dd2a2b132fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80245) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:16:21.643763 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24789fed-7c61-45f0-ba92-2dd2a2b132fb None admin] REQUEST_METHOD: `GET` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:16:21.644045 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24789fed-7c61-45f0-ba92-2dd2a2b132fb None admin] SCRIPT_NAME: `/identity` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:16:21.644322 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24789fed-7c61-45f0-ba92-2dd2a2b132fb None admin] PATH_INFO: `/v3/users/39bf6631ce574c758a674faedecf2254` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:16:21.645702 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24789fed-7c61-45f0-ba92-2dd2a2b132fb None admin] RBAC: Authorizing `identity:get_user(user_id=39bf6631ce574c758a674faedecf2254)` {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:16:21.647762 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24789fed-7c61-45f0-ba92-2dd2a2b132fb None admin] RBAC: Authorization granted {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:16:21.649734 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: [pid: 80245|app: 0|req: 79/161] 10.222.0.47 () {56 vars in 1177 bytes} [Thu Jul 21 08:16:21 2022] GET /identity/v3/users/39bf6631ce574c758a674faedecf2254 => generated 275 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 08:16:21.655896 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13da1fe1-9f5e-4179-8017-fc5e60ee20e8 None admin] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:16:21.677549 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG dbcounter [-] [80244] Writing DB stats keystone:SELECT=100 {{(pid=80244) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:16:21.781829 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13da1fe1-9f5e-4179-8017-fc5e60ee20e8 None admin] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:16:21.823026 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aec07b8-0c06-40d6-8c4e-66bd5a4eff32 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:16:21.823745 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aec07b8-0c06-40d6-8c4e-66bd5a4eff32 None admin] REQUEST_METHOD: `GET` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:16:21.824028 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aec07b8-0c06-40d6-8c4e-66bd5a4eff32 None admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:16:21.824305 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aec07b8-0c06-40d6-8c4e-66bd5a4eff32 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:16:21.882099 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e08c705-e575-4031-a3ab-abdfd1b4e5c4 None None] REQUEST_METHOD: `GET` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:16:21.882588 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e08c705-e575-4031-a3ab-abdfd1b4e5c4 None None] SCRIPT_NAME: `/identity` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:16:21.882657 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e08c705-e575-4031-a3ab-abdfd1b4e5c4 None None] PATH_INFO: `/` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:16:21.884415 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: [pid: 80245|app: 0|req: 80/162] 10.222.0.47 () {54 vars in 926 bytes} [Thu Jul 21 08:16:21 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 21 08:16:21.890106 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a179ad9d-9a29-4673-a1b1-394189bfd0ad None None] REQUEST_METHOD: `POST` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:16:21.890377 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a179ad9d-9a29-4673-a1b1-394189bfd0ad None None] SCRIPT_NAME: `/identity` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:16:21.890648 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a179ad9d-9a29-4673-a1b1-394189bfd0ad None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:16:21.917092 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.auth.core [None req-a179ad9d-9a29-4673-a1b1-394189bfd0ad None None] MFA Rules not processed for user `2dce40b8d74c41ec94cc36e6ff05d412`. Rule list: `[]` (Enabled: `True`). {{(pid=80245) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 08:16:21.919076 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.fernet_utils [None req-a179ad9d-9a29-4673-a1b1-394189bfd0ad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80245) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 08:16:21.937515 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1aec07b8-0c06-40d6-8c4e-66bd5a4eff32 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:16:21.939075 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1aec07b8-0c06-40d6-8c4e-66bd5a4eff32 None admin] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:16:21.967961 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG dbcounter [-] [80244] Writing DB stats keystone:SELECT=100 {{(pid=80244) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:16:22.048184 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: [pid: 80245|app: 0|req: 81/163] 10.222.0.47 () {58 vars in 1023 bytes} [Thu Jul 21 08:16:21 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 159 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 21 08:16:22.074108 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a179ad9d-9a29-4673-a1b1-394189bfd0ad None None] Authenticating user token {{(pid=80245) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:16:22.106690 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 83/164] 10.222.0.47 () {58 vars in 1282 bytes} [Thu Jul 21 08:16:21 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 451 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 21 08:16:22.107996 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-115aa3a4-bf78-49e9-af6d-55b4c2dd664e None None] REQUEST_METHOD: `GET` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:16:22.108289 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-115aa3a4-bf78-49e9-af6d-55b4c2dd664e None None] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:16:22.108569 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-115aa3a4-bf78-49e9-af6d-55b4c2dd664e None None] PATH_INFO: `/` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:16:22.113744 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 84/165] 10.222.0.47 () {54 vars in 926 bytes} [Thu Jul 21 08:16:22 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 08:16:22.114578 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-115aa3a4-bf78-49e9-af6d-55b4c2dd664e None None] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:16:22.241751 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG dbcounter [-] [80245] Writing DB stats keystone:SELECT=100 {{(pid=80245) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:16:22.246671 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a179ad9d-9a29-4673-a1b1-394189bfd0ad None None] Validating token access rules against request {{(pid=80245) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:16:22.279496 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-115aa3a4-bf78-49e9-af6d-55b4c2dd664e None None] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:16:22.298030 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG dbcounter [-] [80244] Writing DB stats keystone:SELECT=100 {{(pid=80244) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:16:22.311376 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9ee5859-a849-4186-8fc0-a1a4a59078f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80245) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:16:22.312224 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9ee5859-a849-4186-8fc0-a1a4a59078f4 None admin] REQUEST_METHOD: `GET` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:16:22.312508 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9ee5859-a849-4186-8fc0-a1a4a59078f4 None admin] SCRIPT_NAME: `/identity` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:16:22.312780 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9ee5859-a849-4186-8fc0-a1a4a59078f4 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:16:22.321595 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-594d2dd6-35cb-46b7-8bdb-47a693162541 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:16:22.322460 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-594d2dd6-35cb-46b7-8bdb-47a693162541 None admin] REQUEST_METHOD: `GET` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:16:22.322736 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-594d2dd6-35cb-46b7-8bdb-47a693162541 None admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:16:22.323008 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-594d2dd6-35cb-46b7-8bdb-47a693162541 None admin] PATH_INFO: `/v3/projects/eab7e5ec54c94574ad68c89747ad5bcc` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:16:22.325434 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-594d2dd6-35cb-46b7-8bdb-47a693162541 None admin] RBAC: Authorizing `identity:get_project(project_id=eab7e5ec54c94574ad68c89747ad5bcc)` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:16:22.326900 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-594d2dd6-35cb-46b7-8bdb-47a693162541 None admin] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:16:22.331063 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 85/166] 10.222.0.47 () {56 vars in 1186 bytes} [Thu Jul 21 08:16:22 2022] GET /identity/v3/projects/eab7e5ec54c94574ad68c89747ad5bcc => generated 298 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 08:16:22.332301 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3de2b59c-1a5e-4998-ae73-c132b12d4699 None None] REQUEST_METHOD: `POST` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:16:22.334773 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3de2b59c-1a5e-4998-ae73-c132b12d4699 None None] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:16:22.335058 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3de2b59c-1a5e-4998-ae73-c132b12d4699 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:16:22.337170 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9ee5859-a849-4186-8fc0-a1a4a59078f4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 21 08:16:22.349001 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application [None req-f9ee5859-a849-4186-8fc0-a1a4a59078f4 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 21 08:16:22.349001 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 08:16:22.349001 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 08:16:22.349001 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 08:16:22.349001 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 08:16:22.349001 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 08:16:22.349001 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 08:16:22.349001 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 08:16:22.349001 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 08:16:22.349001 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 08:16:22.349001 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 08:16:22.349001 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 08:16:22.349001 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 21 08:16:22.349001 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 21 08:16:22.349001 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 21 08:16:22.349001 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 21 08:16:22.349993 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 21 08:16:22.349993 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 21 08:16:22.349993 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 21 08:16:22.349993 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 21 08:16:22.349993 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 08:16:22.349993 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 08:16:22.349993 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 08:16:22.349993 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 08:16:22.349993 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 08:16:22.349993 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 08:16:22.349993 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 08:16:22.349993 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application with Lock( Jul 21 08:16:22.349993 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 08:16:22.349993 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application return self._enter() Jul 21 08:16:22.349993 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 08:16:22.349993 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 08:16:22.350882 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 08:16:22.350882 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application return self.creator() Jul 21 08:16:22.350882 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 08:16:22.350882 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application created_value = creator( Jul 21 08:16:22.350882 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 21 08:16:22.350882 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 21 08:16:22.350882 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 21 08:16:22.350882 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 21 08:16:22.350882 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 21 08:16:22.350882 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 21 08:16:22.350882 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 21 08:16:22.350882 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application Jul 21 08:16:22.350882 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: [pid: 80245|app: 0|req: 82/167] 10.222.0.47 () {56 vars in 1114 bytes} [Thu Jul 21 08:16:22 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 277 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 21 08:16:22.351544 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9ee5859-a849-4186-8fc0-a1a4a59078f4 None admin] Authenticating user token {{(pid=80245) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:16:22.370275 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.auth.core [None req-3de2b59c-1a5e-4998-ae73-c132b12d4699 None None] MFA Rules not processed for user `2dce40b8d74c41ec94cc36e6ff05d412`. Rule list: `[]` (Enabled: `True`). {{(pid=80244) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 08:16:22.371477 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.fernet_utils [None req-3de2b59c-1a5e-4998-ae73-c132b12d4699 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80244) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 08:16:22.544702 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 86/168] 10.222.0.47 () {58 vars in 1023 bytes} [Thu Jul 21 08:16:22 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 213 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 21 08:16:22.546060 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3de2b59c-1a5e-4998-ae73-c132b12d4699 None None] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:16:22.586180 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9ee5859-a849-4186-8fc0-a1a4a59078f4 None admin] Validating token access rules against request {{(pid=80245) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:16:22.644241 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef081ca1-11de-4b26-a9cf-8aa274de81c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80245) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:16:22.645416 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef081ca1-11de-4b26-a9cf-8aa274de81c9 None admin] REQUEST_METHOD: `GET` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:16:22.645922 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef081ca1-11de-4b26-a9cf-8aa274de81c9 None admin] SCRIPT_NAME: `/identity` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:16:22.646391 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef081ca1-11de-4b26-a9cf-8aa274de81c9 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:16:22.647506 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef081ca1-11de-4b26-a9cf-8aa274de81c9 None admin] RBAC: Adding query filter params (role.id=3a074738221e4601b5205715ba87d240, scope.project.id=eab7e5ec54c94574ad68c89747ad5bcc, user.id=39bf6631ce574c758a674faedecf2254) {{(pid=80245) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 08:16:22.648340 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef081ca1-11de-4b26-a9cf-8aa274de81c9 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:16:22.650145 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef081ca1-11de-4b26-a9cf-8aa274de81c9 None admin] RBAC: Authorization granted {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:16:22.665460 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: [pid: 80245|app: 0|req: 83/169] 10.222.0.47 () {56 vars in 1374 bytes} [Thu Jul 21 08:16:22 2022] GET /identity/v3/role_assignments?user.id=39bf6631ce574c758a674faedecf2254&scope.project.id=eab7e5ec54c94574ad68c89747ad5bcc&role.id=3a074738221e4601b5205715ba87d240 => generated 608 bytes in 315 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 08:16:22.666567 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef081ca1-11de-4b26-a9cf-8aa274de81c9 None admin] Authenticating user token {{(pid=80245) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:16:22.722691 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG dbcounter [-] [80245] Writing DB stats keystone:SELECT=100 {{(pid=80245) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:16:22.820656 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3de2b59c-1a5e-4998-ae73-c132b12d4699 None None] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:16:22.840998 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG dbcounter [-] [80244] Writing DB stats keystone:SELECT=100 {{(pid=80244) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:16:22.868016 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20f76896-76ca-4960-aa45-95c6fe6009a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:16:22.868701 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20f76896-76ca-4960-aa45-95c6fe6009a2 None admin] REQUEST_METHOD: `GET` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:16:22.868981 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20f76896-76ca-4960-aa45-95c6fe6009a2 None admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:16:22.869253 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20f76896-76ca-4960-aa45-95c6fe6009a2 None admin] PATH_INFO: `/v3/roles` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:16:22.870012 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20f76896-76ca-4960-aa45-95c6fe6009a2 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80244) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 08:16:22.870320 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20f76896-76ca-4960-aa45-95c6fe6009a2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:16:22.871726 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20f76896-76ca-4960-aa45-95c6fe6009a2 None admin] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:16:22.878429 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef081ca1-11de-4b26-a9cf-8aa274de81c9 None admin] Validating token access rules against request {{(pid=80245) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:16:22.887412 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 87/170] 10.222.0.47 () {56 vars in 1111 bytes} [Thu Jul 21 08:16:22 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 342 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 08:16:22.893395 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20f76896-76ca-4960-aa45-95c6fe6009a2 None admin] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:16:22.944962 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce0166e2-cda9-44e3-8a6b-a4b5d8639e6f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80245) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:16:22.946604 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce0166e2-cda9-44e3-8a6b-a4b5d8639e6f None admin] REQUEST_METHOD: `GET` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:16:22.947336 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce0166e2-cda9-44e3-8a6b-a4b5d8639e6f None admin] SCRIPT_NAME: `/identity` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:16:22.947755 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce0166e2-cda9-44e3-8a6b-a4b5d8639e6f None admin] PATH_INFO: `/v3/roles/member` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:16:22.975645 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce0166e2-cda9-44e3-8a6b-a4b5d8639e6f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 21 08:16:22.988087 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application [None req-ce0166e2-cda9-44e3-8a6b-a4b5d8639e6f None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 21 08:16:22.988087 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 08:16:22.988087 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 08:16:22.988087 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 08:16:22.988087 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 08:16:22.988087 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 08:16:22.988087 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 08:16:22.988087 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 08:16:22.988087 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 08:16:22.988087 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 08:16:22.988087 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 08:16:22.988087 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 08:16:22.988087 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 21 08:16:22.988087 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 21 08:16:22.988087 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 21 08:16:22.988087 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 21 08:16:22.988974 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 21 08:16:22.988974 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 21 08:16:22.988974 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 21 08:16:22.988974 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 21 08:16:22.988974 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 08:16:22.988974 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 08:16:22.988974 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 08:16:22.988974 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 08:16:22.988974 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 08:16:22.988974 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 08:16:22.988974 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 08:16:22.988974 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application with Lock( Jul 21 08:16:22.988974 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 08:16:22.988974 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application return self._enter() Jul 21 08:16:22.988974 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 08:16:22.988974 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 08:16:22.989775 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 08:16:22.989775 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application return self.creator() Jul 21 08:16:22.989775 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 08:16:22.989775 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application created_value = creator( Jul 21 08:16:22.989775 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 21 08:16:22.989775 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 21 08:16:22.989775 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 21 08:16:22.989775 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 21 08:16:22.989775 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 21 08:16:22.989775 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 21 08:16:22.989775 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 21 08:16:22.989775 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application Jul 21 08:16:22.992748 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: [pid: 80245|app: 0|req: 84/171] 10.222.0.47 () {56 vars in 1099 bytes} [Thu Jul 21 08:16:22 2022] GET /identity/v3/roles/member => generated 84 bytes in 326 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 08:16:23.001708 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce0166e2-cda9-44e3-8a6b-a4b5d8639e6f None admin] Authenticating user token {{(pid=80245) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:16:23.036451 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20f76896-76ca-4960-aa45-95c6fe6009a2 None admin] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:16:23.078894 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-792a5998-d528-4890-8c25-e2784bb4f617 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:16:23.079792 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-792a5998-d528-4890-8c25-e2784bb4f617 None admin] REQUEST_METHOD: `GET` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:16:23.080071 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-792a5998-d528-4890-8c25-e2784bb4f617 None admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:16:23.080340 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-792a5998-d528-4890-8c25-e2784bb4f617 None admin] PATH_INFO: `/v3/users/39bf6631ce574c758a674faedecf2254` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:16:23.081578 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-792a5998-d528-4890-8c25-e2784bb4f617 None admin] RBAC: Authorizing `identity:get_user(user_id=39bf6631ce574c758a674faedecf2254)` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:16:23.083030 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-792a5998-d528-4890-8c25-e2784bb4f617 None admin] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:16:23.084752 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 88/172] 10.222.0.47 () {56 vars in 1177 bytes} [Thu Jul 21 08:16:22 2022] GET /identity/v3/users/39bf6631ce574c758a674faedecf2254 => generated 275 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 21 08:16:23.090150 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-792a5998-d528-4890-8c25-e2784bb4f617 None admin] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:16:23.160791 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG dbcounter [-] [80244] Writing DB stats keystone:SELECT=100 {{(pid=80244) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:16:23.195752 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG dbcounter [-] [80245] Writing DB stats keystone:SELECT=100 {{(pid=80245) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:16:23.233340 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce0166e2-cda9-44e3-8a6b-a4b5d8639e6f None admin] Validating token access rules against request {{(pid=80245) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:16:23.246544 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-792a5998-d528-4890-8c25-e2784bb4f617 None admin] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:16:23.276929 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-150b5e95-da68-4ae7-a835-0c9edcd41931 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80245) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:16:23.277863 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-150b5e95-da68-4ae7-a835-0c9edcd41931 None admin] REQUEST_METHOD: `GET` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:16:23.278558 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-150b5e95-da68-4ae7-a835-0c9edcd41931 None admin] SCRIPT_NAME: `/identity` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:16:23.278834 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-150b5e95-da68-4ae7-a835-0c9edcd41931 None admin] PATH_INFO: `/v3/roles` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:16:23.279981 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-150b5e95-da68-4ae7-a835-0c9edcd41931 None admin] RBAC: Adding query filter params (name=member) {{(pid=80245) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 08:16:23.280558 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-150b5e95-da68-4ae7-a835-0c9edcd41931 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:16:23.282414 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-150b5e95-da68-4ae7-a835-0c9edcd41931 None admin] RBAC: Authorization granted {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:16:23.290206 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-485fcb95-9842-4d42-aca1-9fad61bb7859 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:16:23.290881 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-485fcb95-9842-4d42-aca1-9fad61bb7859 None admin] REQUEST_METHOD: `GET` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:16:23.291163 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-485fcb95-9842-4d42-aca1-9fad61bb7859 None admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:16:23.291482 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-485fcb95-9842-4d42-aca1-9fad61bb7859 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:16:23.293084 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: [pid: 80245|app: 0|req: 85/173] 10.222.0.47 () {56 vars in 1101 bytes} [Thu Jul 21 08:16:23 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 292 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 08:16:23.294821 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-018b1975-b48c-4c1a-b34c-6636627490db None None] REQUEST_METHOD: `GET` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:16:23.295292 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-018b1975-b48c-4c1a-b34c-6636627490db None None] SCRIPT_NAME: `/identity` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:16:23.296700 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-018b1975-b48c-4c1a-b34c-6636627490db None None] PATH_INFO: `/` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:16:23.298240 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: [pid: 80245|app: 0|req: 86/174] 10.222.0.47 () {54 vars in 926 bytes} [Thu Jul 21 08:16:23 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 08:16:23.299176 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-018b1975-b48c-4c1a-b34c-6636627490db None None] Authenticating user token {{(pid=80245) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:16:23.387908 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-485fcb95-9842-4d42-aca1-9fad61bb7859 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:16:23.389592 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-485fcb95-9842-4d42-aca1-9fad61bb7859 None admin] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:16:23.435069 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-018b1975-b48c-4c1a-b34c-6636627490db None None] Validating token access rules against request {{(pid=80245) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:16:23.440235 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG dbcounter [-] [80244] Writing DB stats keystone:SELECT=100 {{(pid=80244) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:16:23.487350 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c46a4dc0-8d68-4963-b772-48d81c192f7f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80245) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:16:23.487350 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c46a4dc0-8d68-4963-b772-48d81c192f7f None admin] REQUEST_METHOD: `GET` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:16:23.487350 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c46a4dc0-8d68-4963-b772-48d81c192f7f None admin] SCRIPT_NAME: `/identity` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:16:23.487350 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c46a4dc0-8d68-4963-b772-48d81c192f7f None admin] PATH_INFO: `/v3/users/39bf6631ce574c758a674faedecf2254` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:16:23.487709 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c46a4dc0-8d68-4963-b772-48d81c192f7f None admin] RBAC: Authorizing `identity:get_user(user_id=39bf6631ce574c758a674faedecf2254)` {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:16:23.487709 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c46a4dc0-8d68-4963-b772-48d81c192f7f None admin] RBAC: Authorization granted {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:16:23.491096 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: [pid: 80245|app: 0|req: 87/175] 10.222.0.47 () {56 vars in 1177 bytes} [Thu Jul 21 08:16:23 2022] GET /identity/v3/users/39bf6631ce574c758a674faedecf2254 => generated 275 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 08:16:23.491096 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac2e0d82-dee9-428f-935b-c75ce6d6abc3 None None] REQUEST_METHOD: `POST` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:16:23.491096 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac2e0d82-dee9-428f-935b-c75ce6d6abc3 None None] SCRIPT_NAME: `/identity` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:16:23.491096 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac2e0d82-dee9-428f-935b-c75ce6d6abc3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:16:23.504683 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 89/176] 10.222.0.47 () {58 vars in 1282 bytes} [Thu Jul 21 08:16:23 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 415 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Jul 21 08:16:23.510946 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eacc2af-72c4-4294-a6ef-db635b2a8d8c None None] REQUEST_METHOD: `GET` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:16:23.511068 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.auth.core [None req-ac2e0d82-dee9-428f-935b-c75ce6d6abc3 None None] MFA Rules not processed for user `2dce40b8d74c41ec94cc36e6ff05d412`. Rule list: `[]` (Enabled: `True`). {{(pid=80245) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 08:16:23.511895 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eacc2af-72c4-4294-a6ef-db635b2a8d8c None None] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:16:23.511997 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.fernet_utils [None req-ac2e0d82-dee9-428f-935b-c75ce6d6abc3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80245) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 08:16:23.512464 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eacc2af-72c4-4294-a6ef-db635b2a8d8c None None] PATH_INFO: `/` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:16:23.514195 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 90/177] 10.222.0.47 () {54 vars in 926 bytes} [Thu Jul 21 08:16:23 2022] GET /identity => generated 270 bytes in 9 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 08:16:23.515208 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eacc2af-72c4-4294-a6ef-db635b2a8d8c None None] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:16:23.525913 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG dbcounter [-] [80245] Writing DB stats keystone:SELECT=100 {{(pid=80245) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:16:23.589330 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: [pid: 80245|app: 0|req: 88/178] 10.222.0.47 () {58 vars in 1023 bytes} [Thu Jul 21 08:16:23 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 21 08:16:23.590545 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac2e0d82-dee9-428f-935b-c75ce6d6abc3 None None] Authenticating user token {{(pid=80245) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:16:23.666476 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eacc2af-72c4-4294-a6ef-db635b2a8d8c None None] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:16:23.712506 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f8acd2c-8db3-42ae-b0af-7a8cec7cef23 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:16:23.713073 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f8acd2c-8db3-42ae-b0af-7a8cec7cef23 None admin] REQUEST_METHOD: `GET` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:16:23.713354 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f8acd2c-8db3-42ae-b0af-7a8cec7cef23 None admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:16:23.713515 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac2e0d82-dee9-428f-935b-c75ce6d6abc3 None None] Validating token access rules against request {{(pid=80245) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:16:23.713737 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f8acd2c-8db3-42ae-b0af-7a8cec7cef23 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:16:23.730164 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG dbcounter [-] [80244] Writing DB stats keystone:SELECT=100 {{(pid=80244) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:16:23.749926 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ff51729-4f9b-4c67-8ca5-8b891522dec5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80245) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:16:23.750805 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ff51729-4f9b-4c67-8ca5-8b891522dec5 None admin] REQUEST_METHOD: `GET` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:16:23.751083 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ff51729-4f9b-4c67-8ca5-8b891522dec5 None admin] SCRIPT_NAME: `/identity` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:16:23.751407 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ff51729-4f9b-4c67-8ca5-8b891522dec5 None admin] PATH_INFO: `/v3/projects/eab7e5ec54c94574ad68c89747ad5bcc` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:16:23.752575 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ff51729-4f9b-4c67-8ca5-8b891522dec5 None admin] RBAC: Authorizing `identity:get_project(project_id=eab7e5ec54c94574ad68c89747ad5bcc)` {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:16:23.754018 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ff51729-4f9b-4c67-8ca5-8b891522dec5 None admin] RBAC: Authorization granted {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:16:23.755963 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: [pid: 80245|app: 0|req: 89/179] 10.222.0.47 () {56 vars in 1186 bytes} [Thu Jul 21 08:16:23 2022] GET /identity/v3/projects/eab7e5ec54c94574ad68c89747ad5bcc => generated 298 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 08:16:23.758165 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efa79abc-d2e2-4f8d-a1f9-4271a9676c12 None None] REQUEST_METHOD: `POST` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:16:23.758452 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efa79abc-d2e2-4f8d-a1f9-4271a9676c12 None None] SCRIPT_NAME: `/identity` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:16:23.758726 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efa79abc-d2e2-4f8d-a1f9-4271a9676c12 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:16:23.775766 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.auth.core [None req-efa79abc-d2e2-4f8d-a1f9-4271a9676c12 None None] MFA Rules not processed for user `2dce40b8d74c41ec94cc36e6ff05d412`. Rule list: `[]` (Enabled: `True`). {{(pid=80245) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 08:16:23.776984 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.fernet_utils [None req-efa79abc-d2e2-4f8d-a1f9-4271a9676c12 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80245) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 08:16:23.785490 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG dbcounter [-] [80245] Writing DB stats keystone:SELECT=100 {{(pid=80245) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:16:23.795194 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f8acd2c-8db3-42ae-b0af-7a8cec7cef23 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:16:23.796797 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f8acd2c-8db3-42ae-b0af-7a8cec7cef23 None admin] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:16:23.848409 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: [pid: 80245|app: 0|req: 90/180] 10.222.0.47 () {58 vars in 1023 bytes} [Thu Jul 21 08:16:23 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 21 08:16:23.859541 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efa79abc-d2e2-4f8d-a1f9-4271a9676c12 None None] Authenticating user token {{(pid=80245) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:16:23.903435 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 91/181] 10.222.0.47 () {58 vars in 1282 bytes} [Thu Jul 21 08:16:23 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 385 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 21 08:16:23.903435 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18182009-6603-4f88-96e0-9f829bde8e6a None None] REQUEST_METHOD: `GET` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:16:23.903435 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18182009-6603-4f88-96e0-9f829bde8e6a None None] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:16:23.903435 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18182009-6603-4f88-96e0-9f829bde8e6a None None] PATH_INFO: `/` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:16:23.903435 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 92/182] 10.222.0.47 () {54 vars in 926 bytes} [Thu Jul 21 08:16:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 08:16:23.903988 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18182009-6603-4f88-96e0-9f829bde8e6a None None] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:16:23.952322 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efa79abc-d2e2-4f8d-a1f9-4271a9676c12 None None] Validating token access rules against request {{(pid=80245) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:16:23.953280 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG dbcounter [-] [80244] Writing DB stats keystone:SELECT=100 {{(pid=80244) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:16:23.988295 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9ae2790-5b92-4034-96fb-f4bfd0105614 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80245) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:16:23.989016 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9ae2790-5b92-4034-96fb-f4bfd0105614 None admin] REQUEST_METHOD: `GET` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:16:23.989301 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9ae2790-5b92-4034-96fb-f4bfd0105614 None admin] SCRIPT_NAME: `/identity` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:16:23.989574 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9ae2790-5b92-4034-96fb-f4bfd0105614 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:16:24.000248 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9ae2790-5b92-4034-96fb-f4bfd0105614 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 21 08:16:24.007837 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application [None req-e9ae2790-5b92-4034-96fb-f4bfd0105614 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 21 08:16:24.007837 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 08:16:24.007837 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 08:16:24.007837 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 08:16:24.007837 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 08:16:24.007837 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 08:16:24.007837 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 08:16:24.007837 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 08:16:24.007837 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 08:16:24.007837 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 08:16:24.007837 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 08:16:24.007837 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 08:16:24.007837 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 21 08:16:24.007837 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 21 08:16:24.007837 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 21 08:16:24.007837 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 21 08:16:24.008949 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 21 08:16:24.008949 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 21 08:16:24.008949 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 21 08:16:24.008949 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 21 08:16:24.008949 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 08:16:24.008949 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 08:16:24.008949 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 08:16:24.008949 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 08:16:24.008949 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 08:16:24.008949 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 08:16:24.008949 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 08:16:24.008949 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application with Lock( Jul 21 08:16:24.008949 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 08:16:24.008949 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application return self._enter() Jul 21 08:16:24.008949 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 08:16:24.008949 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 08:16:24.009700 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 08:16:24.009700 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application return self.creator() Jul 21 08:16:24.009700 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 08:16:24.009700 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application created_value = creator( Jul 21 08:16:24.009700 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 21 08:16:24.009700 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 21 08:16:24.009700 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 21 08:16:24.009700 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 21 08:16:24.009700 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 21 08:16:24.009700 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 21 08:16:24.009700 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 21 08:16:24.009700 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application Jul 21 08:16:24.009700 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: [pid: 80245|app: 0|req: 91/183] 10.222.0.47 () {56 vars in 1096 bytes} [Thu Jul 21 08:16:23 2022] GET /identity/v3/roles/admin => generated 83 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 08:16:24.010525 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a692c3be-56e6-47b1-b412-cc8377b7d225 None None] REQUEST_METHOD: `GET` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:16:24.010752 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a692c3be-56e6-47b1-b412-cc8377b7d225 None None] SCRIPT_NAME: `/identity` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:16:24.011099 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a692c3be-56e6-47b1-b412-cc8377b7d225 None None] PATH_INFO: `/` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:16:24.015441 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: [pid: 80245|app: 0|req: 92/184] 10.222.0.47 () {54 vars in 926 bytes} [Thu Jul 21 08:16:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 08:16:24.015571 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18182009-6603-4f88-96e0-9f829bde8e6a None None] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:16:24.015651 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a692c3be-56e6-47b1-b412-cc8377b7d225 None None] Authenticating user token {{(pid=80245) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:16:24.018349 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG dbcounter [-] [80245] Writing DB stats keystone:SELECT=100 {{(pid=80245) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:16:24.047275 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00697a54-9ee5-48f0-ab67-1637eda75d5d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:16:24.047958 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00697a54-9ee5-48f0-ab67-1637eda75d5d None admin] REQUEST_METHOD: `GET` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:16:24.048245 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00697a54-9ee5-48f0-ab67-1637eda75d5d None admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:16:24.048521 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00697a54-9ee5-48f0-ab67-1637eda75d5d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:16:24.049362 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00697a54-9ee5-48f0-ab67-1637eda75d5d None admin] RBAC: Adding query filter params (role.id=99a638c0aac44b79a1f2858574724495, scope.project.id=eab7e5ec54c94574ad68c89747ad5bcc, user.id=39bf6631ce574c758a674faedecf2254) {{(pid=80244) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 08:16:24.049680 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00697a54-9ee5-48f0-ab67-1637eda75d5d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:16:24.051142 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00697a54-9ee5-48f0-ab67-1637eda75d5d None admin] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:16:24.057633 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 93/185] 10.222.0.47 () {56 vars in 1374 bytes} [Thu Jul 21 08:16:23 2022] GET /identity/v3/role_assignments?user.id=39bf6631ce574c758a674faedecf2254&scope.project.id=eab7e5ec54c94574ad68c89747ad5bcc&role.id=99a638c0aac44b79a1f2858574724495 => generated 608 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 08:16:24.058460 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00697a54-9ee5-48f0-ab67-1637eda75d5d None admin] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:16:24.130018 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a692c3be-56e6-47b1-b412-cc8377b7d225 None None] Validating token access rules against request {{(pid=80245) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:16:24.168284 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-462d9130-bf0b-41d6-b7d2-d7ed41472e58 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80245) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:16:24.169117 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-462d9130-bf0b-41d6-b7d2-d7ed41472e58 None admin] REQUEST_METHOD: `GET` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:16:24.169405 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-462d9130-bf0b-41d6-b7d2-d7ed41472e58 None admin] SCRIPT_NAME: `/identity` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:16:24.169773 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-462d9130-bf0b-41d6-b7d2-d7ed41472e58 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:16:24.182360 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-462d9130-bf0b-41d6-b7d2-d7ed41472e58 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 21 08:16:24.190355 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application [None req-462d9130-bf0b-41d6-b7d2-d7ed41472e58 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jul 21 08:16:24.190355 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 08:16:24.190355 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 08:16:24.190355 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 08:16:24.190355 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 08:16:24.190355 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 08:16:24.190355 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 08:16:24.190355 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 08:16:24.190355 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 08:16:24.190355 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 08:16:24.190355 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 08:16:24.190355 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 08:16:24.190355 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 21 08:16:24.190355 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 21 08:16:24.190355 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 21 08:16:24.190355 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 21 08:16:24.191265 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 21 08:16:24.191265 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 21 08:16:24.191265 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 21 08:16:24.191265 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 21 08:16:24.191265 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 08:16:24.191265 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 08:16:24.191265 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 08:16:24.191265 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 08:16:24.191265 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 08:16:24.191265 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 08:16:24.191265 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 08:16:24.191265 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application with Lock( Jul 21 08:16:24.191265 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 08:16:24.191265 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application return self._enter() Jul 21 08:16:24.191265 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 08:16:24.191265 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 08:16:24.192225 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 08:16:24.192225 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application return self.creator() Jul 21 08:16:24.192225 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 08:16:24.192225 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application created_value = creator( Jul 21 08:16:24.192225 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 21 08:16:24.192225 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 21 08:16:24.192225 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 21 08:16:24.192225 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 21 08:16:24.192225 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 21 08:16:24.192225 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 21 08:16:24.192225 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jul 21 08:16:24.192225 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application Jul 21 08:16:24.192225 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: [pid: 80245|app: 0|req: 93/186] 10.222.0.47 () {56 vars in 1099 bytes} [Thu Jul 21 08:16:24 2022] GET /identity/v3/roles/reader => generated 84 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 08:16:24.197415 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d931d3df-350c-4269-9e93-b16747688449 None None] REQUEST_METHOD: `POST` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:16:24.197926 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d931d3df-350c-4269-9e93-b16747688449 None None] SCRIPT_NAME: `/identity` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:16:24.198271 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00697a54-9ee5-48f0-ab67-1637eda75d5d None admin] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:16:24.198844 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d931d3df-350c-4269-9e93-b16747688449 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:16:24.210388 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG dbcounter [-] [80244] Writing DB stats keystone:SELECT=100 {{(pid=80244) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:16:24.222525 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.auth.core [None req-d931d3df-350c-4269-9e93-b16747688449 None None] MFA Rules not processed for user `2dce40b8d74c41ec94cc36e6ff05d412`. Rule list: `[]` (Enabled: `True`). {{(pid=80245) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 08:16:24.226263 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.fernet_utils [None req-d931d3df-350c-4269-9e93-b16747688449 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80245) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 08:16:24.244373 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0a7f4c4-ae90-4019-a56e-ec9fb6b7a3c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:16:24.245652 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0a7f4c4-ae90-4019-a56e-ec9fb6b7a3c6 None admin] REQUEST_METHOD: `GET` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:16:24.245942 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0a7f4c4-ae90-4019-a56e-ec9fb6b7a3c6 None admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:16:24.246222 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0a7f4c4-ae90-4019-a56e-ec9fb6b7a3c6 None admin] PATH_INFO: `/v3/projects/d77ff705f0ca44808214e0d3b6c4a1cf` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:16:24.247477 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0a7f4c4-ae90-4019-a56e-ec9fb6b7a3c6 None admin] RBAC: Authorizing `identity:get_project(project_id=d77ff705f0ca44808214e0d3b6c4a1cf)` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:16:24.249511 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0a7f4c4-ae90-4019-a56e-ec9fb6b7a3c6 None admin] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:16:24.250756 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 94/187] 10.222.0.47 () {56 vars in 1186 bytes} [Thu Jul 21 08:16:24 2022] GET /identity/v3/projects/d77ff705f0ca44808214e0d3b6c4a1cf => generated 312 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 08:16:24.251565 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0a7f4c4-ae90-4019-a56e-ec9fb6b7a3c6 None admin] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:16:24.299911 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG dbcounter [-] [80245] Writing DB stats keystone:SELECT=100 {{(pid=80245) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:16:24.312114 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: [pid: 80245|app: 0|req: 94/188] 10.222.0.47 () {58 vars in 1023 bytes} [Thu Jul 21 08:16:24 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 21 08:16:24.313391 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b7d41a6-5210-4b28-939a-432c92f1562d None None] REQUEST_METHOD: `POST` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:16:24.313679 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b7d41a6-5210-4b28-939a-432c92f1562d None None] SCRIPT_NAME: `/identity` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:16:24.313959 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b7d41a6-5210-4b28-939a-432c92f1562d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:16:24.329907 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.auth.core [None req-5b7d41a6-5210-4b28-939a-432c92f1562d None None] MFA Rules not processed for user `2dce40b8d74c41ec94cc36e6ff05d412`. Rule list: `[]` (Enabled: `True`). {{(pid=80245) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 08:16:24.331094 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.fernet_utils [None req-5b7d41a6-5210-4b28-939a-432c92f1562d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80245) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 08:16:24.370266 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0a7f4c4-ae90-4019-a56e-ec9fb6b7a3c6 None admin] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:16:24.398466 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: [pid: 80245|app: 0|req: 95/189] 10.222.0.47 () {58 vars in 1023 bytes} [Thu Jul 21 08:16:24 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 21 08:16:24.399424 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b7d41a6-5210-4b28-939a-432c92f1562d None None] Authenticating user token {{(pid=80245) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:16:24.405221 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-251b8147-f1dc-4cb7-99b9-b0801ff4223e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:16:24.406221 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-251b8147-f1dc-4cb7-99b9-b0801ff4223e None admin] REQUEST_METHOD: `GET` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:16:24.406886 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-251b8147-f1dc-4cb7-99b9-b0801ff4223e None admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:16:24.407242 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-251b8147-f1dc-4cb7-99b9-b0801ff4223e None admin] PATH_INFO: `/v3/roles` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:16:24.408072 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-251b8147-f1dc-4cb7-99b9-b0801ff4223e None admin] RBAC: Adding query filter params (name=admin) {{(pid=80244) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 08:16:24.408421 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-251b8147-f1dc-4cb7-99b9-b0801ff4223e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:16:24.409991 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-251b8147-f1dc-4cb7-99b9-b0801ff4223e None admin] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:16:24.424139 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 95/190] 10.222.0.47 () {56 vars in 1099 bytes} [Thu Jul 21 08:16:24 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 08:16:24.424139 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-251b8147-f1dc-4cb7-99b9-b0801ff4223e None admin] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:16:24.462187 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG dbcounter [-] [80244] Writing DB stats keystone:SELECT=100 {{(pid=80244) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:16:24.503975 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b7d41a6-5210-4b28-939a-432c92f1562d None None] Validating token access rules against request {{(pid=80245) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:16:24.525990 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-251b8147-f1dc-4cb7-99b9-b0801ff4223e None admin] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:16:24.526859 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG dbcounter [-] [80245] Writing DB stats keystone:SELECT=100 {{(pid=80245) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:16:24.539075 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7160d52-e2c2-43fa-99c7-52f8c6e3488e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80245) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:16:24.539778 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7160d52-e2c2-43fa-99c7-52f8c6e3488e None admin] REQUEST_METHOD: `GET` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:16:24.540052 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7160d52-e2c2-43fa-99c7-52f8c6e3488e None admin] SCRIPT_NAME: `/identity` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:16:24.540328 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7160d52-e2c2-43fa-99c7-52f8c6e3488e None admin] PATH_INFO: `/v3/roles` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:16:24.541068 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7160d52-e2c2-43fa-99c7-52f8c6e3488e None admin] RBAC: Adding query filter params (name=reader) {{(pid=80245) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 08:16:24.541375 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7160d52-e2c2-43fa-99c7-52f8c6e3488e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:16:24.542758 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7160d52-e2c2-43fa-99c7-52f8c6e3488e None admin] RBAC: Authorization granted {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:16:24.550817 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: [pid: 80245|app: 0|req: 96/191] 10.222.0.47 () {56 vars in 1101 bytes} [Thu Jul 21 08:16:24 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 08:16:24.551717 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7160d52-e2c2-43fa-99c7-52f8c6e3488e None admin] Authenticating user token {{(pid=80245) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:16:24.562214 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b2ab4ab-28e0-4dce-b76a-5572ec6a5f91 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:16:24.563319 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b2ab4ab-28e0-4dce-b76a-5572ec6a5f91 None admin] REQUEST_METHOD: `GET` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:16:24.563843 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b2ab4ab-28e0-4dce-b76a-5572ec6a5f91 None admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:16:24.564371 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b2ab4ab-28e0-4dce-b76a-5572ec6a5f91 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:16:24.565946 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b2ab4ab-28e0-4dce-b76a-5572ec6a5f91 None admin] RBAC: Adding query filter params (role.id=3a074738221e4601b5205715ba87d240, scope.project.id=d77ff705f0ca44808214e0d3b6c4a1cf, user.id=39bf6631ce574c758a674faedecf2254) {{(pid=80244) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 08:16:24.568715 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b2ab4ab-28e0-4dce-b76a-5572ec6a5f91 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:16:24.568715 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b2ab4ab-28e0-4dce-b76a-5572ec6a5f91 None admin] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:16:24.575670 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 96/192] 10.222.0.47 () {56 vars in 1374 bytes} [Thu Jul 21 08:16:24 2022] GET /identity/v3/role_assignments?user.id=39bf6631ce574c758a674faedecf2254&scope.project.id=d77ff705f0ca44808214e0d3b6c4a1cf&role.id=3a074738221e4601b5205715ba87d240 => generated 608 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 08:16:24.576952 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b2ab4ab-28e0-4dce-b76a-5572ec6a5f91 None admin] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:16:24.675427 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7160d52-e2c2-43fa-99c7-52f8c6e3488e None admin] Validating token access rules against request {{(pid=80245) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:16:24.704790 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b2ab4ab-28e0-4dce-b76a-5572ec6a5f91 None admin] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:16:24.707446 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG dbcounter [-] [80244] Writing DB stats keystone:SELECT=100 {{(pid=80244) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:16:24.715798 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca9a402c-32e2-4f54-85d4-14bdb9724e6a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80245) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:16:24.717019 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca9a402c-32e2-4f54-85d4-14bdb9724e6a None admin] REQUEST_METHOD: `GET` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:16:24.717303 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca9a402c-32e2-4f54-85d4-14bdb9724e6a None admin] SCRIPT_NAME: `/identity` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:16:24.717578 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca9a402c-32e2-4f54-85d4-14bdb9724e6a None admin] PATH_INFO: `/v3/domains/default` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:16:24.719462 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca9a402c-32e2-4f54-85d4-14bdb9724e6a None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:16:24.721134 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca9a402c-32e2-4f54-85d4-14bdb9724e6a None admin] RBAC: Authorization granted {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:16:24.728488 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: [pid: 80245|app: 0|req: 97/193] 10.222.0.47 () {56 vars in 1108 bytes} [Thu Jul 21 08:16:24 2022] GET /identity/v3/domains/default => generated 199 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 08:16:24.728488 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca9a402c-32e2-4f54-85d4-14bdb9724e6a None admin] Authenticating user token {{(pid=80245) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:16:24.750106 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fda57113-f742-489d-9428-7101d9534bd2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:16:24.751090 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fda57113-f742-489d-9428-7101d9534bd2 None admin] REQUEST_METHOD: `GET` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:16:24.751285 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fda57113-f742-489d-9428-7101d9534bd2 None admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:16:24.751539 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fda57113-f742-489d-9428-7101d9534bd2 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:16:24.764267 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fda57113-f742-489d-9428-7101d9534bd2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 21 08:16:24.772956 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application [None req-fda57113-f742-489d-9428-7101d9534bd2 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 21 08:16:24.772956 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 08:16:24.772956 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 08:16:24.772956 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 08:16:24.772956 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 08:16:24.772956 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 08:16:24.772956 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 08:16:24.772956 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 08:16:24.772956 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 08:16:24.772956 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 08:16:24.772956 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 08:16:24.772956 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 08:16:24.772956 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 21 08:16:24.772956 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 21 08:16:24.772956 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 21 08:16:24.772956 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 21 08:16:24.773826 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 21 08:16:24.773826 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 21 08:16:24.773826 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 21 08:16:24.773826 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 21 08:16:24.773826 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 08:16:24.773826 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 08:16:24.773826 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 08:16:24.773826 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 08:16:24.773826 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 08:16:24.773826 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 08:16:24.773826 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 08:16:24.773826 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application with Lock( Jul 21 08:16:24.773826 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 08:16:24.773826 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self._enter() Jul 21 08:16:24.773826 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 08:16:24.773826 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 08:16:24.774584 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 08:16:24.774584 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self.creator() Jul 21 08:16:24.774584 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 08:16:24.774584 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application created_value = creator( Jul 21 08:16:24.774584 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 21 08:16:24.774584 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 21 08:16:24.774584 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 21 08:16:24.774584 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 21 08:16:24.774584 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 21 08:16:24.774584 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 21 08:16:24.774584 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 21 08:16:24.774584 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application Jul 21 08:16:24.774584 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 97/194] 10.222.0.47 () {56 vars in 1114 bytes} [Thu Jul 21 08:16:24 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 08:16:24.776238 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fda57113-f742-489d-9428-7101d9534bd2 None admin] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:16:24.799057 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG dbcounter [-] [80245] Writing DB stats keystone:SELECT=100 {{(pid=80245) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:16:24.847016 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca9a402c-32e2-4f54-85d4-14bdb9724e6a None admin] Validating token access rules against request {{(pid=80245) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:16:24.880743 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78d98627-cf94-49f3-a118-d03151889c83 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80245) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:16:24.881674 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78d98627-cf94-49f3-a118-d03151889c83 None admin] REQUEST_METHOD: `GET` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:16:24.881960 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78d98627-cf94-49f3-a118-d03151889c83 None admin] SCRIPT_NAME: `/identity` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:16:24.882239 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78d98627-cf94-49f3-a118-d03151889c83 None admin] PATH_INFO: `/v3/users/70ffa790321f4f19b7935a59cb2fcfd9` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:16:24.882888 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fda57113-f742-489d-9428-7101d9534bd2 None admin] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:16:24.897976 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78d98627-cf94-49f3-a118-d03151889c83 None admin] RBAC: Authorizing `identity:get_user(user_id=70ffa790321f4f19b7935a59cb2fcfd9)` {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:16:24.899477 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78d98627-cf94-49f3-a118-d03151889c83 None admin] RBAC: Authorization granted {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:16:24.901214 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: [pid: 80245|app: 0|req: 98/195] 10.222.0.47 () {56 vars in 1177 bytes} [Thu Jul 21 08:16:24 2022] GET /identity/v3/users/70ffa790321f4f19b7935a59cb2fcfd9 => generated 283 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 08:16:24.902063 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78d98627-cf94-49f3-a118-d03151889c83 None admin] Authenticating user token {{(pid=80245) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:16:24.917763 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfe7b613-a781-4f8f-9645-ae2462327376 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:16:24.918656 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfe7b613-a781-4f8f-9645-ae2462327376 None admin] REQUEST_METHOD: `GET` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:16:24.918945 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfe7b613-a781-4f8f-9645-ae2462327376 None admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:16:24.919217 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfe7b613-a781-4f8f-9645-ae2462327376 None admin] PATH_INFO: `/v3/users/62835f20a4f74cb8a96c4979d095b168` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:16:24.920581 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfe7b613-a781-4f8f-9645-ae2462327376 None admin] RBAC: Authorizing `identity:get_user(user_id=62835f20a4f74cb8a96c4979d095b168)` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:16:24.922150 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfe7b613-a781-4f8f-9645-ae2462327376 None admin] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:16:24.924084 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 98/196] 10.222.0.47 () {56 vars in 1177 bytes} [Thu Jul 21 08:16:24 2022] GET /identity/v3/users/62835f20a4f74cb8a96c4979d095b168 => generated 289 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 08:16:24.924718 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfe7b613-a781-4f8f-9645-ae2462327376 None admin] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:16:24.959782 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG dbcounter [-] [80244] Writing DB stats keystone:SELECT=100 {{(pid=80244) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:16:25.010044 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78d98627-cf94-49f3-a118-d03151889c83 None admin] Validating token access rules against request {{(pid=80245) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:16:25.020593 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG dbcounter [-] [80245] Writing DB stats keystone:SELECT=100 {{(pid=80245) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:16:25.033475 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfe7b613-a781-4f8f-9645-ae2462327376 None admin] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:16:25.044645 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecbc90f7-c4db-43a2-9689-1795b55eb835 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80245) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:16:25.045321 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecbc90f7-c4db-43a2-9689-1795b55eb835 None admin] REQUEST_METHOD: `POST` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:16:25.045604 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecbc90f7-c4db-43a2-9689-1795b55eb835 None admin] SCRIPT_NAME: `/identity` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:16:25.045878 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecbc90f7-c4db-43a2-9689-1795b55eb835 None admin] PATH_INFO: `/v3/users` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:16:25.046672 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecbc90f7-c4db-43a2-9689-1795b55eb835 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:16:25.048068 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecbc90f7-c4db-43a2-9689-1795b55eb835 None admin] RBAC: Authorization granted {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:16:25.067615 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: [pid: 80245|app: 0|req: 99/197] 10.222.0.47 () {60 vars in 1132 bytes} [Thu Jul 21 08:16:24 2022] POST /identity/v3/users => generated 297 bytes in 166 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 08:16:25.069599 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-902cdec7-4f55-4fba-aa4d-3b37250007e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:16:25.070341 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-902cdec7-4f55-4fba-aa4d-3b37250007e6 None admin] REQUEST_METHOD: `GET` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:16:25.070631 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-902cdec7-4f55-4fba-aa4d-3b37250007e6 None admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:16:25.070958 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-902cdec7-4f55-4fba-aa4d-3b37250007e6 None admin] PATH_INFO: `/v3/roles` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:16:25.071851 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-902cdec7-4f55-4fba-aa4d-3b37250007e6 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80244) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 08:16:25.072169 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-902cdec7-4f55-4fba-aa4d-3b37250007e6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:16:25.072372 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecbc90f7-c4db-43a2-9689-1795b55eb835 None admin] Authenticating user token {{(pid=80245) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:16:25.073676 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-902cdec7-4f55-4fba-aa4d-3b37250007e6 None admin] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:16:25.084481 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 99/198] 10.222.0.47 () {56 vars in 1111 bytes} [Thu Jul 21 08:16:24 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 08:16:25.085464 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-902cdec7-4f55-4fba-aa4d-3b37250007e6 None admin] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:16:25.195645 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecbc90f7-c4db-43a2-9689-1795b55eb835 None admin] Validating token access rules against request {{(pid=80245) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:16:25.195848 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG dbcounter [-] [80244] Writing DB stats keystone:SELECT=100 {{(pid=80244) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:16:25.204142 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-902cdec7-4f55-4fba-aa4d-3b37250007e6 None admin] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:16:25.243472 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff97c725-17e9-46d7-bba9-457f35ea2cec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80245) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:16:25.244381 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff97c725-17e9-46d7-bba9-457f35ea2cec None admin] REQUEST_METHOD: `GET` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:16:25.246030 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff97c725-17e9-46d7-bba9-457f35ea2cec None admin] SCRIPT_NAME: `/identity` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:16:25.246030 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff97c725-17e9-46d7-bba9-457f35ea2cec None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:16:25.249477 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7b1b877-1ebf-4553-8dd8-2150467eece7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:16:25.250045 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7b1b877-1ebf-4553-8dd8-2150467eece7 None admin] REQUEST_METHOD: `GET` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:16:25.250334 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7b1b877-1ebf-4553-8dd8-2150467eece7 None admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:16:25.250614 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7b1b877-1ebf-4553-8dd8-2150467eece7 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:16:25.287007 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG dbcounter [-] [80245] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80245) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:16:25.328847 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff97c725-17e9-46d7-bba9-457f35ea2cec None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:16:25.330666 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff97c725-17e9-46d7-bba9-457f35ea2cec None admin] RBAC: Authorization granted {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:16:25.342197 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7b1b877-1ebf-4553-8dd8-2150467eece7 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:16:25.344077 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7b1b877-1ebf-4553-8dd8-2150467eece7 None admin] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:16:25.450873 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: [pid: 80245|app: 0|req: 100/199] 10.222.0.47 () {58 vars in 1282 bytes} [Thu Jul 21 08:16:25 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 379 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 21 08:16:25.451749 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff97c725-17e9-46d7-bba9-457f35ea2cec None admin] Authenticating user token {{(pid=80245) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:16:25.479389 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG dbcounter [-] [80244] Writing DB stats keystone:SELECT=100 {{(pid=80244) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:16:25.491421 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 100/200] 10.222.0.47 () {58 vars in 1282 bytes} [Thu Jul 21 08:16:25 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 406 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 21 08:16:25.492379 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7b1b877-1ebf-4553-8dd8-2150467eece7 None admin] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:16:25.548777 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG dbcounter [-] [80245] Writing DB stats keystone:SELECT=100 {{(pid=80245) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:16:25.599369 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff97c725-17e9-46d7-bba9-457f35ea2cec None admin] Validating token access rules against request {{(pid=80245) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:16:25.605738 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7b1b877-1ebf-4553-8dd8-2150467eece7 None admin] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:16:25.637694 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-026565be-7889-46d7-a6da-d2be1e1de649 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80245) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:16:25.638692 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-026565be-7889-46d7-a6da-d2be1e1de649 None admin] REQUEST_METHOD: `GET` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:16:25.639058 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-026565be-7889-46d7-a6da-d2be1e1de649 None admin] SCRIPT_NAME: `/identity` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:16:25.639440 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-026565be-7889-46d7-a6da-d2be1e1de649 None admin] PATH_INFO: `/v3/users/70ffa790321f4f19b7935a59cb2fcfd9` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:16:25.640738 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-026565be-7889-46d7-a6da-d2be1e1de649 None admin] RBAC: Authorizing `identity:get_user(user_id=70ffa790321f4f19b7935a59cb2fcfd9)` {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:16:25.642270 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-026565be-7889-46d7-a6da-d2be1e1de649 None admin] RBAC: Authorization granted {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:16:25.644412 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: [pid: 80245|app: 0|req: 101/201] 10.222.0.47 () {56 vars in 1177 bytes} [Thu Jul 21 08:16:25 2022] GET /identity/v3/users/70ffa790321f4f19b7935a59cb2fcfd9 => generated 283 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 08:16:25.645386 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-026565be-7889-46d7-a6da-d2be1e1de649 None admin] Authenticating user token {{(pid=80245) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:16:25.656583 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-719ffbb7-38b0-4809-961a-52dfee8b0262 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:16:25.657464 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-719ffbb7-38b0-4809-961a-52dfee8b0262 None admin] REQUEST_METHOD: `GET` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:16:25.657756 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-719ffbb7-38b0-4809-961a-52dfee8b0262 None admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:16:25.658511 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-719ffbb7-38b0-4809-961a-52dfee8b0262 None admin] PATH_INFO: `/v3/projects/d4fa4cba54f348b68df92b37fb5fd5d1` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:16:25.660276 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-719ffbb7-38b0-4809-961a-52dfee8b0262 None admin] RBAC: Authorizing `identity:get_project(project_id=d4fa4cba54f348b68df92b37fb5fd5d1)` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:16:25.662034 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-719ffbb7-38b0-4809-961a-52dfee8b0262 None admin] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:16:25.665147 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 101/202] 10.222.0.47 () {56 vars in 1186 bytes} [Thu Jul 21 08:16:25 2022] GET /identity/v3/projects/d4fa4cba54f348b68df92b37fb5fd5d1 => generated 302 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 08:16:25.665972 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-719ffbb7-38b0-4809-961a-52dfee8b0262 None admin] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:16:25.751685 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG dbcounter [-] [80244] Writing DB stats keystone:SELECT=100 {{(pid=80244) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:16:25.756919 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-026565be-7889-46d7-a6da-d2be1e1de649 None admin] Validating token access rules against request {{(pid=80245) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:16:25.793494 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05d66e1d-1217-42ae-82b6-8dd41928e1cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80245) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:16:25.794527 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05d66e1d-1217-42ae-82b6-8dd41928e1cb None admin] REQUEST_METHOD: `GET` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:16:25.794939 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05d66e1d-1217-42ae-82b6-8dd41928e1cb None admin] SCRIPT_NAME: `/identity` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:16:25.795318 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05d66e1d-1217-42ae-82b6-8dd41928e1cb None admin] PATH_INFO: `/v3/projects/eab7e5ec54c94574ad68c89747ad5bcc` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:16:25.796688 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05d66e1d-1217-42ae-82b6-8dd41928e1cb None admin] RBAC: Authorizing `identity:get_project(project_id=eab7e5ec54c94574ad68c89747ad5bcc)` {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:16:25.798250 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05d66e1d-1217-42ae-82b6-8dd41928e1cb None admin] RBAC: Authorization granted {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:16:25.800356 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: [pid: 80245|app: 0|req: 102/203] 10.222.0.47 () {56 vars in 1186 bytes} [Thu Jul 21 08:16:25 2022] GET /identity/v3/projects/eab7e5ec54c94574ad68c89747ad5bcc => generated 298 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 08:16:25.801119 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05d66e1d-1217-42ae-82b6-8dd41928e1cb None admin] Authenticating user token {{(pid=80245) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:16:25.801794 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-719ffbb7-38b0-4809-961a-52dfee8b0262 None admin] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:16:25.807630 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG dbcounter [-] [80245] Writing DB stats keystone:SELECT=100 {{(pid=80245) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:16:25.859721 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e9707f0-17ad-4b7c-8cfa-9e6b0fab1652 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:16:25.860532 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e9707f0-17ad-4b7c-8cfa-9e6b0fab1652 None admin] REQUEST_METHOD: `GET` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:16:25.861077 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e9707f0-17ad-4b7c-8cfa-9e6b0fab1652 None admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:16:25.861368 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e9707f0-17ad-4b7c-8cfa-9e6b0fab1652 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:16:25.936005 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05d66e1d-1217-42ae-82b6-8dd41928e1cb None admin] Validating token access rules against request {{(pid=80245) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:16:25.955496 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e9707f0-17ad-4b7c-8cfa-9e6b0fab1652 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:16:25.957491 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e9707f0-17ad-4b7c-8cfa-9e6b0fab1652 None admin] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:16:25.982057 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a99ef2fb-51fe-4dfa-8f8f-b56202041ce5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80245) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:16:25.982729 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a99ef2fb-51fe-4dfa-8f8f-b56202041ce5 None admin] REQUEST_METHOD: `GET` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:16:25.983012 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a99ef2fb-51fe-4dfa-8f8f-b56202041ce5 None admin] SCRIPT_NAME: `/identity` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:16:25.983289 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a99ef2fb-51fe-4dfa-8f8f-b56202041ce5 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:16:25.984219 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a99ef2fb-51fe-4dfa-8f8f-b56202041ce5 None admin] RBAC: Adding query filter params (role.id=e29870f6e9e2411a9db0d5bf45f32ca5, scope.project.id=d4fa4cba54f348b68df92b37fb5fd5d1, user.id=70ffa790321f4f19b7935a59cb2fcfd9) {{(pid=80245) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 08:16:25.985064 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a99ef2fb-51fe-4dfa-8f8f-b56202041ce5 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:16:25.986900 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a99ef2fb-51fe-4dfa-8f8f-b56202041ce5 None admin] RBAC: Authorization granted {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:16:25.995846 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: [pid: 80245|app: 0|req: 103/204] 10.222.0.47 () {56 vars in 1374 bytes} [Thu Jul 21 08:16:25 2022] GET /identity/v3/role_assignments?user.id=70ffa790321f4f19b7935a59cb2fcfd9&scope.project.id=d4fa4cba54f348b68df92b37fb5fd5d1&role.id=e29870f6e9e2411a9db0d5bf45f32ca5 => generated 259 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 08:16:25.997001 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a99ef2fb-51fe-4dfa-8f8f-b56202041ce5 None admin] Authenticating user token {{(pid=80245) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:16:26.032342 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG dbcounter [-] [80244] Writing DB stats keystone:SELECT=100 {{(pid=80244) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:16:26.079973 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 102/205] 10.222.0.47 () {58 vars in 1282 bytes} [Thu Jul 21 08:16:25 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 414 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 21 08:16:26.092747 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG dbcounter [-] [80245] Writing DB stats keystone:SELECT=100 {{(pid=80245) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:16:26.093824 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e9707f0-17ad-4b7c-8cfa-9e6b0fab1652 None admin] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:16:26.143349 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a99ef2fb-51fe-4dfa-8f8f-b56202041ce5 None admin] Validating token access rules against request {{(pid=80245) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:16:26.181884 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b50f751a-90ac-457c-aa25-1b8b702f8582 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80245) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:16:26.182626 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b50f751a-90ac-457c-aa25-1b8b702f8582 None admin] REQUEST_METHOD: `GET` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:16:26.182918 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b50f751a-90ac-457c-aa25-1b8b702f8582 None admin] SCRIPT_NAME: `/identity` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:16:26.183201 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b50f751a-90ac-457c-aa25-1b8b702f8582 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:16:26.184095 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b50f751a-90ac-457c-aa25-1b8b702f8582 None admin] RBAC: Adding query filter params (role.id=9e5171884e9f401ab2337b74db49546e, scope.project.id=eab7e5ec54c94574ad68c89747ad5bcc, user.id=62835f20a4f74cb8a96c4979d095b168) {{(pid=80245) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 08:16:26.184591 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b50f751a-90ac-457c-aa25-1b8b702f8582 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:16:26.186153 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b50f751a-90ac-457c-aa25-1b8b702f8582 None admin] RBAC: Authorization granted {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:16:26.192999 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: [pid: 80245|app: 0|req: 104/206] 10.222.0.47 () {56 vars in 1374 bytes} [Thu Jul 21 08:16:25 2022] GET /identity/v3/role_assignments?user.id=62835f20a4f74cb8a96c4979d095b168&scope.project.id=eab7e5ec54c94574ad68c89747ad5bcc&role.id=9e5171884e9f401ab2337b74db49546e => generated 608 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 08:16:26.262867 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e9707f0-17ad-4b7c-8cfa-9e6b0fab1652 None admin] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:16:26.315181 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80e7b079-7d46-4345-a1ed-0c0f7e5124a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:16:26.316096 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80e7b079-7d46-4345-a1ed-0c0f7e5124a3 None admin] REQUEST_METHOD: `GET` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:16:26.316385 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80e7b079-7d46-4345-a1ed-0c0f7e5124a3 None admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:16:26.316662 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80e7b079-7d46-4345-a1ed-0c0f7e5124a3 None admin] PATH_INFO: `/v3/projects/d4fa4cba54f348b68df92b37fb5fd5d1` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:16:26.319949 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80e7b079-7d46-4345-a1ed-0c0f7e5124a3 None admin] RBAC: Authorizing `identity:get_project(project_id=d4fa4cba54f348b68df92b37fb5fd5d1)` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:16:26.321502 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80e7b079-7d46-4345-a1ed-0c0f7e5124a3 None admin] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:16:26.323692 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 103/207] 10.222.0.47 () {56 vars in 1186 bytes} [Thu Jul 21 08:16:26 2022] GET /identity/v3/projects/d4fa4cba54f348b68df92b37fb5fd5d1 => generated 302 bytes in 232 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 08:16:26.329827 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b50f751a-90ac-457c-aa25-1b8b702f8582 None admin] Authenticating user token {{(pid=80245) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:16:26.454846 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b50f751a-90ac-457c-aa25-1b8b702f8582 None admin] Validating token access rules against request {{(pid=80245) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:16:26.488455 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG dbcounter [-] [80245] Writing DB stats keystone:SELECT=100 {{(pid=80245) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:16:26.492889 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-713658d9-75d4-49e2-9f6a-260077ca9f02 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80245) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:16:26.493763 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-713658d9-75d4-49e2-9f6a-260077ca9f02 None admin] REQUEST_METHOD: `GET` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:16:26.494229 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-713658d9-75d4-49e2-9f6a-260077ca9f02 None admin] SCRIPT_NAME: `/identity` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:16:26.494715 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-713658d9-75d4-49e2-9f6a-260077ca9f02 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:16:26.495594 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-713658d9-75d4-49e2-9f6a-260077ca9f02 None admin] RBAC: Adding query filter params (role.id=99a638c0aac44b79a1f2858574724495, scope.project.id=d4fa4cba54f348b68df92b37fb5fd5d1, user.id=70ffa790321f4f19b7935a59cb2fcfd9) {{(pid=80245) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 08:16:26.495908 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-713658d9-75d4-49e2-9f6a-260077ca9f02 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:16:26.497407 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-713658d9-75d4-49e2-9f6a-260077ca9f02 None admin] RBAC: Authorization granted {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:16:26.503716 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: [pid: 80245|app: 0|req: 105/208] 10.222.0.47 () {56 vars in 1374 bytes} [Thu Jul 21 08:16:26 2022] GET /identity/v3/role_assignments?user.id=70ffa790321f4f19b7935a59cb2fcfd9&scope.project.id=d4fa4cba54f348b68df92b37fb5fd5d1&role.id=99a638c0aac44b79a1f2858574724495 => generated 259 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 08:16:27.344173 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af029b88-b182-4e59-a331-99712c8ed4d9 None None] REQUEST_METHOD: `GET` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:16:27.345223 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af029b88-b182-4e59-a331-99712c8ed4d9 None None] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:16:27.345738 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af029b88-b182-4e59-a331-99712c8ed4d9 None None] PATH_INFO: `/` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:16:27.347353 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 104/209] 10.222.0.47 () {54 vars in 926 bytes} [Thu Jul 21 08:16:27 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 08:16:27.353021 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efdd83f8-6309-4084-a4b8-59b256f09a9c None None] REQUEST_METHOD: `POST` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:16:27.353845 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efdd83f8-6309-4084-a4b8-59b256f09a9c None None] SCRIPT_NAME: `/identity` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:16:27.354327 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efdd83f8-6309-4084-a4b8-59b256f09a9c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:16:27.371933 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.auth.core [None req-efdd83f8-6309-4084-a4b8-59b256f09a9c None None] MFA Rules not processed for user `2dce40b8d74c41ec94cc36e6ff05d412`. Rule list: `[]` (Enabled: `True`). {{(pid=80245) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 08:16:27.373445 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.fernet_utils [None req-efdd83f8-6309-4084-a4b8-59b256f09a9c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80245) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 08:16:27.441284 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: [pid: 80245|app: 0|req: 106/210] 10.222.0.47 () {58 vars in 1023 bytes} [Thu Jul 21 08:16:27 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 21 08:16:27.466226 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af029b88-b182-4e59-a331-99712c8ed4d9 None None] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:16:27.471514 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1513715-8c1a-41bc-b455-654715146fff None None] REQUEST_METHOD: `GET` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:16:27.472057 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1513715-8c1a-41bc-b455-654715146fff None None] SCRIPT_NAME: `/identity` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:16:27.472559 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1513715-8c1a-41bc-b455-654715146fff None None] PATH_INFO: `/` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:16:27.474152 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: [pid: 80245|app: 0|req: 107/211] 10.222.0.47 () {54 vars in 926 bytes} [Thu Jul 21 08:16:27 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 08:16:27.485059 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66cb58a4-a7be-4c0b-8c1b-6713647e8c3c None None] REQUEST_METHOD: `POST` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:16:27.485855 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66cb58a4-a7be-4c0b-8c1b-6713647e8c3c None None] SCRIPT_NAME: `/identity` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:16:27.485963 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66cb58a4-a7be-4c0b-8c1b-6713647e8c3c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:16:27.503113 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.auth.core [None req-66cb58a4-a7be-4c0b-8c1b-6713647e8c3c None None] MFA Rules not processed for user `2dce40b8d74c41ec94cc36e6ff05d412`. Rule list: `[]` (Enabled: `True`). {{(pid=80245) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 08:16:27.504335 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.fernet_utils [None req-66cb58a4-a7be-4c0b-8c1b-6713647e8c3c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80245) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 08:16:27.508830 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG dbcounter [-] [80244] Writing DB stats keystone:SELECT=100 {{(pid=80244) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:16:27.588152 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: [pid: 80245|app: 0|req: 108/212] 10.222.0.47 () {58 vars in 1023 bytes} [Thu Jul 21 08:16:27 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 21 08:16:27.596124 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af029b88-b182-4e59-a331-99712c8ed4d9 None None] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:16:27.613406 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66cb58a4-a7be-4c0b-8c1b-6713647e8c3c None None] Authenticating user token {{(pid=80245) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:16:27.641311 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e4972f8-fc75-44d3-947f-613dbc424c6d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:16:27.642479 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e4972f8-fc75-44d3-947f-613dbc424c6d None admin] REQUEST_METHOD: `GET` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:16:27.642966 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e4972f8-fc75-44d3-947f-613dbc424c6d None admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:16:27.643498 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e4972f8-fc75-44d3-947f-613dbc424c6d None admin] PATH_INFO: `/v3/roles/member` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:16:27.656371 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e4972f8-fc75-44d3-947f-613dbc424c6d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 21 08:16:27.664541 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application [None req-0e4972f8-fc75-44d3-947f-613dbc424c6d None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 21 08:16:27.664541 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 08:16:27.664541 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 08:16:27.664541 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 08:16:27.664541 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 08:16:27.664541 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 08:16:27.664541 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 08:16:27.664541 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 08:16:27.664541 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 08:16:27.664541 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 08:16:27.664541 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 08:16:27.664541 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 08:16:27.664541 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 21 08:16:27.664541 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 21 08:16:27.664541 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 21 08:16:27.664541 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 21 08:16:27.665411 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 21 08:16:27.665411 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 21 08:16:27.665411 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 21 08:16:27.665411 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 21 08:16:27.665411 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 08:16:27.665411 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 08:16:27.665411 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 08:16:27.665411 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 08:16:27.665411 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 08:16:27.665411 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 08:16:27.665411 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 08:16:27.665411 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application with Lock( Jul 21 08:16:27.665411 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 08:16:27.665411 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self._enter() Jul 21 08:16:27.665411 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 08:16:27.665411 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 08:16:27.666166 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 08:16:27.666166 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self.creator() Jul 21 08:16:27.666166 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 08:16:27.666166 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application created_value = creator( Jul 21 08:16:27.666166 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 21 08:16:27.666166 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 21 08:16:27.666166 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 21 08:16:27.666166 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 21 08:16:27.666166 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 21 08:16:27.666166 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 21 08:16:27.666166 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 21 08:16:27.666166 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application Jul 21 08:16:27.673536 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 105/213] 10.222.0.47 () {56 vars in 1099 bytes} [Thu Jul 21 08:16:27 2022] GET /identity/v3/roles/member => generated 84 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 08:16:27.675065 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e4972f8-fc75-44d3-947f-613dbc424c6d None admin] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:16:27.683729 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG dbcounter [-] [80245] Writing DB stats keystone:SELECT=100 {{(pid=80245) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:16:27.739708 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66cb58a4-a7be-4c0b-8c1b-6713647e8c3c None None] Validating token access rules against request {{(pid=80245) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:16:27.783607 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5da4e843-caf4-4c7a-aba3-29e10850b178 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80245) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:16:27.784415 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5da4e843-caf4-4c7a-aba3-29e10850b178 None admin] REQUEST_METHOD: `GET` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:16:27.784693 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5da4e843-caf4-4c7a-aba3-29e10850b178 None admin] SCRIPT_NAME: `/identity` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:16:27.784970 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5da4e843-caf4-4c7a-aba3-29e10850b178 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:16:27.786642 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5da4e843-caf4-4c7a-aba3-29e10850b178 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:16:27.788668 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5da4e843-caf4-4c7a-aba3-29e10850b178 None admin] RBAC: Authorization granted {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:16:27.791169 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: [pid: 80245|app: 0|req: 109/214] 10.222.0.47 () {56 vars in 1108 bytes} [Thu Jul 21 08:16:27 2022] GET /identity/v3/domains/default => generated 199 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 21 08:16:27.796374 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5da4e843-caf4-4c7a-aba3-29e10850b178 None admin] Authenticating user token {{(pid=80245) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:16:27.811223 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG dbcounter [-] [80244] Writing DB stats keystone:SELECT=100 {{(pid=80244) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:16:27.825606 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e4972f8-fc75-44d3-947f-613dbc424c6d None admin] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:16:27.883732 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d8e0f47-b340-48c8-a740-f49ccee37f41 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:16:27.884237 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d8e0f47-b340-48c8-a740-f49ccee37f41 None admin] REQUEST_METHOD: `GET` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:16:27.884544 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d8e0f47-b340-48c8-a740-f49ccee37f41 None admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:16:27.884815 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d8e0f47-b340-48c8-a740-f49ccee37f41 None admin] PATH_INFO: `/v3/roles` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:16:27.885764 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d8e0f47-b340-48c8-a740-f49ccee37f41 None admin] RBAC: Adding query filter params (name=member) {{(pid=80244) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 08:16:27.886284 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d8e0f47-b340-48c8-a740-f49ccee37f41 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:16:27.888207 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d8e0f47-b340-48c8-a740-f49ccee37f41 None admin] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:16:27.902347 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 106/215] 10.222.0.47 () {56 vars in 1101 bytes} [Thu Jul 21 08:16:27 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 223 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 08:16:27.904628 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d8e0f47-b340-48c8-a740-f49ccee37f41 None admin] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:16:27.958715 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5da4e843-caf4-4c7a-aba3-29e10850b178 None admin] Validating token access rules against request {{(pid=80245) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:16:27.993038 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG dbcounter [-] [80245] Writing DB stats keystone:SELECT=100 {{(pid=80245) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:16:28.011217 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e21b5b5-bd06-4ea9-9e2d-6fe78fe67c1c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80245) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:16:28.012503 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e21b5b5-bd06-4ea9-9e2d-6fe78fe67c1c None admin] REQUEST_METHOD: `POST` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:16:28.013393 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e21b5b5-bd06-4ea9-9e2d-6fe78fe67c1c None admin] SCRIPT_NAME: `/identity` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:16:28.013931 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e21b5b5-bd06-4ea9-9e2d-6fe78fe67c1c None admin] PATH_INFO: `/v3/users` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:16:28.014854 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e21b5b5-bd06-4ea9-9e2d-6fe78fe67c1c None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:16:28.017425 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e21b5b5-bd06-4ea9-9e2d-6fe78fe67c1c None admin] RBAC: Authorization granted {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:16:28.028348 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d8e0f47-b340-48c8-a740-f49ccee37f41 None admin] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:16:28.046325 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: [pid: 80245|app: 0|req: 110/216] 10.222.0.47 () {60 vars in 1132 bytes} [Thu Jul 21 08:16:27 2022] POST /identity/v3/users => generated 297 bytes in 250 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 08:16:28.089645 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-127b7959-d487-41f3-84af-496880ced26b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:16:28.090923 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-127b7959-d487-41f3-84af-496880ced26b None admin] REQUEST_METHOD: `GET` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:16:28.091458 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-127b7959-d487-41f3-84af-496880ced26b None admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:16:28.091940 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-127b7959-d487-41f3-84af-496880ced26b None admin] PATH_INFO: `/v3/users/5a245876268b4f3797a25e891079b053` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:16:28.109493 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-127b7959-d487-41f3-84af-496880ced26b None admin] RBAC: Authorizing `identity:get_user(user_id=5a245876268b4f3797a25e891079b053)` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:16:28.111626 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-127b7959-d487-41f3-84af-496880ced26b None admin] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:16:28.118009 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 107/217] 10.222.0.47 () {56 vars in 1177 bytes} [Thu Jul 21 08:16:27 2022] GET /identity/v3/users/5a245876268b4f3797a25e891079b053 => generated 297 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 21 08:16:28.120662 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e21b5b5-bd06-4ea9-9e2d-6fe78fe67c1c None admin] Authenticating user token {{(pid=80245) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:16:28.217056 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d15da9d5-003d-44ea-bb01-2d7595de554a None None] REQUEST_METHOD: `GET` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:16:28.217695 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d15da9d5-003d-44ea-bb01-2d7595de554a None None] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:16:28.218195 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d15da9d5-003d-44ea-bb01-2d7595de554a None None] PATH_INFO: `/` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:16:28.219818 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 108/218] 10.222.0.47 () {54 vars in 926 bytes} [Thu Jul 21 08:16:28 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 21 08:16:28.225734 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d457b924-7da3-4860-98a5-087ef98d6324 None None] REQUEST_METHOD: `POST` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:16:28.226021 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d457b924-7da3-4860-98a5-087ef98d6324 None None] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:16:28.226298 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d457b924-7da3-4860-98a5-087ef98d6324 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:16:28.248453 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.auth.core [None req-d457b924-7da3-4860-98a5-087ef98d6324 None None] MFA Rules not processed for user `2dce40b8d74c41ec94cc36e6ff05d412`. Rule list: `[]` (Enabled: `True`). {{(pid=80244) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 08:16:28.249003 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e21b5b5-bd06-4ea9-9e2d-6fe78fe67c1c None admin] Validating token access rules against request {{(pid=80245) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:16:28.249630 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.fernet_utils [None req-d457b924-7da3-4860-98a5-087ef98d6324 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80244) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 08:16:28.253734 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG dbcounter [-] [80244] Writing DB stats keystone:SELECT=100 {{(pid=80244) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:16:28.292812 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8acdcd24-9df2-429d-9296-f909e795750d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80245) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:16:28.293480 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8acdcd24-9df2-429d-9296-f909e795750d None admin] REQUEST_METHOD: `GET` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:16:28.293758 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8acdcd24-9df2-429d-9296-f909e795750d None admin] SCRIPT_NAME: `/identity` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:16:28.294029 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8acdcd24-9df2-429d-9296-f909e795750d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:16:28.342111 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 109/219] 10.222.0.47 () {58 vars in 1023 bytes} [Thu Jul 21 08:16:28 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 21 08:16:28.357042 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG dbcounter [-] [80245] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80245) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:16:28.366222 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d457b924-7da3-4860-98a5-087ef98d6324 None None] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:16:28.395534 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8acdcd24-9df2-429d-9296-f909e795750d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:16:28.397804 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8acdcd24-9df2-429d-9296-f909e795750d None admin] RBAC: Authorization granted {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:16:28.528129 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d457b924-7da3-4860-98a5-087ef98d6324 None None] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:16:28.529376 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: [pid: 80245|app: 0|req: 111/220] 10.222.0.47 () {58 vars in 1282 bytes} [Thu Jul 21 08:16:28 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 409 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 21 08:16:28.536776 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8acdcd24-9df2-429d-9296-f909e795750d None admin] Authenticating user token {{(pid=80245) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:16:28.582489 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90f5d266-c25f-4e06-8107-537a84137b6c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:16:28.583304 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90f5d266-c25f-4e06-8107-537a84137b6c None admin] REQUEST_METHOD: `GET` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:16:28.583639 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90f5d266-c25f-4e06-8107-537a84137b6c None admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:16:28.583921 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90f5d266-c25f-4e06-8107-537a84137b6c None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:16:28.595421 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG dbcounter [-] [80244] Writing DB stats keystone:SELECT=100 {{(pid=80244) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:16:28.601030 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90f5d266-c25f-4e06-8107-537a84137b6c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 21 08:16:28.611832 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application [None req-90f5d266-c25f-4e06-8107-537a84137b6c None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 21 08:16:28.611832 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 08:16:28.611832 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 08:16:28.611832 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 08:16:28.611832 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 08:16:28.611832 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 08:16:28.611832 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 08:16:28.611832 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 08:16:28.611832 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 08:16:28.611832 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 08:16:28.611832 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 08:16:28.611832 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 08:16:28.611832 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 21 08:16:28.611832 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 21 08:16:28.611832 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 21 08:16:28.611832 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 21 08:16:28.612691 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 21 08:16:28.612691 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 21 08:16:28.612691 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 21 08:16:28.612691 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 21 08:16:28.612691 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 08:16:28.612691 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 08:16:28.612691 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 08:16:28.612691 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 08:16:28.612691 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 08:16:28.612691 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 08:16:28.612691 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 08:16:28.612691 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application with Lock( Jul 21 08:16:28.612691 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 08:16:28.612691 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self._enter() Jul 21 08:16:28.612691 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 08:16:28.612691 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 08:16:28.613442 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 08:16:28.613442 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self.creator() Jul 21 08:16:28.613442 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 08:16:28.613442 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application created_value = creator( Jul 21 08:16:28.613442 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 21 08:16:28.613442 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 21 08:16:28.613442 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 21 08:16:28.613442 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 21 08:16:28.613442 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 21 08:16:28.613442 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 21 08:16:28.613442 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 21 08:16:28.613442 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application Jul 21 08:16:28.618100 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 110/221] 10.222.0.47 () {56 vars in 1096 bytes} [Thu Jul 21 08:16:28 2022] GET /identity/v3/roles/admin => generated 83 bytes in 248 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 08:16:28.619475 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90f5d266-c25f-4e06-8107-537a84137b6c None admin] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:16:28.630552 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG dbcounter [-] [80245] Writing DB stats keystone:SELECT=100 {{(pid=80245) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:16:28.656974 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8acdcd24-9df2-429d-9296-f909e795750d None admin] Validating token access rules against request {{(pid=80245) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:16:28.700434 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca801f63-e269-44de-b5a8-c3a3d38b67f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80245) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:16:28.701331 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca801f63-e269-44de-b5a8-c3a3d38b67f4 None admin] REQUEST_METHOD: `GET` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:16:28.701620 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca801f63-e269-44de-b5a8-c3a3d38b67f4 None admin] SCRIPT_NAME: `/identity` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:16:28.701901 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca801f63-e269-44de-b5a8-c3a3d38b67f4 None admin] PATH_INFO: `/v3/projects/d4fa4cba54f348b68df92b37fb5fd5d1` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:16:28.703153 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca801f63-e269-44de-b5a8-c3a3d38b67f4 None admin] RBAC: Authorizing `identity:get_project(project_id=d4fa4cba54f348b68df92b37fb5fd5d1)` {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:16:28.705556 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca801f63-e269-44de-b5a8-c3a3d38b67f4 None admin] RBAC: Authorization granted {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:16:28.707648 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: [pid: 80245|app: 0|req: 112/222] 10.222.0.47 () {56 vars in 1186 bytes} [Thu Jul 21 08:16:28 2022] GET /identity/v3/projects/d4fa4cba54f348b68df92b37fb5fd5d1 => generated 302 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 21 08:16:28.714139 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca801f63-e269-44de-b5a8-c3a3d38b67f4 None admin] Authenticating user token {{(pid=80245) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:16:28.750416 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90f5d266-c25f-4e06-8107-537a84137b6c None admin] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:16:28.788554 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dd1fe93-27e6-46a0-9bab-3660ad326eb1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:16:28.789596 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dd1fe93-27e6-46a0-9bab-3660ad326eb1 None admin] REQUEST_METHOD: `GET` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:16:28.790088 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dd1fe93-27e6-46a0-9bab-3660ad326eb1 None admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:16:28.790552 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dd1fe93-27e6-46a0-9bab-3660ad326eb1 None admin] PATH_INFO: `/v3/roles` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:16:28.791583 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dd1fe93-27e6-46a0-9bab-3660ad326eb1 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80244) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 08:16:28.792132 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dd1fe93-27e6-46a0-9bab-3660ad326eb1 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:16:28.793880 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dd1fe93-27e6-46a0-9bab-3660ad326eb1 None admin] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:16:28.803794 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 111/223] 10.222.0.47 () {56 vars in 1099 bytes} [Thu Jul 21 08:16:28 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 08:16:28.805410 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83eb8668-4f76-4949-aae7-904f56b18dca None None] REQUEST_METHOD: `GET` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:16:28.805913 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83eb8668-4f76-4949-aae7-904f56b18dca None None] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:16:28.806428 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83eb8668-4f76-4949-aae7-904f56b18dca None None] PATH_INFO: `/` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:16:28.808226 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 112/224] 10.222.0.47 () {54 vars in 926 bytes} [Thu Jul 21 08:16:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 08:16:28.813149 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dc3d91d-cd77-4bdf-bc89-7c6b449af905 None None] REQUEST_METHOD: `POST` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:16:28.813677 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dc3d91d-cd77-4bdf-bc89-7c6b449af905 None None] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:16:28.814163 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dc3d91d-cd77-4bdf-bc89-7c6b449af905 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:16:28.825206 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca801f63-e269-44de-b5a8-c3a3d38b67f4 None admin] Validating token access rules against request {{(pid=80245) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:16:28.833254 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.auth.core [None req-8dc3d91d-cd77-4bdf-bc89-7c6b449af905 None None] MFA Rules not processed for user `2dce40b8d74c41ec94cc36e6ff05d412`. Rule list: `[]` (Enabled: `True`). {{(pid=80244) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 08:16:28.835271 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.fernet_utils [None req-8dc3d91d-cd77-4bdf-bc89-7c6b449af905 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80244) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 08:16:28.862607 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51650060-cf49-4ada-9584-4fad0b70f77a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80245) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:16:28.863298 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51650060-cf49-4ada-9584-4fad0b70f77a None admin] REQUEST_METHOD: `GET` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:16:28.863632 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51650060-cf49-4ada-9584-4fad0b70f77a None admin] SCRIPT_NAME: `/identity` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:16:28.863912 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51650060-cf49-4ada-9584-4fad0b70f77a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:16:28.864822 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51650060-cf49-4ada-9584-4fad0b70f77a None admin] RBAC: Adding query filter params (role.id=3a074738221e4601b5205715ba87d240, scope.project.id=d4fa4cba54f348b68df92b37fb5fd5d1, user.id=5a245876268b4f3797a25e891079b053) {{(pid=80245) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 08:16:28.865195 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51650060-cf49-4ada-9584-4fad0b70f77a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:16:28.867192 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51650060-cf49-4ada-9584-4fad0b70f77a None admin] RBAC: Authorization granted {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:16:28.883488 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: [pid: 80245|app: 0|req: 113/225] 10.222.0.47 () {56 vars in 1374 bytes} [Thu Jul 21 08:16:28 2022] GET /identity/v3/role_assignments?user.id=5a245876268b4f3797a25e891079b053&scope.project.id=d4fa4cba54f348b68df92b37fb5fd5d1&role.id=3a074738221e4601b5205715ba87d240 => generated 259 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 08:16:28.884525 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51650060-cf49-4ada-9584-4fad0b70f77a None admin] Authenticating user token {{(pid=80245) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:16:28.905799 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG dbcounter [-] [80245] Writing DB stats keystone:SELECT=100 {{(pid=80245) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:16:28.909486 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG dbcounter [-] [80244] Writing DB stats keystone:SELECT=100 {{(pid=80244) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:16:28.923079 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 113/226] 10.222.0.47 () {58 vars in 1023 bytes} [Thu Jul 21 08:16:28 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 21 08:16:28.957799 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dc3d91d-cd77-4bdf-bc89-7c6b449af905 None None] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:16:29.076527 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dc3d91d-cd77-4bdf-bc89-7c6b449af905 None None] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:16:29.092384 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51650060-cf49-4ada-9584-4fad0b70f77a None admin] Validating token access rules against request {{(pid=80245) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:16:29.114443 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f025c971-45ad-4f50-b71d-7d81dceb50b9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:16:29.116162 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f025c971-45ad-4f50-b71d-7d81dceb50b9 None admin] REQUEST_METHOD: `GET` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:16:29.116668 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f025c971-45ad-4f50-b71d-7d81dceb50b9 None admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:16:29.117137 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f025c971-45ad-4f50-b71d-7d81dceb50b9 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:16:29.130109 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f025c971-45ad-4f50-b71d-7d81dceb50b9 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 21 08:16:29.138941 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7db7cccc-9179-48a3-878a-eb6beedab19a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80245) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:16:29.139143 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7db7cccc-9179-48a3-878a-eb6beedab19a None admin] REQUEST_METHOD: `GET` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:16:29.139465 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7db7cccc-9179-48a3-878a-eb6beedab19a None admin] SCRIPT_NAME: `/identity` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:16:29.139741 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7db7cccc-9179-48a3-878a-eb6beedab19a None admin] PATH_INFO: `/v3/users/70ffa790321f4f19b7935a59cb2fcfd9` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:16:29.141930 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application [None req-f025c971-45ad-4f50-b71d-7d81dceb50b9 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 21 08:16:29.141930 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 08:16:29.141930 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 08:16:29.141930 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 08:16:29.141930 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 08:16:29.141930 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 08:16:29.141930 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 08:16:29.141930 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 08:16:29.141930 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 08:16:29.141930 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 08:16:29.141930 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 08:16:29.141930 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 08:16:29.141930 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 21 08:16:29.141930 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 21 08:16:29.141930 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 21 08:16:29.141930 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 21 08:16:29.142763 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 21 08:16:29.142763 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 21 08:16:29.142763 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 21 08:16:29.142763 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 21 08:16:29.142763 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 08:16:29.142763 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 08:16:29.142763 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 08:16:29.142763 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 08:16:29.142763 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 08:16:29.142763 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 08:16:29.142763 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 08:16:29.142763 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application with Lock( Jul 21 08:16:29.142763 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 08:16:29.142763 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self._enter() Jul 21 08:16:29.142763 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 08:16:29.142763 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 08:16:29.143545 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 08:16:29.143545 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self.creator() Jul 21 08:16:29.143545 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 08:16:29.143545 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application created_value = creator( Jul 21 08:16:29.143545 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 21 08:16:29.143545 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 21 08:16:29.143545 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 21 08:16:29.143545 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 21 08:16:29.143545 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 21 08:16:29.143545 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 21 08:16:29.143545 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 21 08:16:29.143545 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application Jul 21 08:16:29.144123 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7db7cccc-9179-48a3-878a-eb6beedab19a None admin] RBAC: Authorizing `identity:get_user(user_id=70ffa790321f4f19b7935a59cb2fcfd9)` {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:16:29.144123 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7db7cccc-9179-48a3-878a-eb6beedab19a None admin] RBAC: Authorization granted {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:16:29.145612 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: [pid: 80245|app: 0|req: 114/227] 10.222.0.47 () {56 vars in 1177 bytes} [Thu Jul 21 08:16:28 2022] GET /identity/v3/users/70ffa790321f4f19b7935a59cb2fcfd9 => generated 283 bytes in 261 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 08:16:29.147136 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 114/228] 10.222.0.47 () {56 vars in 1114 bytes} [Thu Jul 21 08:16:28 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 08:16:29.151154 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7db7cccc-9179-48a3-878a-eb6beedab19a None admin] Authenticating user token {{(pid=80245) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:16:29.152466 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f025c971-45ad-4f50-b71d-7d81dceb50b9 None admin] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:16:29.212676 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG dbcounter [-] [80244] Writing DB stats keystone:SELECT=100 {{(pid=80244) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:16:29.264178 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG dbcounter [-] [80245] Writing DB stats keystone:SELECT=100 {{(pid=80245) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:16:29.269798 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f025c971-45ad-4f50-b71d-7d81dceb50b9 None admin] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:16:29.283377 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7db7cccc-9179-48a3-878a-eb6beedab19a None admin] Validating token access rules against request {{(pid=80245) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:16:29.322671 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07cbe1c2-2c60-41dd-96d1-f72fb2921640 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80245) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:16:29.323678 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07cbe1c2-2c60-41dd-96d1-f72fb2921640 None admin] REQUEST_METHOD: `GET` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:16:29.323967 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07cbe1c2-2c60-41dd-96d1-f72fb2921640 None admin] SCRIPT_NAME: `/identity` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:16:29.324243 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07cbe1c2-2c60-41dd-96d1-f72fb2921640 None admin] PATH_INFO: `/v3/projects/d4fa4cba54f348b68df92b37fb5fd5d1` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:16:29.325538 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07cbe1c2-2c60-41dd-96d1-f72fb2921640 None admin] RBAC: Authorizing `identity:get_project(project_id=d4fa4cba54f348b68df92b37fb5fd5d1)` {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:16:29.334315 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07cbe1c2-2c60-41dd-96d1-f72fb2921640 None admin] RBAC: Authorization granted {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:16:29.334315 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: [pid: 80245|app: 0|req: 115/229] 10.222.0.47 () {56 vars in 1186 bytes} [Thu Jul 21 08:16:29 2022] GET /identity/v3/projects/d4fa4cba54f348b68df92b37fb5fd5d1 => generated 302 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 08:16:29.336965 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07cbe1c2-2c60-41dd-96d1-f72fb2921640 None admin] Authenticating user token {{(pid=80245) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:16:29.349480 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83ed2170-fc2a-409a-a8ce-baddd21bb1a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:16:29.351349 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83ed2170-fc2a-409a-a8ce-baddd21bb1a0 None admin] REQUEST_METHOD: `GET` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:16:29.351802 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83ed2170-fc2a-409a-a8ce-baddd21bb1a0 None admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:16:29.352235 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83ed2170-fc2a-409a-a8ce-baddd21bb1a0 None admin] PATH_INFO: `/v3/roles` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:16:29.353136 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83ed2170-fc2a-409a-a8ce-baddd21bb1a0 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80244) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 08:16:29.353583 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83ed2170-fc2a-409a-a8ce-baddd21bb1a0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:16:29.355111 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83ed2170-fc2a-409a-a8ce-baddd21bb1a0 None admin] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:16:29.370915 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 115/230] 10.222.0.47 () {56 vars in 1111 bytes} [Thu Jul 21 08:16:29 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 219 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 21 08:16:29.376450 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83ed2170-fc2a-409a-a8ce-baddd21bb1a0 None admin] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:16:29.458803 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07cbe1c2-2c60-41dd-96d1-f72fb2921640 None admin] Validating token access rules against request {{(pid=80245) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:16:29.516216 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bca00db3-3ae1-4ec4-ba1e-d8cae145388f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80245) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:16:29.517258 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bca00db3-3ae1-4ec4-ba1e-d8cae145388f None admin] REQUEST_METHOD: `PUT` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:16:29.517546 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bca00db3-3ae1-4ec4-ba1e-d8cae145388f None admin] SCRIPT_NAME: `/identity` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:16:29.517824 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bca00db3-3ae1-4ec4-ba1e-d8cae145388f None admin] PATH_INFO: `/v3/projects/d4fa4cba54f348b68df92b37fb5fd5d1/users/70ffa790321f4f19b7935a59cb2fcfd9/roles/e29870f6e9e2411a9db0d5bf45f32ca5` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:16:29.522268 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bca00db3-3ae1-4ec4-ba1e-d8cae145388f None admin] RBAC: Authorizing `identity:create_grant(project_id=d4fa4cba54f348b68df92b37fb5fd5d1, user_id=70ffa790321f4f19b7935a59cb2fcfd9, role_id=e29870f6e9e2411a9db0d5bf45f32ca5)` {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:16:29.523893 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bca00db3-3ae1-4ec4-ba1e-d8cae145388f None admin] RBAC: Authorization granted {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:16:29.535481 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83ed2170-fc2a-409a-a8ce-baddd21bb1a0 None admin] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:16:29.549694 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: [pid: 80245|app: 0|req: 116/231] 10.222.0.47 () {58 vars in 1439 bytes} [Thu Jul 21 08:16:29 2022] PUT /identity/v3/projects/d4fa4cba54f348b68df92b37fb5fd5d1/users/70ffa790321f4f19b7935a59cb2fcfd9/roles/e29870f6e9e2411a9db0d5bf45f32ca5 => generated 0 bytes in 213 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 21 08:16:29.554046 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG dbcounter [-] [80244] Writing DB stats keystone:SELECT=100 {{(pid=80244) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:16:29.578774 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e2a185c-17e5-4b8f-aa8e-ec543cb36984 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:16:29.579706 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e2a185c-17e5-4b8f-aa8e-ec543cb36984 None admin] REQUEST_METHOD: `GET` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:16:29.580039 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e2a185c-17e5-4b8f-aa8e-ec543cb36984 None admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:16:29.580322 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e2a185c-17e5-4b8f-aa8e-ec543cb36984 None admin] PATH_INFO: `/v3/users/70ffa790321f4f19b7935a59cb2fcfd9` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:16:29.581663 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e2a185c-17e5-4b8f-aa8e-ec543cb36984 None admin] RBAC: Authorizing `identity:get_user(user_id=70ffa790321f4f19b7935a59cb2fcfd9)` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:16:29.583848 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e2a185c-17e5-4b8f-aa8e-ec543cb36984 None admin] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:16:29.585145 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 116/232] 10.222.0.47 () {56 vars in 1177 bytes} [Thu Jul 21 08:16:29 2022] GET /identity/v3/users/70ffa790321f4f19b7935a59cb2fcfd9 => generated 283 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 21 08:16:29.590751 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bca00db3-3ae1-4ec4-ba1e-d8cae145388f None admin] Authenticating user token {{(pid=80245) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:16:29.611980 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG dbcounter [-] [80245] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80245) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:16:29.731965 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bca00db3-3ae1-4ec4-ba1e-d8cae145388f None admin] Validating token access rules against request {{(pid=80245) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:16:29.774272 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fb3bc1b-ddb7-4019-8533-9854a3025952 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80245) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:16:29.775160 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fb3bc1b-ddb7-4019-8533-9854a3025952 None admin] REQUEST_METHOD: `GET` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:16:29.775541 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fb3bc1b-ddb7-4019-8533-9854a3025952 None admin] SCRIPT_NAME: `/identity` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:16:29.775822 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fb3bc1b-ddb7-4019-8533-9854a3025952 None admin] PATH_INFO: `/v3/projects/d4fa4cba54f348b68df92b37fb5fd5d1` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:16:29.777310 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fb3bc1b-ddb7-4019-8533-9854a3025952 None admin] RBAC: Authorizing `identity:get_project(project_id=d4fa4cba54f348b68df92b37fb5fd5d1)` {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:16:29.778769 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fb3bc1b-ddb7-4019-8533-9854a3025952 None admin] RBAC: Authorization granted {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:16:29.780678 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: [pid: 80245|app: 0|req: 117/233] 10.222.0.47 () {56 vars in 1186 bytes} [Thu Jul 21 08:16:29 2022] GET /identity/v3/projects/d4fa4cba54f348b68df92b37fb5fd5d1 => generated 302 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 08:16:29.786983 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e2a185c-17e5-4b8f-aa8e-ec543cb36984 None admin] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:16:29.905647 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e2a185c-17e5-4b8f-aa8e-ec543cb36984 None admin] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:16:29.948734 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aac9d407-a51a-4a72-9db9-063a7583f431 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:16:29.949633 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aac9d407-a51a-4a72-9db9-063a7583f431 None admin] REQUEST_METHOD: `PUT` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:16:29.949917 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aac9d407-a51a-4a72-9db9-063a7583f431 None admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:16:29.950195 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aac9d407-a51a-4a72-9db9-063a7583f431 None admin] PATH_INFO: `/v3/projects/d4fa4cba54f348b68df92b37fb5fd5d1/users/70ffa790321f4f19b7935a59cb2fcfd9/roles/99a638c0aac44b79a1f2858574724495` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:16:29.953385 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aac9d407-a51a-4a72-9db9-063a7583f431 None admin] RBAC: Authorizing `identity:create_grant(project_id=d4fa4cba54f348b68df92b37fb5fd5d1, user_id=70ffa790321f4f19b7935a59cb2fcfd9, role_id=99a638c0aac44b79a1f2858574724495)` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:16:29.954862 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aac9d407-a51a-4a72-9db9-063a7583f431 None admin] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:16:29.973964 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 117/234] 10.222.0.47 () {58 vars in 1439 bytes} [Thu Jul 21 08:16:29 2022] PUT /identity/v3/projects/d4fa4cba54f348b68df92b37fb5fd5d1/users/70ffa790321f4f19b7935a59cb2fcfd9/roles/99a638c0aac44b79a1f2858574724495 => generated 0 bytes in 188 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 08:16:30.747467 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc78000d-a519-4adb-8830-f8352ab89374 None None] REQUEST_METHOD: `GET` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:16:30.747938 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc78000d-a519-4adb-8830-f8352ab89374 None None] SCRIPT_NAME: `/identity` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:16:30.747938 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc78000d-a519-4adb-8830-f8352ab89374 None None] PATH_INFO: `/` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:16:30.749574 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: [pid: 80245|app: 0|req: 118/235] 10.222.0.47 () {54 vars in 926 bytes} [Thu Jul 21 08:16:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 08:16:30.761481 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e57596eb-cab0-45ef-b991-aa1229b137bd None None] REQUEST_METHOD: `POST` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:16:30.761958 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e57596eb-cab0-45ef-b991-aa1229b137bd None None] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:16:30.762148 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e57596eb-cab0-45ef-b991-aa1229b137bd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:16:30.783797 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.auth.core [None req-e57596eb-cab0-45ef-b991-aa1229b137bd None None] MFA Rules not processed for user `2dce40b8d74c41ec94cc36e6ff05d412`. Rule list: `[]` (Enabled: `True`). {{(pid=80244) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 08:16:30.785019 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.fernet_utils [None req-e57596eb-cab0-45ef-b991-aa1229b137bd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80244) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 08:16:30.830260 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG dbcounter [-] [80244] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80244) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:16:30.866891 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 118/236] 10.222.0.47 () {58 vars in 1023 bytes} [Thu Jul 21 08:16:30 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 21 08:16:30.911505 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc78000d-a519-4adb-8830-f8352ab89374 None None] Authenticating user token {{(pid=80245) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:16:30.931267 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18dbb04d-0167-454c-9e64-a1cac268da05 None None] REQUEST_METHOD: `GET` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:16:30.931595 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18dbb04d-0167-454c-9e64-a1cac268da05 None None] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:16:30.931871 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18dbb04d-0167-454c-9e64-a1cac268da05 None None] PATH_INFO: `/` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:16:30.933279 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 119/237] 10.222.0.47 () {54 vars in 926 bytes} [Thu Jul 21 08:16:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 08:16:30.938517 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bc16eed-aebe-438a-9c48-e2fb72d69b31 None None] REQUEST_METHOD: `POST` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:16:30.938812 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bc16eed-aebe-438a-9c48-e2fb72d69b31 None None] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:16:30.939096 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bc16eed-aebe-438a-9c48-e2fb72d69b31 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:16:30.962458 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.auth.core [None req-1bc16eed-aebe-438a-9c48-e2fb72d69b31 None None] MFA Rules not processed for user `2dce40b8d74c41ec94cc36e6ff05d412`. Rule list: `[]` (Enabled: `True`). {{(pid=80244) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 08:16:30.964487 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.fernet_utils [None req-1bc16eed-aebe-438a-9c48-e2fb72d69b31 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80244) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 08:16:31.052940 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG dbcounter [-] [80245] Writing DB stats keystone:SELECT=100 {{(pid=80245) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:16:31.073463 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc78000d-a519-4adb-8830-f8352ab89374 None None] Validating token access rules against request {{(pid=80245) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:16:31.111651 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b34e9444-dec0-4510-87c7-6f3c8fff71b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80245) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:16:31.112497 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b34e9444-dec0-4510-87c7-6f3c8fff71b0 None admin] REQUEST_METHOD: `GET` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:16:31.112891 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b34e9444-dec0-4510-87c7-6f3c8fff71b0 None admin] SCRIPT_NAME: `/identity` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:16:31.113260 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b34e9444-dec0-4510-87c7-6f3c8fff71b0 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:16:31.114548 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b34e9444-dec0-4510-87c7-6f3c8fff71b0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:16:31.116575 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b34e9444-dec0-4510-87c7-6f3c8fff71b0 None admin] RBAC: Authorization granted {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:16:31.118381 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: [pid: 80245|app: 0|req: 119/238] 10.222.0.47 () {56 vars in 1108 bytes} [Thu Jul 21 08:16:30 2022] GET /identity/v3/domains/default => generated 199 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 08:16:31.120164 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 120/239] 10.222.0.47 () {58 vars in 1023 bytes} [Thu Jul 21 08:16:30 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 182 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 21 08:16:31.124582 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b34e9444-dec0-4510-87c7-6f3c8fff71b0 None admin] Authenticating user token {{(pid=80245) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:16:31.204889 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bc16eed-aebe-438a-9c48-e2fb72d69b31 None None] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:16:31.257287 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b34e9444-dec0-4510-87c7-6f3c8fff71b0 None admin] Validating token access rules against request {{(pid=80245) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:16:31.320017 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56767aff-a055-43f1-8ac2-9cb39b3dbaa9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80245) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:16:31.320700 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56767aff-a055-43f1-8ac2-9cb39b3dbaa9 None admin] REQUEST_METHOD: `POST` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:16:31.320993 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56767aff-a055-43f1-8ac2-9cb39b3dbaa9 None admin] SCRIPT_NAME: `/identity` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:16:31.321270 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56767aff-a055-43f1-8ac2-9cb39b3dbaa9 None admin] PATH_INFO: `/v3/users` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:16:31.322000 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56767aff-a055-43f1-8ac2-9cb39b3dbaa9 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:16:31.327662 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56767aff-a055-43f1-8ac2-9cb39b3dbaa9 None admin] RBAC: Authorization granted {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:16:31.361361 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG dbcounter [-] [80245] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80245) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:16:31.368588 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: [pid: 80245|app: 0|req: 120/240] 10.222.0.47 () {60 vars in 1132 bytes} [Thu Jul 21 08:16:31 2022] POST /identity/v3/users => generated 293 bytes in 244 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 08:16:31.434405 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bc16eed-aebe-438a-9c48-e2fb72d69b31 None None] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:16:31.436205 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG dbcounter [-] [80244] Writing DB stats keystone:SELECT=100 {{(pid=80244) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:16:31.497829 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff68a731-096a-4736-a7de-35226b559b3b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:16:31.498659 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff68a731-096a-4736-a7de-35226b559b3b None admin] REQUEST_METHOD: `GET` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:16:31.498946 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff68a731-096a-4736-a7de-35226b559b3b None admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:16:31.499225 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff68a731-096a-4736-a7de-35226b559b3b None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:16:31.524481 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff68a731-096a-4736-a7de-35226b559b3b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 21 08:16:31.532382 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application [None req-ff68a731-096a-4736-a7de-35226b559b3b None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jul 21 08:16:31.532382 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 08:16:31.532382 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 08:16:31.532382 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 08:16:31.532382 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 08:16:31.532382 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 08:16:31.532382 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 08:16:31.532382 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 08:16:31.532382 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 08:16:31.532382 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 08:16:31.532382 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 08:16:31.532382 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 08:16:31.532382 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 21 08:16:31.532382 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 21 08:16:31.532382 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 21 08:16:31.532382 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 21 08:16:31.533264 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 21 08:16:31.533264 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 21 08:16:31.533264 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 21 08:16:31.533264 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 21 08:16:31.533264 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 08:16:31.533264 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 08:16:31.533264 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 08:16:31.533264 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 08:16:31.533264 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 08:16:31.533264 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 08:16:31.533264 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 08:16:31.533264 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application with Lock( Jul 21 08:16:31.533264 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 08:16:31.533264 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self._enter() Jul 21 08:16:31.533264 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 08:16:31.533264 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 08:16:31.534031 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 08:16:31.534031 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self.creator() Jul 21 08:16:31.534031 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 08:16:31.534031 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application created_value = creator( Jul 21 08:16:31.534031 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 21 08:16:31.534031 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 21 08:16:31.534031 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 21 08:16:31.534031 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 21 08:16:31.534031 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 21 08:16:31.534031 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 21 08:16:31.534031 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jul 21 08:16:31.534031 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application Jul 21 08:16:31.534031 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 121/241] 10.222.0.47 () {56 vars in 1099 bytes} [Thu Jul 21 08:16:31 2022] GET /identity/v3/roles/reader => generated 84 bytes in 329 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 08:16:31.544950 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56767aff-a055-43f1-8ac2-9cb39b3dbaa9 None admin] Authenticating user token {{(pid=80245) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:16:31.546644 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15f696b2-7969-4505-a1fa-850248c8da66 None None] REQUEST_METHOD: `GET` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:16:31.546644 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15f696b2-7969-4505-a1fa-850248c8da66 None None] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:16:31.546644 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15f696b2-7969-4505-a1fa-850248c8da66 None None] PATH_INFO: `/` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:16:31.547702 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 122/242] 10.222.0.47 () {54 vars in 926 bytes} [Thu Jul 21 08:16:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 21 08:16:31.553646 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4afc613-5f32-474a-b4ea-c35775aa4851 None None] REQUEST_METHOD: `POST` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:16:31.553935 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4afc613-5f32-474a-b4ea-c35775aa4851 None None] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:16:31.554218 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4afc613-5f32-474a-b4ea-c35775aa4851 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:16:31.575012 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.auth.core [None req-f4afc613-5f32-474a-b4ea-c35775aa4851 None None] MFA Rules not processed for user `2dce40b8d74c41ec94cc36e6ff05d412`. Rule list: `[]` (Enabled: `True`). {{(pid=80244) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 08:16:31.576599 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.fernet_utils [None req-f4afc613-5f32-474a-b4ea-c35775aa4851 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80244) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 08:16:31.659702 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 123/243] 10.222.0.47 () {58 vars in 1023 bytes} [Thu Jul 21 08:16:31 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 21 08:16:31.674816 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4afc613-5f32-474a-b4ea-c35775aa4851 None None] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:16:31.717778 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56767aff-a055-43f1-8ac2-9cb39b3dbaa9 None admin] Validating token access rules against request {{(pid=80245) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:16:31.760002 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be2b5552-5b1a-4954-9744-010b9fb19506 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80245) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:16:31.760731 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be2b5552-5b1a-4954-9744-010b9fb19506 None admin] REQUEST_METHOD: `GET` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:16:31.760970 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be2b5552-5b1a-4954-9744-010b9fb19506 None admin] SCRIPT_NAME: `/identity` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:16:31.761246 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be2b5552-5b1a-4954-9744-010b9fb19506 None admin] PATH_INFO: `/v3/roles` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:16:31.762000 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be2b5552-5b1a-4954-9744-010b9fb19506 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80245) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 08:16:31.762305 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be2b5552-5b1a-4954-9744-010b9fb19506 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:16:31.763704 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be2b5552-5b1a-4954-9744-010b9fb19506 None admin] RBAC: Authorization granted {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:16:31.773223 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: [pid: 80245|app: 0|req: 121/244] 10.222.0.47 () {56 vars in 1101 bytes} [Thu Jul 21 08:16:31 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 229 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 08:16:31.779007 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be2b5552-5b1a-4954-9744-010b9fb19506 None admin] Authenticating user token {{(pid=80245) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:16:31.785339 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4afc613-5f32-474a-b4ea-c35775aa4851 None None] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:16:31.787635 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG dbcounter [-] [80244] Writing DB stats keystone:SELECT=100 {{(pid=80244) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:16:31.832919 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97207d5b-9adc-4400-9370-e6bf98b95359 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:16:31.833760 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97207d5b-9adc-4400-9370-e6bf98b95359 None admin] REQUEST_METHOD: `GET` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:16:31.834678 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97207d5b-9adc-4400-9370-e6bf98b95359 None admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:16:31.834961 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97207d5b-9adc-4400-9370-e6bf98b95359 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:16:31.848931 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG dbcounter [-] [80245] Writing DB stats keystone:SELECT=100 {{(pid=80245) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:16:31.853202 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97207d5b-9adc-4400-9370-e6bf98b95359 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 21 08:16:31.862475 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application [None req-97207d5b-9adc-4400-9370-e6bf98b95359 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 21 08:16:31.862475 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 08:16:31.862475 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 08:16:31.862475 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 08:16:31.862475 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 08:16:31.862475 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 08:16:31.862475 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 08:16:31.862475 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 08:16:31.862475 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 08:16:31.862475 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 08:16:31.862475 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 08:16:31.862475 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 08:16:31.862475 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 21 08:16:31.862475 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 21 08:16:31.862475 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 21 08:16:31.862475 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 21 08:16:31.863421 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 21 08:16:31.863421 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 21 08:16:31.863421 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 21 08:16:31.863421 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 21 08:16:31.863421 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 08:16:31.863421 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 08:16:31.863421 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 08:16:31.863421 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 08:16:31.863421 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 08:16:31.863421 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 08:16:31.863421 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 08:16:31.863421 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application with Lock( Jul 21 08:16:31.863421 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 08:16:31.863421 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self._enter() Jul 21 08:16:31.863421 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 08:16:31.863421 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 08:16:31.864203 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 08:16:31.864203 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self.creator() Jul 21 08:16:31.864203 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 08:16:31.864203 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application created_value = creator( Jul 21 08:16:31.864203 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 21 08:16:31.864203 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 21 08:16:31.864203 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 21 08:16:31.864203 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 21 08:16:31.864203 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 21 08:16:31.864203 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 21 08:16:31.864203 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 21 08:16:31.864203 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application Jul 21 08:16:31.868443 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 124/245] 10.222.0.47 () {56 vars in 1099 bytes} [Thu Jul 21 08:16:31 2022] GET /identity/v3/roles/member => generated 84 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 21 08:16:31.871572 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97207d5b-9adc-4400-9370-e6bf98b95359 None admin] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:16:31.897376 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be2b5552-5b1a-4954-9744-010b9fb19506 None admin] Validating token access rules against request {{(pid=80245) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:16:31.942196 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a31ce416-ae08-4fd3-b46d-9abcf6992036 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80245) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:16:31.943085 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a31ce416-ae08-4fd3-b46d-9abcf6992036 None admin] REQUEST_METHOD: `GET` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:16:31.943462 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a31ce416-ae08-4fd3-b46d-9abcf6992036 None admin] SCRIPT_NAME: `/identity` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:16:31.944239 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a31ce416-ae08-4fd3-b46d-9abcf6992036 None admin] PATH_INFO: `/v3/users/578bf32a7bab47b3829483e36917158c` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:16:31.965015 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a31ce416-ae08-4fd3-b46d-9abcf6992036 None admin] RBAC: Authorizing `identity:get_user(user_id=578bf32a7bab47b3829483e36917158c)` {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:16:31.967639 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a31ce416-ae08-4fd3-b46d-9abcf6992036 None admin] RBAC: Authorization granted {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:16:31.969563 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: [pid: 80245|app: 0|req: 122/246] 10.222.0.47 () {56 vars in 1177 bytes} [Thu Jul 21 08:16:31 2022] GET /identity/v3/users/578bf32a7bab47b3829483e36917158c => generated 297 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 21 08:16:31.974759 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a31ce416-ae08-4fd3-b46d-9abcf6992036 None admin] Authenticating user token {{(pid=80245) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:16:32.045463 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97207d5b-9adc-4400-9370-e6bf98b95359 None admin] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:16:32.097271 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b62e6cf-d255-4a98-a193-feca5d4c042b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:16:32.098024 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b62e6cf-d255-4a98-a193-feca5d4c042b None admin] REQUEST_METHOD: `GET` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:16:32.098315 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b62e6cf-d255-4a98-a193-feca5d4c042b None admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:16:32.098598 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b62e6cf-d255-4a98-a193-feca5d4c042b None admin] PATH_INFO: `/v3/roles` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:16:32.099556 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b62e6cf-d255-4a98-a193-feca5d4c042b None admin] RBAC: Adding query filter params (name=member) {{(pid=80244) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 08:16:32.099864 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b62e6cf-d255-4a98-a193-feca5d4c042b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:16:32.101296 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b62e6cf-d255-4a98-a193-feca5d4c042b None admin] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:16:32.111584 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 125/247] 10.222.0.47 () {56 vars in 1101 bytes} [Thu Jul 21 08:16:31 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 241 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 08:16:32.118715 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b62e6cf-d255-4a98-a193-feca5d4c042b None admin] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:16:32.139071 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG dbcounter [-] [80244] Writing DB stats keystone:SELECT=100 {{(pid=80244) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:16:32.176332 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a31ce416-ae08-4fd3-b46d-9abcf6992036 None admin] Validating token access rules against request {{(pid=80245) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:16:32.214746 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG dbcounter [-] [80245] Writing DB stats keystone:SELECT=100 {{(pid=80245) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:16:32.243306 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b62e6cf-d255-4a98-a193-feca5d4c042b None admin] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:16:32.253739 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66414356-7381-4916-be09-1af73f04046d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80245) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:16:32.254873 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66414356-7381-4916-be09-1af73f04046d None admin] REQUEST_METHOD: `GET` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:16:32.255385 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66414356-7381-4916-be09-1af73f04046d None admin] SCRIPT_NAME: `/identity` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:16:32.256400 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66414356-7381-4916-be09-1af73f04046d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:16:32.284261 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aee93071-9445-4417-bb5c-a1ec980d6fc6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:16:32.285155 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aee93071-9445-4417-bb5c-a1ec980d6fc6 None admin] REQUEST_METHOD: `GET` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:16:32.285437 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aee93071-9445-4417-bb5c-a1ec980d6fc6 None admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:16:32.285711 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aee93071-9445-4417-bb5c-a1ec980d6fc6 None admin] PATH_INFO: `/v3/users/5a245876268b4f3797a25e891079b053` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:16:32.290005 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aee93071-9445-4417-bb5c-a1ec980d6fc6 None admin] RBAC: Authorizing `identity:get_user(user_id=5a245876268b4f3797a25e891079b053)` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:16:32.292574 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aee93071-9445-4417-bb5c-a1ec980d6fc6 None admin] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:16:32.294827 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 126/248] 10.222.0.47 () {56 vars in 1177 bytes} [Thu Jul 21 08:16:32 2022] GET /identity/v3/users/5a245876268b4f3797a25e891079b053 => generated 297 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 21 08:16:32.316373 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aee93071-9445-4417-bb5c-a1ec980d6fc6 None admin] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:16:32.359281 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66414356-7381-4916-be09-1af73f04046d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:16:32.360727 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66414356-7381-4916-be09-1af73f04046d None admin] RBAC: Authorization granted {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:16:32.481715 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: [pid: 80245|app: 0|req: 123/249] 10.222.0.47 () {58 vars in 1282 bytes} [Thu Jul 21 08:16:31 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 507 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 21 08:16:32.483279 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a28933b5-b47f-4e9f-9a45-0d6f0100e42d None None] REQUEST_METHOD: `GET` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:16:32.483596 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a28933b5-b47f-4e9f-9a45-0d6f0100e42d None None] SCRIPT_NAME: `/identity` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:16:32.483933 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a28933b5-b47f-4e9f-9a45-0d6f0100e42d None None] PATH_INFO: `/` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:16:32.485819 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: [pid: 80245|app: 0|req: 124/250] 10.222.0.47 () {54 vars in 926 bytes} [Thu Jul 21 08:16:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 08:16:32.492303 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a28933b5-b47f-4e9f-9a45-0d6f0100e42d None None] Authenticating user token {{(pid=80245) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:16:32.535362 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG dbcounter [-] [80245] Writing DB stats keystone:SELECT=100 {{(pid=80245) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:16:32.544422 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG dbcounter [-] [80244] Writing DB stats keystone:SELECT=100 {{(pid=80244) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:16:32.606586 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aee93071-9445-4417-bb5c-a1ec980d6fc6 None admin] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:16:32.614565 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a28933b5-b47f-4e9f-9a45-0d6f0100e42d None None] Validating token access rules against request {{(pid=80245) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:16:32.654101 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8305168c-4afc-49a2-804a-cd0d67af1c11 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80245) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:16:32.655372 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8305168c-4afc-49a2-804a-cd0d67af1c11 None admin] REQUEST_METHOD: `GET` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:16:32.655872 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8305168c-4afc-49a2-804a-cd0d67af1c11 None admin] SCRIPT_NAME: `/identity` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:16:32.656354 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8305168c-4afc-49a2-804a-cd0d67af1c11 None admin] PATH_INFO: `/v3/projects/d4fa4cba54f348b68df92b37fb5fd5d1` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:16:32.657738 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8305168c-4afc-49a2-804a-cd0d67af1c11 None admin] RBAC: Authorizing `identity:get_project(project_id=d4fa4cba54f348b68df92b37fb5fd5d1)` {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:16:32.659414 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8305168c-4afc-49a2-804a-cd0d67af1c11 None admin] RBAC: Authorization granted {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:16:32.661637 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: [pid: 80245|app: 0|req: 125/251] 10.222.0.47 () {56 vars in 1186 bytes} [Thu Jul 21 08:16:32 2022] GET /identity/v3/projects/d4fa4cba54f348b68df92b37fb5fd5d1 => generated 302 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 08:16:32.663912 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05baeedc-001d-479a-a198-99ebf5683eda None None] REQUEST_METHOD: `POST` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:16:32.664493 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05baeedc-001d-479a-a198-99ebf5683eda None None] SCRIPT_NAME: `/identity` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:16:32.664965 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05baeedc-001d-479a-a198-99ebf5683eda None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:16:32.690619 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.auth.core [None req-05baeedc-001d-479a-a198-99ebf5683eda None None] MFA Rules not processed for user `2dce40b8d74c41ec94cc36e6ff05d412`. Rule list: `[]` (Enabled: `True`). {{(pid=80245) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 08:16:32.692739 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.fernet_utils [None req-05baeedc-001d-479a-a198-99ebf5683eda None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80245) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 08:16:32.697942 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b2dc1ae-6a2f-49d5-932a-246348485e4b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:16:32.698833 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b2dc1ae-6a2f-49d5-932a-246348485e4b None admin] REQUEST_METHOD: `GET` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:16:32.699117 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b2dc1ae-6a2f-49d5-932a-246348485e4b None admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:16:32.699416 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b2dc1ae-6a2f-49d5-932a-246348485e4b None admin] PATH_INFO: `/v3/projects/d4fa4cba54f348b68df92b37fb5fd5d1` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:16:32.701994 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b2dc1ae-6a2f-49d5-932a-246348485e4b None admin] RBAC: Authorizing `identity:get_project(project_id=d4fa4cba54f348b68df92b37fb5fd5d1)` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:16:32.704560 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b2dc1ae-6a2f-49d5-932a-246348485e4b None admin] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:16:32.713279 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 127/252] 10.222.0.47 () {56 vars in 1186 bytes} [Thu Jul 21 08:16:32 2022] GET /identity/v3/projects/d4fa4cba54f348b68df92b37fb5fd5d1 => generated 302 bytes in 397 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 08:16:32.713957 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b2dc1ae-6a2f-49d5-932a-246348485e4b None admin] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:16:32.834750 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: [pid: 80245|app: 0|req: 126/253] 10.222.0.47 () {58 vars in 1023 bytes} [Thu Jul 21 08:16:32 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 21 08:16:32.835446 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05baeedc-001d-479a-a198-99ebf5683eda None None] Authenticating user token {{(pid=80245) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:16:32.873444 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG dbcounter [-] [80245] Writing DB stats keystone:SELECT=100 {{(pid=80245) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:16:32.910144 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b2dc1ae-6a2f-49d5-932a-246348485e4b None admin] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:16:32.957987 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fe55281-26a6-4361-b04d-2e724492c75e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:16:32.958658 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fe55281-26a6-4361-b04d-2e724492c75e None admin] REQUEST_METHOD: `GET` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:16:32.958976 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fe55281-26a6-4361-b04d-2e724492c75e None admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:16:32.959251 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fe55281-26a6-4361-b04d-2e724492c75e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:16:32.960393 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fe55281-26a6-4361-b04d-2e724492c75e None admin] RBAC: Adding query filter params (role.id=9e5171884e9f401ab2337b74db49546e, scope.project.id=d4fa4cba54f348b68df92b37fb5fd5d1, user.id=578bf32a7bab47b3829483e36917158c) {{(pid=80244) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 08:16:32.960705 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fe55281-26a6-4361-b04d-2e724492c75e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:16:32.962905 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fe55281-26a6-4361-b04d-2e724492c75e None admin] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:16:32.969859 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 128/254] 10.222.0.47 () {56 vars in 1374 bytes} [Thu Jul 21 08:16:32 2022] GET /identity/v3/role_assignments?user.id=578bf32a7bab47b3829483e36917158c&scope.project.id=d4fa4cba54f348b68df92b37fb5fd5d1&role.id=9e5171884e9f401ab2337b74db49546e => generated 259 bytes in 256 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 08:16:32.971286 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fe55281-26a6-4361-b04d-2e724492c75e None admin] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:16:33.047762 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG dbcounter [-] [80244] Writing DB stats keystone:SELECT=100 {{(pid=80244) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:16:33.078736 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05baeedc-001d-479a-a198-99ebf5683eda None None] Validating token access rules against request {{(pid=80245) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:16:33.126287 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a14894c3-888c-403b-a21a-352cf4a8ae7a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80245) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:16:33.127188 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a14894c3-888c-403b-a21a-352cf4a8ae7a None admin] REQUEST_METHOD: `PUT` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:16:33.127497 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a14894c3-888c-403b-a21a-352cf4a8ae7a None admin] SCRIPT_NAME: `/identity` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:16:33.127778 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a14894c3-888c-403b-a21a-352cf4a8ae7a None admin] PATH_INFO: `/v3/projects/d4fa4cba54f348b68df92b37fb5fd5d1/users/5a245876268b4f3797a25e891079b053/roles/3a074738221e4601b5205715ba87d240` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:16:33.132264 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a14894c3-888c-403b-a21a-352cf4a8ae7a None admin] RBAC: Authorizing `identity:create_grant(project_id=d4fa4cba54f348b68df92b37fb5fd5d1, user_id=5a245876268b4f3797a25e891079b053, role_id=3a074738221e4601b5205715ba87d240)` {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:16:33.134063 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a14894c3-888c-403b-a21a-352cf4a8ae7a None admin] RBAC: Authorization granted {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:16:33.142777 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fe55281-26a6-4361-b04d-2e724492c75e None admin] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:16:33.157745 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: [pid: 80245|app: 0|req: 127/255] 10.222.0.47 () {58 vars in 1439 bytes} [Thu Jul 21 08:16:32 2022] PUT /identity/v3/projects/d4fa4cba54f348b68df92b37fb5fd5d1/users/5a245876268b4f3797a25e891079b053/roles/3a074738221e4601b5205715ba87d240 => generated 0 bytes in 322 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 08:16:33.160196 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa1ba13a-2a52-4c4e-ae40-fe9a342f3574 None None] REQUEST_METHOD: `GET` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:16:33.160704 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa1ba13a-2a52-4c4e-ae40-fe9a342f3574 None None] SCRIPT_NAME: `/identity` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:16:33.161277 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa1ba13a-2a52-4c4e-ae40-fe9a342f3574 None None] PATH_INFO: `/` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:16:33.163127 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: [pid: 80245|app: 0|req: 128/256] 10.222.0.47 () {54 vars in 926 bytes} [Thu Jul 21 08:16:33 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 08:16:33.170626 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-931f862f-6b8b-4d24-8ec0-28947105232a None None] REQUEST_METHOD: `POST` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:16:33.171288 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-931f862f-6b8b-4d24-8ec0-28947105232a None None] SCRIPT_NAME: `/identity` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:16:33.172087 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-931f862f-6b8b-4d24-8ec0-28947105232a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:16:33.196666 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9879f679-f18d-4eaf-aa8c-e6263486ea43 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:16:33.197594 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.auth.core [None req-931f862f-6b8b-4d24-8ec0-28947105232a None None] MFA Rules not processed for user `2dce40b8d74c41ec94cc36e6ff05d412`. Rule list: `[]` (Enabled: `True`). {{(pid=80245) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 08:16:33.197724 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9879f679-f18d-4eaf-aa8c-e6263486ea43 None admin] REQUEST_METHOD: `GET` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:16:33.197887 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9879f679-f18d-4eaf-aa8c-e6263486ea43 None admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:16:33.198172 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9879f679-f18d-4eaf-aa8c-e6263486ea43 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:16:33.199905 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.fernet_utils [None req-931f862f-6b8b-4d24-8ec0-28947105232a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80245) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 08:16:33.213128 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9879f679-f18d-4eaf-aa8c-e6263486ea43 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 21 08:16:33.230175 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application [None req-9879f679-f18d-4eaf-aa8c-e6263486ea43 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 21 08:16:33.230175 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 08:16:33.230175 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 08:16:33.230175 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 08:16:33.230175 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 08:16:33.230175 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 08:16:33.230175 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 08:16:33.230175 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 08:16:33.230175 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 08:16:33.230175 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 08:16:33.230175 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 08:16:33.230175 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 08:16:33.230175 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 21 08:16:33.230175 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 21 08:16:33.230175 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 21 08:16:33.230175 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 21 08:16:33.231130 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 21 08:16:33.231130 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 21 08:16:33.231130 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 21 08:16:33.231130 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 21 08:16:33.231130 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 08:16:33.231130 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 08:16:33.231130 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 08:16:33.231130 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 08:16:33.231130 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 08:16:33.231130 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 08:16:33.231130 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 08:16:33.231130 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application with Lock( Jul 21 08:16:33.231130 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 08:16:33.231130 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self._enter() Jul 21 08:16:33.231130 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 08:16:33.231130 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 08:16:33.231938 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 08:16:33.231938 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self.creator() Jul 21 08:16:33.231938 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 08:16:33.231938 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application created_value = creator( Jul 21 08:16:33.231938 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 21 08:16:33.231938 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 21 08:16:33.231938 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 21 08:16:33.231938 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 21 08:16:33.231938 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 21 08:16:33.231938 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 21 08:16:33.231938 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 21 08:16:33.231938 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application Jul 21 08:16:33.231938 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 129/257] 10.222.0.47 () {56 vars in 1096 bytes} [Thu Jul 21 08:16:32 2022] GET /identity/v3/roles/admin => generated 83 bytes in 261 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 08:16:33.237857 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9879f679-f18d-4eaf-aa8c-e6263486ea43 None admin] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:16:33.286839 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: [pid: 80245|app: 0|req: 129/258] 10.222.0.47 () {58 vars in 1023 bytes} [Thu Jul 21 08:16:33 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 21 08:16:33.323248 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-931f862f-6b8b-4d24-8ec0-28947105232a None None] Authenticating user token {{(pid=80245) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:16:33.337820 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG dbcounter [-] [80245] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80245) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:16:33.416682 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG dbcounter [-] [80244] Writing DB stats keystone:SELECT=100 {{(pid=80244) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:16:33.444977 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9879f679-f18d-4eaf-aa8c-e6263486ea43 None admin] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:16:33.484106 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62ecf48d-3e45-48dc-b457-2ea856c38e4b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:16:33.484798 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62ecf48d-3e45-48dc-b457-2ea856c38e4b None admin] REQUEST_METHOD: `GET` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:16:33.485178 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62ecf48d-3e45-48dc-b457-2ea856c38e4b None admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:16:33.485466 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62ecf48d-3e45-48dc-b457-2ea856c38e4b None admin] PATH_INFO: `/v3/roles` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:16:33.486332 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62ecf48d-3e45-48dc-b457-2ea856c38e4b None admin] RBAC: Adding query filter params (name=admin) {{(pid=80244) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 08:16:33.486643 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62ecf48d-3e45-48dc-b457-2ea856c38e4b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:16:33.488174 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62ecf48d-3e45-48dc-b457-2ea856c38e4b None admin] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:16:33.495050 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-931f862f-6b8b-4d24-8ec0-28947105232a None None] Validating token access rules against request {{(pid=80245) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:16:33.498554 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 130/259] 10.222.0.47 () {56 vars in 1099 bytes} [Thu Jul 21 08:16:33 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 261 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 08:16:33.513383 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62ecf48d-3e45-48dc-b457-2ea856c38e4b None admin] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:16:33.537880 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf5ec5a9-e4ca-4d5a-9534-f6268727cc6d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80245) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:16:33.538707 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf5ec5a9-e4ca-4d5a-9534-f6268727cc6d None admin] REQUEST_METHOD: `GET` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:16:33.538993 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf5ec5a9-e4ca-4d5a-9534-f6268727cc6d None admin] SCRIPT_NAME: `/identity` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:16:33.539272 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf5ec5a9-e4ca-4d5a-9534-f6268727cc6d None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:16:33.550216 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf5ec5a9-e4ca-4d5a-9534-f6268727cc6d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 21 08:16:33.558539 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application [None req-bf5ec5a9-e4ca-4d5a-9534-f6268727cc6d None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 21 08:16:33.558539 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 08:16:33.558539 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 08:16:33.558539 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 08:16:33.558539 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 08:16:33.558539 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 08:16:33.558539 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 08:16:33.558539 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 08:16:33.558539 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 08:16:33.558539 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 08:16:33.558539 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 08:16:33.558539 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 08:16:33.558539 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 21 08:16:33.558539 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 21 08:16:33.558539 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 21 08:16:33.558539 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 21 08:16:33.559452 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 21 08:16:33.559452 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 21 08:16:33.559452 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 21 08:16:33.559452 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 21 08:16:33.559452 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 08:16:33.559452 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 08:16:33.559452 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 08:16:33.559452 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 08:16:33.559452 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 08:16:33.559452 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 08:16:33.559452 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 08:16:33.559452 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application with Lock( Jul 21 08:16:33.559452 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 08:16:33.559452 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application return self._enter() Jul 21 08:16:33.559452 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 08:16:33.559452 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 08:16:33.560226 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 08:16:33.560226 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application return self.creator() Jul 21 08:16:33.560226 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 08:16:33.560226 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application created_value = creator( Jul 21 08:16:33.560226 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 21 08:16:33.560226 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 21 08:16:33.560226 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 21 08:16:33.560226 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 21 08:16:33.560226 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 21 08:16:33.560226 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 21 08:16:33.560226 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 21 08:16:33.560226 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application Jul 21 08:16:33.560226 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: [pid: 80245|app: 0|req: 130/260] 10.222.0.47 () {56 vars in 1114 bytes} [Thu Jul 21 08:16:33 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 237 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 21 08:16:33.565697 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf5ec5a9-e4ca-4d5a-9534-f6268727cc6d None admin] Authenticating user token {{(pid=80245) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:16:33.667008 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG dbcounter [-] [80245] Writing DB stats keystone:SELECT=100 {{(pid=80245) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:16:33.690043 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf5ec5a9-e4ca-4d5a-9534-f6268727cc6d None admin] Validating token access rules against request {{(pid=80245) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:16:33.747927 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62ecf48d-3e45-48dc-b457-2ea856c38e4b None admin] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:16:33.788932 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a35387a-eaa1-47f6-82fb-1b6f738ae4f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:16:33.789925 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a35387a-eaa1-47f6-82fb-1b6f738ae4f1 None admin] REQUEST_METHOD: `GET` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:16:33.790216 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a35387a-eaa1-47f6-82fb-1b6f738ae4f1 None admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:16:33.790496 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a35387a-eaa1-47f6-82fb-1b6f738ae4f1 None admin] PATH_INFO: `/v3/users/70ffa790321f4f19b7935a59cb2fcfd9` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:16:33.791922 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a35387a-eaa1-47f6-82fb-1b6f738ae4f1 None admin] RBAC: Authorizing `identity:get_user(user_id=70ffa790321f4f19b7935a59cb2fcfd9)` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:16:33.793488 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a35387a-eaa1-47f6-82fb-1b6f738ae4f1 None admin] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:16:33.795150 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 131/261] 10.222.0.47 () {56 vars in 1177 bytes} [Thu Jul 21 08:16:33 2022] GET /identity/v3/users/70ffa790321f4f19b7935a59cb2fcfd9 => generated 283 bytes in 282 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 08:16:33.800099 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b362042-7473-45bd-a7fd-c15148122a8e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80245) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:16:33.800797 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b362042-7473-45bd-a7fd-c15148122a8e None admin] REQUEST_METHOD: `GET` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:16:33.800922 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a35387a-eaa1-47f6-82fb-1b6f738ae4f1 None admin] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:16:33.801088 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b362042-7473-45bd-a7fd-c15148122a8e None admin] SCRIPT_NAME: `/identity` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:16:33.801350 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b362042-7473-45bd-a7fd-c15148122a8e None admin] PATH_INFO: `/v3/roles` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:16:33.802105 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b362042-7473-45bd-a7fd-c15148122a8e None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80245) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 08:16:33.802408 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b362042-7473-45bd-a7fd-c15148122a8e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:16:33.803863 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b362042-7473-45bd-a7fd-c15148122a8e None admin] RBAC: Authorization granted {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:16:33.826178 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: [pid: 80245|app: 0|req: 131/262] 10.222.0.47 () {56 vars in 1111 bytes} [Thu Jul 21 08:16:33 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 261 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 08:16:33.832286 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b362042-7473-45bd-a7fd-c15148122a8e None admin] Authenticating user token {{(pid=80245) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:16:33.835694 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG dbcounter [-] [80244] Writing DB stats keystone:SELECT=100 {{(pid=80244) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:16:33.913362 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a35387a-eaa1-47f6-82fb-1b6f738ae4f1 None admin] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:16:33.970750 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc2358af-96bb-462a-89b4-1a992371b700 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:16:33.971504 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc2358af-96bb-462a-89b4-1a992371b700 None admin] REQUEST_METHOD: `GET` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:16:33.971795 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc2358af-96bb-462a-89b4-1a992371b700 None admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:16:33.972073 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc2358af-96bb-462a-89b4-1a992371b700 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:16:34.053412 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b362042-7473-45bd-a7fd-c15148122a8e None admin] Validating token access rules against request {{(pid=80245) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:16:34.084748 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc2358af-96bb-462a-89b4-1a992371b700 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:16:34.086316 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc2358af-96bb-462a-89b4-1a992371b700 None admin] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:16:34.100329 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG dbcounter [-] [80245] Writing DB stats keystone:SELECT=100 {{(pid=80245) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:16:34.102069 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG dbcounter [-] [80244] Writing DB stats keystone:SELECT=100 {{(pid=80244) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:16:34.113147 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cae74f1-01c9-405a-8c9f-6950ee3952be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80245) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:16:34.114039 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cae74f1-01c9-405a-8c9f-6950ee3952be None admin] REQUEST_METHOD: `GET` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:16:34.114319 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cae74f1-01c9-405a-8c9f-6950ee3952be None admin] SCRIPT_NAME: `/identity` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:16:34.114597 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cae74f1-01c9-405a-8c9f-6950ee3952be None admin] PATH_INFO: `/v3/users/70ffa790321f4f19b7935a59cb2fcfd9` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:16:34.120146 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cae74f1-01c9-405a-8c9f-6950ee3952be None admin] RBAC: Authorizing `identity:get_user(user_id=70ffa790321f4f19b7935a59cb2fcfd9)` {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:16:34.121867 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cae74f1-01c9-405a-8c9f-6950ee3952be None admin] RBAC: Authorization granted {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:16:34.123582 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: [pid: 80245|app: 0|req: 132/263] 10.222.0.47 () {56 vars in 1177 bytes} [Thu Jul 21 08:16:33 2022] GET /identity/v3/users/70ffa790321f4f19b7935a59cb2fcfd9 => generated 283 bytes in 292 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 08:16:34.129310 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cae74f1-01c9-405a-8c9f-6950ee3952be None admin] Authenticating user token {{(pid=80245) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:16:34.290618 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 132/264] 10.222.0.47 () {58 vars in 1282 bytes} [Thu Jul 21 08:16:33 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 490 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Jul 21 08:16:34.296010 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc2358af-96bb-462a-89b4-1a992371b700 None admin] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:16:34.305280 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cae74f1-01c9-405a-8c9f-6950ee3952be None admin] Validating token access rules against request {{(pid=80245) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:16:34.374228 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcff3bbf-8432-4a89-ab41-739ce12f07b9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80245) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:16:34.374908 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcff3bbf-8432-4a89-ab41-739ce12f07b9 None admin] REQUEST_METHOD: `GET` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:16:34.375195 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcff3bbf-8432-4a89-ab41-739ce12f07b9 None admin] SCRIPT_NAME: `/identity` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:16:34.375516 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcff3bbf-8432-4a89-ab41-739ce12f07b9 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:16:34.526886 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG dbcounter [-] [80245] Writing DB stats keystone:SELECT=100 {{(pid=80245) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:16:34.541311 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcff3bbf-8432-4a89-ab41-739ce12f07b9 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:16:34.542732 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcff3bbf-8432-4a89-ab41-739ce12f07b9 None admin] RBAC: Authorization granted {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:16:34.565657 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc2358af-96bb-462a-89b4-1a992371b700 None admin] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:16:34.570849 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG dbcounter [-] [80244] Writing DB stats keystone:SELECT=100 {{(pid=80244) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:16:34.613649 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c29ea183-67c5-4729-919b-f86663f44c72 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:16:34.614524 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c29ea183-67c5-4729-919b-f86663f44c72 None admin] REQUEST_METHOD: `GET` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:16:34.615304 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c29ea183-67c5-4729-919b-f86663f44c72 None admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:16:34.615623 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c29ea183-67c5-4729-919b-f86663f44c72 None admin] PATH_INFO: `/v3/projects/d4fa4cba54f348b68df92b37fb5fd5d1` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:16:34.616863 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c29ea183-67c5-4729-919b-f86663f44c72 None admin] RBAC: Authorizing `identity:get_project(project_id=d4fa4cba54f348b68df92b37fb5fd5d1)` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:16:34.618323 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c29ea183-67c5-4729-919b-f86663f44c72 None admin] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:16:34.620386 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 133/265] 10.222.0.47 () {56 vars in 1186 bytes} [Thu Jul 21 08:16:34 2022] GET /identity/v3/projects/d4fa4cba54f348b68df92b37fb5fd5d1 => generated 302 bytes in 325 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 08:16:34.622043 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48f99781-cf91-4e80-9fcd-0aab4081b370 None None] REQUEST_METHOD: `GET` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:16:34.622332 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48f99781-cf91-4e80-9fcd-0aab4081b370 None None] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:16:34.622613 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48f99781-cf91-4e80-9fcd-0aab4081b370 None None] PATH_INFO: `/` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:16:34.624391 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 134/266] 10.222.0.47 () {54 vars in 926 bytes} [Thu Jul 21 08:16:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 08:16:34.638239 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2654fe5-cf82-4a36-910c-bbe4f44ef284 None None] REQUEST_METHOD: `POST` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:16:34.638532 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2654fe5-cf82-4a36-910c-bbe4f44ef284 None None] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:16:34.638815 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2654fe5-cf82-4a36-910c-bbe4f44ef284 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:16:34.666960 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.auth.core [None req-b2654fe5-cf82-4a36-910c-bbe4f44ef284 None None] MFA Rules not processed for user `2dce40b8d74c41ec94cc36e6ff05d412`. Rule list: `[]` (Enabled: `True`). {{(pid=80244) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 08:16:34.668200 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.fernet_utils [None req-b2654fe5-cf82-4a36-910c-bbe4f44ef284 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80244) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 08:16:34.736997 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: [pid: 80245|app: 0|req: 133/267] 10.222.0.47 () {58 vars in 1282 bytes} [Thu Jul 21 08:16:34 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 608 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 21 08:16:34.737861 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcff3bbf-8432-4a89-ab41-739ce12f07b9 None admin] Authenticating user token {{(pid=80245) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:16:34.772356 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 135/268] 10.222.0.47 () {58 vars in 1023 bytes} [Thu Jul 21 08:16:34 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 21 08:16:34.773681 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5cdabfb-0e6b-41f2-b190-cccd57427325 None None] REQUEST_METHOD: `GET` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:16:34.774066 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5cdabfb-0e6b-41f2-b190-cccd57427325 None None] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:16:34.774344 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5cdabfb-0e6b-41f2-b190-cccd57427325 None None] PATH_INFO: `/` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:16:34.775997 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 136/269] 10.222.0.47 () {54 vars in 926 bytes} [Thu Jul 21 08:16:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 08:16:34.776695 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5cdabfb-0e6b-41f2-b190-cccd57427325 None None] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:16:34.840666 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG dbcounter [-] [80245] Writing DB stats keystone:SELECT=100 {{(pid=80245) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:16:34.852127 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcff3bbf-8432-4a89-ab41-739ce12f07b9 None admin] Validating token access rules against request {{(pid=80245) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:16:34.890162 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8517b89-768a-4013-a803-69b63a21480b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80245) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:16:34.890926 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8517b89-768a-4013-a803-69b63a21480b None admin] REQUEST_METHOD: `GET` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:16:34.891283 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8517b89-768a-4013-a803-69b63a21480b None admin] SCRIPT_NAME: `/identity` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:16:34.891654 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8517b89-768a-4013-a803-69b63a21480b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:16:34.892604 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8517b89-768a-4013-a803-69b63a21480b None admin] RBAC: Adding query filter params (role.id=e29870f6e9e2411a9db0d5bf45f32ca5, scope.project.id=d4fa4cba54f348b68df92b37fb5fd5d1, user.id=70ffa790321f4f19b7935a59cb2fcfd9) {{(pid=80245) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 08:16:34.892960 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8517b89-768a-4013-a803-69b63a21480b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:16:34.894486 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8517b89-768a-4013-a803-69b63a21480b None admin] RBAC: Authorization granted {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:16:34.899129 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5cdabfb-0e6b-41f2-b190-cccd57427325 None None] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:16:34.900701 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG dbcounter [-] [80244] Writing DB stats keystone:SELECT=100 {{(pid=80244) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:16:34.901234 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: [pid: 80245|app: 0|req: 134/270] 10.222.0.47 () {56 vars in 1374 bytes} [Thu Jul 21 08:16:34 2022] GET /identity/v3/role_assignments?user.id=70ffa790321f4f19b7935a59cb2fcfd9&scope.project.id=d4fa4cba54f348b68df92b37fb5fd5d1&role.id=e29870f6e9e2411a9db0d5bf45f32ca5 => generated 608 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 08:16:34.902618 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f05e9d41-00a1-4c9a-b5d3-f9f915c6c4b7 None None] REQUEST_METHOD: `POST` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:16:34.902975 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f05e9d41-00a1-4c9a-b5d3-f9f915c6c4b7 None None] SCRIPT_NAME: `/identity` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:16:34.903372 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f05e9d41-00a1-4c9a-b5d3-f9f915c6c4b7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:16:34.921172 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.auth.core [None req-f05e9d41-00a1-4c9a-b5d3-f9f915c6c4b7 None None] MFA Rules not processed for user `2dce40b8d74c41ec94cc36e6ff05d412`. Rule list: `[]` (Enabled: `True`). {{(pid=80245) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 08:16:34.922441 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.fernet_utils [None req-f05e9d41-00a1-4c9a-b5d3-f9f915c6c4b7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80245) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 08:16:34.939705 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a64bbc27-a8a1-47e9-92eb-bc8d39f55f2f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:16:34.940571 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a64bbc27-a8a1-47e9-92eb-bc8d39f55f2f None admin] REQUEST_METHOD: `GET` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:16:34.940950 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a64bbc27-a8a1-47e9-92eb-bc8d39f55f2f None admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:16:34.941224 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a64bbc27-a8a1-47e9-92eb-bc8d39f55f2f None admin] PATH_INFO: `/v3/projects/d4fa4cba54f348b68df92b37fb5fd5d1` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:16:34.942492 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a64bbc27-a8a1-47e9-92eb-bc8d39f55f2f None admin] RBAC: Authorizing `identity:get_project(project_id=d4fa4cba54f348b68df92b37fb5fd5d1)` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:16:34.944523 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a64bbc27-a8a1-47e9-92eb-bc8d39f55f2f None admin] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:16:34.946636 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 137/271] 10.222.0.47 () {56 vars in 1186 bytes} [Thu Jul 21 08:16:34 2022] GET /identity/v3/projects/d4fa4cba54f348b68df92b37fb5fd5d1 => generated 302 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 08:16:34.947457 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a64bbc27-a8a1-47e9-92eb-bc8d39f55f2f None admin] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:16:35.002371 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: [pid: 80245|app: 0|req: 135/272] 10.222.0.47 () {58 vars in 1023 bytes} [Thu Jul 21 08:16:34 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 21 08:16:35.003275 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f05e9d41-00a1-4c9a-b5d3-f9f915c6c4b7 None None] Authenticating user token {{(pid=80245) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:16:35.096538 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a64bbc27-a8a1-47e9-92eb-bc8d39f55f2f None admin] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:16:35.098980 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG dbcounter [-] [80245] Writing DB stats keystone:SELECT=100 {{(pid=80245) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:16:35.125906 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f05e9d41-00a1-4c9a-b5d3-f9f915c6c4b7 None None] Validating token access rules against request {{(pid=80245) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:16:35.168601 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1826198e-c6b5-4f88-86c4-2b4c3ad82789 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:16:35.169751 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1826198e-c6b5-4f88-86c4-2b4c3ad82789 None admin] REQUEST_METHOD: `GET` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:16:35.170187 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1826198e-c6b5-4f88-86c4-2b4c3ad82789 None admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:16:35.170470 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1826198e-c6b5-4f88-86c4-2b4c3ad82789 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:16:35.171637 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f04f9c7-4448-47a1-9e87-6e3fe165a745 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80245) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:16:35.172353 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f04f9c7-4448-47a1-9e87-6e3fe165a745 None admin] REQUEST_METHOD: `GET` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:16:35.172633 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f04f9c7-4448-47a1-9e87-6e3fe165a745 None admin] SCRIPT_NAME: `/identity` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:16:35.172927 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f04f9c7-4448-47a1-9e87-6e3fe165a745 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:16:35.173750 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f04f9c7-4448-47a1-9e87-6e3fe165a745 None admin] RBAC: Adding query filter params (role.id=99a638c0aac44b79a1f2858574724495, scope.project.id=d4fa4cba54f348b68df92b37fb5fd5d1, user.id=70ffa790321f4f19b7935a59cb2fcfd9) {{(pid=80245) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 08:16:35.174062 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f04f9c7-4448-47a1-9e87-6e3fe165a745 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:16:35.177315 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f04f9c7-4448-47a1-9e87-6e3fe165a745 None admin] RBAC: Authorization granted {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:16:35.182391 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: [pid: 80245|app: 0|req: 136/273] 10.222.0.47 () {56 vars in 1374 bytes} [Thu Jul 21 08:16:35 2022] GET /identity/v3/role_assignments?user.id=70ffa790321f4f19b7935a59cb2fcfd9&scope.project.id=d4fa4cba54f348b68df92b37fb5fd5d1&role.id=99a638c0aac44b79a1f2858574724495 => generated 608 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 08:16:35.183342 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f04f9c7-4448-47a1-9e87-6e3fe165a745 None admin] Authenticating user token {{(pid=80245) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:16:35.185871 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1826198e-c6b5-4f88-86c4-2b4c3ad82789 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 21 08:16:35.193921 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application [None req-1826198e-c6b5-4f88-86c4-2b4c3ad82789 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 21 08:16:35.193921 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 08:16:35.193921 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 08:16:35.193921 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 08:16:35.193921 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 08:16:35.193921 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 08:16:35.193921 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 08:16:35.193921 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 08:16:35.193921 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 08:16:35.193921 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 08:16:35.193921 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 08:16:35.193921 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 08:16:35.193921 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 21 08:16:35.193921 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 21 08:16:35.193921 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 21 08:16:35.193921 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 21 08:16:35.194861 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 21 08:16:35.194861 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 21 08:16:35.194861 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 21 08:16:35.194861 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 21 08:16:35.194861 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 08:16:35.194861 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 08:16:35.194861 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 08:16:35.194861 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 08:16:35.194861 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 08:16:35.194861 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 08:16:35.194861 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 08:16:35.194861 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application with Lock( Jul 21 08:16:35.194861 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 08:16:35.194861 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self._enter() Jul 21 08:16:35.194861 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 08:16:35.194861 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 08:16:35.195671 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 08:16:35.195671 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self.creator() Jul 21 08:16:35.195671 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 08:16:35.195671 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application created_value = creator( Jul 21 08:16:35.195671 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 21 08:16:35.195671 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 21 08:16:35.195671 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 21 08:16:35.195671 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 21 08:16:35.195671 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 21 08:16:35.195671 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 21 08:16:35.195671 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 21 08:16:35.195671 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application Jul 21 08:16:35.195671 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 138/274] 10.222.0.47 () {56 vars in 1099 bytes} [Thu Jul 21 08:16:34 2022] GET /identity/v3/roles/member => generated 84 bytes in 248 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 08:16:35.203082 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1826198e-c6b5-4f88-86c4-2b4c3ad82789 None admin] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:16:35.266867 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG dbcounter [-] [80244] Writing DB stats keystone:SELECT=100 {{(pid=80244) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:16:35.308074 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f04f9c7-4448-47a1-9e87-6e3fe165a745 None admin] Validating token access rules against request {{(pid=80245) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:16:35.356699 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-202dd92d-6d63-4f79-beae-b73db3f5e817 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80245) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:16:35.357610 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-202dd92d-6d63-4f79-beae-b73db3f5e817 None admin] REQUEST_METHOD: `GET` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:16:35.358022 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-202dd92d-6d63-4f79-beae-b73db3f5e817 None admin] SCRIPT_NAME: `/identity` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:16:35.358403 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-202dd92d-6d63-4f79-beae-b73db3f5e817 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:16:35.359907 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-202dd92d-6d63-4f79-beae-b73db3f5e817 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:16:35.361514 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-202dd92d-6d63-4f79-beae-b73db3f5e817 None admin] RBAC: Authorization granted {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:16:35.363319 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: [pid: 80245|app: 0|req: 137/275] 10.222.0.47 () {56 vars in 1108 bytes} [Thu Jul 21 08:16:35 2022] GET /identity/v3/domains/default => generated 199 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 08:16:35.372373 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-202dd92d-6d63-4f79-beae-b73db3f5e817 None admin] Authenticating user token {{(pid=80245) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:16:35.373499 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1826198e-c6b5-4f88-86c4-2b4c3ad82789 None admin] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:16:35.414748 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG dbcounter [-] [80245] Writing DB stats keystone:SELECT=100 {{(pid=80245) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:16:35.416869 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fec2d867-246c-4a32-acdb-b1e221ff4981 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:16:35.417889 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fec2d867-246c-4a32-acdb-b1e221ff4981 None admin] REQUEST_METHOD: `GET` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:16:35.418252 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fec2d867-246c-4a32-acdb-b1e221ff4981 None admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:16:35.418751 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fec2d867-246c-4a32-acdb-b1e221ff4981 None admin] PATH_INFO: `/v3/roles` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:16:35.419667 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fec2d867-246c-4a32-acdb-b1e221ff4981 None admin] RBAC: Adding query filter params (name=member) {{(pid=80244) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 08:16:35.420074 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fec2d867-246c-4a32-acdb-b1e221ff4981 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:16:35.421979 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fec2d867-246c-4a32-acdb-b1e221ff4981 None admin] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:16:35.431495 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 139/276] 10.222.0.47 () {56 vars in 1101 bytes} [Thu Jul 21 08:16:35 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 229 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 21 08:16:35.437471 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fec2d867-246c-4a32-acdb-b1e221ff4981 None admin] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:16:35.540663 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG dbcounter [-] [80244] Writing DB stats keystone:SELECT=100 {{(pid=80244) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:16:35.557006 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fec2d867-246c-4a32-acdb-b1e221ff4981 None admin] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:16:35.560941 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-202dd92d-6d63-4f79-beae-b73db3f5e817 None admin] Validating token access rules against request {{(pid=80245) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:16:35.603048 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be128c97-940f-4157-aa50-6b30fde4d503 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:16:35.604318 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be128c97-940f-4157-aa50-6b30fde4d503 None admin] REQUEST_METHOD: `GET` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:16:35.604801 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be128c97-940f-4157-aa50-6b30fde4d503 None admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:16:35.605152 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb81be89-72ea-4098-95df-18f442317d9f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80245) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:16:35.605526 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be128c97-940f-4157-aa50-6b30fde4d503 None admin] PATH_INFO: `/v3/users/cbcab1143ff64eb89e9759367d95e744` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:16:35.605839 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb81be89-72ea-4098-95df-18f442317d9f None admin] REQUEST_METHOD: `POST` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:16:35.606126 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb81be89-72ea-4098-95df-18f442317d9f None admin] SCRIPT_NAME: `/identity` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:16:35.606403 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb81be89-72ea-4098-95df-18f442317d9f None admin] PATH_INFO: `/v3/users` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:16:35.607172 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb81be89-72ea-4098-95df-18f442317d9f None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:16:35.608637 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb81be89-72ea-4098-95df-18f442317d9f None admin] RBAC: Authorization granted {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:16:35.626090 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be128c97-940f-4157-aa50-6b30fde4d503 None admin] RBAC: Authorizing `identity:get_user(user_id=cbcab1143ff64eb89e9759367d95e744)` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:16:35.628765 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be128c97-940f-4157-aa50-6b30fde4d503 None admin] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:16:35.635347 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 140/277] 10.222.0.47 () {56 vars in 1177 bytes} [Thu Jul 21 08:16:35 2022] GET /identity/v3/users/cbcab1143ff64eb89e9759367d95e744 => generated 293 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 21 08:16:35.639181 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: [pid: 80245|app: 0|req: 138/278] 10.222.0.47 () {60 vars in 1132 bytes} [Thu Jul 21 08:16:35 2022] POST /identity/v3/users => generated 293 bytes in 267 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 08:16:35.639965 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be128c97-940f-4157-aa50-6b30fde4d503 None admin] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:16:35.758828 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be128c97-940f-4157-aa50-6b30fde4d503 None admin] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:16:35.795683 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cf2d18a-0a9b-4084-bc10-7c7e2013977b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:16:35.796362 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cf2d18a-0a9b-4084-bc10-7c7e2013977b None admin] REQUEST_METHOD: `GET` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:16:35.796649 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cf2d18a-0a9b-4084-bc10-7c7e2013977b None admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:16:35.796925 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cf2d18a-0a9b-4084-bc10-7c7e2013977b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:16:35.797845 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cf2d18a-0a9b-4084-bc10-7c7e2013977b None admin] RBAC: Adding query filter params (role.id=3a074738221e4601b5205715ba87d240, user.id=cbcab1143ff64eb89e9759367d95e744, scope.system=all) {{(pid=80244) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 08:16:35.798166 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cf2d18a-0a9b-4084-bc10-7c7e2013977b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:16:35.799760 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cf2d18a-0a9b-4084-bc10-7c7e2013977b None admin] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:16:35.810818 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 141/279] 10.222.0.47 () {56 vars in 1308 bytes} [Thu Jul 21 08:16:35 2022] GET /identity/v3/role_assignments?user.id=cbcab1143ff64eb89e9759367d95e744&scope.system=all&role.id=3a074738221e4601b5205715ba87d240 => generated 226 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 08:16:35.914117 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccc37c50-af45-4797-a475-ea1fbb318f0a None None] REQUEST_METHOD: `GET` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:16:35.914632 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccc37c50-af45-4797-a475-ea1fbb318f0a None None] SCRIPT_NAME: `/identity` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:16:35.915070 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccc37c50-af45-4797-a475-ea1fbb318f0a None None] PATH_INFO: `/` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:16:35.916855 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: [pid: 80245|app: 0|req: 139/280] 10.222.0.47 () {54 vars in 926 bytes} [Thu Jul 21 08:16:35 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 08:16:35.922604 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-215964f9-1f71-488a-8536-e1c5a6e1c086 None None] REQUEST_METHOD: `POST` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:16:35.922894 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-215964f9-1f71-488a-8536-e1c5a6e1c086 None None] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:16:35.923165 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-215964f9-1f71-488a-8536-e1c5a6e1c086 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:16:35.941892 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.auth.core [None req-215964f9-1f71-488a-8536-e1c5a6e1c086 None None] MFA Rules not processed for user `2dce40b8d74c41ec94cc36e6ff05d412`. Rule list: `[]` (Enabled: `True`). {{(pid=80244) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 08:16:35.943089 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.fernet_utils [None req-215964f9-1f71-488a-8536-e1c5a6e1c086 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80244) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 08:16:35.955794 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG dbcounter [-] [80244] Writing DB stats keystone:SELECT=100 {{(pid=80244) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:16:35.997081 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f328d241-43ce-4108-a99f-70aa81aa4613 None None] REQUEST_METHOD: `GET` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:16:35.997369 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f328d241-43ce-4108-a99f-70aa81aa4613 None None] SCRIPT_NAME: `/identity` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:16:35.997644 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f328d241-43ce-4108-a99f-70aa81aa4613 None None] PATH_INFO: `/` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:16:35.999120 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: [pid: 80245|app: 0|req: 140/281] 10.222.0.47 () {54 vars in 926 bytes} [Thu Jul 21 08:16:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 21 08:16:36.004701 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97ea7890-1e33-4b6f-8c61-3f48bd63bd8e None None] REQUEST_METHOD: `POST` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:16:36.004989 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97ea7890-1e33-4b6f-8c61-3f48bd63bd8e None None] SCRIPT_NAME: `/identity` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:16:36.005268 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97ea7890-1e33-4b6f-8c61-3f48bd63bd8e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:16:36.025772 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 142/282] 10.222.0.47 () {58 vars in 1023 bytes} [Thu Jul 21 08:16:35 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 21 08:16:36.026824 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.auth.core [None req-97ea7890-1e33-4b6f-8c61-3f48bd63bd8e None None] MFA Rules not processed for user `2dce40b8d74c41ec94cc36e6ff05d412`. Rule list: `[]` (Enabled: `True`). {{(pid=80245) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 08:16:36.028665 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.fernet_utils [None req-97ea7890-1e33-4b6f-8c61-3f48bd63bd8e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80245) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 08:16:36.056850 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-215964f9-1f71-488a-8536-e1c5a6e1c086 None None] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:16:36.121007 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: [pid: 80245|app: 0|req: 141/283] 10.222.0.47 () {58 vars in 1023 bytes} [Thu Jul 21 08:16:36 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 21 08:16:36.145493 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97ea7890-1e33-4b6f-8c61-3f48bd63bd8e None None] Authenticating user token {{(pid=80245) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:16:36.156740 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG dbcounter [-] [80245] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80245) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:16:36.204903 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-215964f9-1f71-488a-8536-e1c5a6e1c086 None None] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:16:36.262254 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4843c4e3-16dc-4096-938e-a5d3fb07685d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:16:36.263063 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4843c4e3-16dc-4096-938e-a5d3fb07685d None admin] REQUEST_METHOD: `GET` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:16:36.263360 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4843c4e3-16dc-4096-938e-a5d3fb07685d None admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:16:36.263634 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4843c4e3-16dc-4096-938e-a5d3fb07685d None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:16:36.286612 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4843c4e3-16dc-4096-938e-a5d3fb07685d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 21 08:16:36.295500 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application [None req-4843c4e3-16dc-4096-938e-a5d3fb07685d None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jul 21 08:16:36.295500 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 08:16:36.295500 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 08:16:36.295500 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 08:16:36.295500 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 08:16:36.295500 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 08:16:36.295500 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 08:16:36.295500 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 08:16:36.295500 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 08:16:36.295500 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 08:16:36.295500 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 08:16:36.295500 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 08:16:36.295500 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 21 08:16:36.295500 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 21 08:16:36.295500 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 21 08:16:36.295500 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 21 08:16:36.296408 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 21 08:16:36.296408 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 21 08:16:36.296408 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 21 08:16:36.296408 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 21 08:16:36.296408 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 08:16:36.296408 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 08:16:36.296408 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 08:16:36.296408 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 08:16:36.296408 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 08:16:36.296408 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 08:16:36.296408 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 08:16:36.296408 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application with Lock( Jul 21 08:16:36.296408 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 08:16:36.296408 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self._enter() Jul 21 08:16:36.296408 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 08:16:36.296408 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 08:16:36.297344 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 08:16:36.297344 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self.creator() Jul 21 08:16:36.297344 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 08:16:36.297344 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application created_value = creator( Jul 21 08:16:36.297344 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 21 08:16:36.297344 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 21 08:16:36.297344 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 21 08:16:36.297344 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 21 08:16:36.297344 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 21 08:16:36.297344 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 21 08:16:36.297344 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jul 21 08:16:36.297344 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application Jul 21 08:16:36.297344 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 143/284] 10.222.0.47 () {56 vars in 1099 bytes} [Thu Jul 21 08:16:36 2022] GET /identity/v3/roles/reader => generated 84 bytes in 242 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 21 08:16:36.304019 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4843c4e3-16dc-4096-938e-a5d3fb07685d None admin] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:16:36.308251 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG dbcounter [-] [80244] Writing DB stats keystone:SELECT=100 {{(pid=80244) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:16:36.373540 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97ea7890-1e33-4b6f-8c61-3f48bd63bd8e None None] Validating token access rules against request {{(pid=80245) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:16:36.432617 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf2b641a-f459-4551-bcf2-e0fa2513187d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80245) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:16:36.433897 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf2b641a-f459-4551-bcf2-e0fa2513187d None admin] REQUEST_METHOD: `GET` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:16:36.434193 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf2b641a-f459-4551-bcf2-e0fa2513187d None admin] SCRIPT_NAME: `/identity` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:16:36.434470 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf2b641a-f459-4551-bcf2-e0fa2513187d None admin] PATH_INFO: `/v3/roles/member` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:16:36.446793 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf2b641a-f459-4551-bcf2-e0fa2513187d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 21 08:16:36.455749 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application [None req-cf2b641a-f459-4551-bcf2-e0fa2513187d None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 21 08:16:36.455749 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 08:16:36.455749 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 08:16:36.455749 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 08:16:36.455749 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 08:16:36.455749 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 08:16:36.455749 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 08:16:36.455749 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 08:16:36.455749 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 08:16:36.455749 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 08:16:36.455749 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 08:16:36.455749 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 08:16:36.455749 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 21 08:16:36.455749 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 21 08:16:36.455749 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 21 08:16:36.455749 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 21 08:16:36.456605 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 21 08:16:36.456605 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 21 08:16:36.456605 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 21 08:16:36.456605 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 21 08:16:36.456605 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 08:16:36.456605 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 08:16:36.456605 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 08:16:36.456605 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 08:16:36.456605 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 08:16:36.456605 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 08:16:36.456605 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 08:16:36.456605 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application with Lock( Jul 21 08:16:36.456605 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 08:16:36.456605 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application return self._enter() Jul 21 08:16:36.456605 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 08:16:36.456605 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 08:16:36.457373 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 08:16:36.457373 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application return self.creator() Jul 21 08:16:36.457373 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 08:16:36.457373 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application created_value = creator( Jul 21 08:16:36.457373 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 21 08:16:36.457373 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 21 08:16:36.457373 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 21 08:16:36.457373 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 21 08:16:36.457373 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 21 08:16:36.457373 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 21 08:16:36.457373 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 21 08:16:36.457373 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application Jul 21 08:16:36.457373 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: [pid: 80245|app: 0|req: 142/285] 10.222.0.47 () {56 vars in 1099 bytes} [Thu Jul 21 08:16:36 2022] GET /identity/v3/roles/member => generated 84 bytes in 312 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 21 08:16:36.464706 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf2b641a-f459-4551-bcf2-e0fa2513187d None admin] Authenticating user token {{(pid=80245) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:16:36.496119 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4843c4e3-16dc-4096-938e-a5d3fb07685d None admin] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:16:36.573135 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG dbcounter [-] [80245] Writing DB stats keystone:SELECT=100 {{(pid=80245) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:16:36.605408 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf2b641a-f459-4551-bcf2-e0fa2513187d None admin] Validating token access rules against request {{(pid=80245) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:16:36.616624 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb6d5a9a-4c6d-4710-8f6d-8cc87b199a3d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:16:36.617314 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb6d5a9a-4c6d-4710-8f6d-8cc87b199a3d None admin] REQUEST_METHOD: `GET` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:16:36.617595 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb6d5a9a-4c6d-4710-8f6d-8cc87b199a3d None admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:16:36.617868 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb6d5a9a-4c6d-4710-8f6d-8cc87b199a3d None admin] PATH_INFO: `/v3/roles` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:16:36.618657 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb6d5a9a-4c6d-4710-8f6d-8cc87b199a3d None admin] RBAC: Adding query filter params (name=reader) {{(pid=80244) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 08:16:36.618917 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb6d5a9a-4c6d-4710-8f6d-8cc87b199a3d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:16:36.620330 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb6d5a9a-4c6d-4710-8f6d-8cc87b199a3d None admin] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:16:36.645937 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 144/286] 10.222.0.47 () {56 vars in 1101 bytes} [Thu Jul 21 08:16:36 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 338 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 08:16:36.648094 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6794f9f-6319-4f31-a518-91ecb41cb3c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80245) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:16:36.649021 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6794f9f-6319-4f31-a518-91ecb41cb3c0 None admin] REQUEST_METHOD: `GET` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:16:36.649555 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6794f9f-6319-4f31-a518-91ecb41cb3c0 None admin] SCRIPT_NAME: `/identity` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:16:36.650048 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6794f9f-6319-4f31-a518-91ecb41cb3c0 None admin] PATH_INFO: `/v3/roles` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:16:36.651066 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6794f9f-6319-4f31-a518-91ecb41cb3c0 None admin] RBAC: Adding query filter params (name=member) {{(pid=80245) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 08:16:36.651649 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6794f9f-6319-4f31-a518-91ecb41cb3c0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:16:36.653238 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6794f9f-6319-4f31-a518-91ecb41cb3c0 None admin] RBAC: Authorization granted {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:16:36.664164 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: [pid: 80245|app: 0|req: 143/287] 10.222.0.47 () {56 vars in 1101 bytes} [Thu Jul 21 08:16:36 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 08:16:36.664416 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb6d5a9a-4c6d-4710-8f6d-8cc87b199a3d None admin] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:16:36.670043 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6794f9f-6319-4f31-a518-91ecb41cb3c0 None admin] Authenticating user token {{(pid=80245) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:16:36.801868 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG dbcounter [-] [80244] Writing DB stats keystone:SELECT=100 {{(pid=80244) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:16:36.842757 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb6d5a9a-4c6d-4710-8f6d-8cc87b199a3d None admin] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:16:36.879967 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73d0774c-3a1a-4047-a3fc-e27f7ada99dc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:16:36.881446 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73d0774c-3a1a-4047-a3fc-e27f7ada99dc None admin] REQUEST_METHOD: `GET` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:16:36.881734 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73d0774c-3a1a-4047-a3fc-e27f7ada99dc None admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:16:36.882011 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73d0774c-3a1a-4047-a3fc-e27f7ada99dc None admin] PATH_INFO: `/v3/users/578bf32a7bab47b3829483e36917158c` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:16:36.883491 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73d0774c-3a1a-4047-a3fc-e27f7ada99dc None admin] RBAC: Authorizing `identity:get_user(user_id=578bf32a7bab47b3829483e36917158c)` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:16:36.885169 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73d0774c-3a1a-4047-a3fc-e27f7ada99dc None admin] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:16:36.887123 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 145/288] 10.222.0.47 () {56 vars in 1177 bytes} [Thu Jul 21 08:16:36 2022] GET /identity/v3/users/578bf32a7bab47b3829483e36917158c => generated 297 bytes in 223 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 08:16:36.895630 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73d0774c-3a1a-4047-a3fc-e27f7ada99dc None admin] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:16:36.905228 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6794f9f-6319-4f31-a518-91ecb41cb3c0 None admin] Validating token access rules against request {{(pid=80245) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:16:36.962705 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG dbcounter [-] [80245] Writing DB stats keystone:SELECT=100 {{(pid=80245) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:16:36.976372 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65fc46d6-b707-4913-a895-b95471c930e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80245) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:16:36.977256 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65fc46d6-b707-4913-a895-b95471c930e1 None admin] REQUEST_METHOD: `GET` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:16:36.977534 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65fc46d6-b707-4913-a895-b95471c930e1 None admin] SCRIPT_NAME: `/identity` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:16:36.977802 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65fc46d6-b707-4913-a895-b95471c930e1 None admin] PATH_INFO: `/v3/users/5a245876268b4f3797a25e891079b053` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:16:36.979596 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65fc46d6-b707-4913-a895-b95471c930e1 None admin] RBAC: Authorizing `identity:get_user(user_id=5a245876268b4f3797a25e891079b053)` {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:16:36.981099 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65fc46d6-b707-4913-a895-b95471c930e1 None admin] RBAC: Authorization granted {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:16:36.982943 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: [pid: 80245|app: 0|req: 144/289] 10.222.0.47 () {56 vars in 1177 bytes} [Thu Jul 21 08:16:36 2022] GET /identity/v3/users/5a245876268b4f3797a25e891079b053 => generated 297 bytes in 314 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 08:16:36.989791 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65fc46d6-b707-4913-a895-b95471c930e1 None admin] Authenticating user token {{(pid=80245) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:16:37.109541 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73d0774c-3a1a-4047-a3fc-e27f7ada99dc None admin] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:16:37.126652 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65fc46d6-b707-4913-a895-b95471c930e1 None admin] Validating token access rules against request {{(pid=80245) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:16:37.171637 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf43e8a2-3101-470b-875f-ce318dad6b79 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80245) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:16:37.172369 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf43e8a2-3101-470b-875f-ce318dad6b79 None admin] REQUEST_METHOD: `GET` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:16:37.172648 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf43e8a2-3101-470b-875f-ce318dad6b79 None admin] SCRIPT_NAME: `/identity` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:16:37.172920 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf43e8a2-3101-470b-875f-ce318dad6b79 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:16:37.187439 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-919fea06-7bb9-4291-b2f6-2d73d0cb93d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:16:37.188345 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-919fea06-7bb9-4291-b2f6-2d73d0cb93d3 None admin] REQUEST_METHOD: `GET` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:16:37.188622 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-919fea06-7bb9-4291-b2f6-2d73d0cb93d3 None admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:16:37.188894 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-919fea06-7bb9-4291-b2f6-2d73d0cb93d3 None admin] PATH_INFO: `/v3/projects/d4fa4cba54f348b68df92b37fb5fd5d1` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:16:37.190082 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-919fea06-7bb9-4291-b2f6-2d73d0cb93d3 None admin] RBAC: Authorizing `identity:get_project(project_id=d4fa4cba54f348b68df92b37fb5fd5d1)` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:16:37.191564 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-919fea06-7bb9-4291-b2f6-2d73d0cb93d3 None admin] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:16:37.194164 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 146/290] 10.222.0.47 () {56 vars in 1186 bytes} [Thu Jul 21 08:16:36 2022] GET /identity/v3/projects/d4fa4cba54f348b68df92b37fb5fd5d1 => generated 302 bytes in 299 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 08:16:37.199807 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-919fea06-7bb9-4291-b2f6-2d73d0cb93d3 None admin] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:16:37.204243 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG dbcounter [-] [80244] Writing DB stats keystone:SELECT=100 {{(pid=80244) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:16:37.244999 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG dbcounter [-] [80245] Writing DB stats keystone:SELECT=100 {{(pid=80245) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:16:37.247591 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf43e8a2-3101-470b-875f-ce318dad6b79 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:16:37.249118 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf43e8a2-3101-470b-875f-ce318dad6b79 None admin] RBAC: Authorization granted {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:16:37.311582 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-919fea06-7bb9-4291-b2f6-2d73d0cb93d3 None admin] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:16:37.347190 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68ff6d49-f4c5-490d-8097-9c371353e036 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:16:37.348264 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68ff6d49-f4c5-490d-8097-9c371353e036 None admin] REQUEST_METHOD: `PUT` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:16:37.348547 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68ff6d49-f4c5-490d-8097-9c371353e036 None admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:16:37.348821 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68ff6d49-f4c5-490d-8097-9c371353e036 None admin] PATH_INFO: `/v3/projects/d4fa4cba54f348b68df92b37fb5fd5d1/users/578bf32a7bab47b3829483e36917158c/roles/9e5171884e9f401ab2337b74db49546e` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:16:37.351317 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68ff6d49-f4c5-490d-8097-9c371353e036 None admin] RBAC: Authorizing `identity:create_grant(project_id=d4fa4cba54f348b68df92b37fb5fd5d1, user_id=578bf32a7bab47b3829483e36917158c, role_id=9e5171884e9f401ab2337b74db49546e)` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:16:37.353398 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68ff6d49-f4c5-490d-8097-9c371353e036 None admin] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:16:37.369383 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 147/291] 10.222.0.47 () {58 vars in 1439 bytes} [Thu Jul 21 08:16:37 2022] PUT /identity/v3/projects/d4fa4cba54f348b68df92b37fb5fd5d1/users/578bf32a7bab47b3829483e36917158c/roles/9e5171884e9f401ab2337b74db49546e => generated 0 bytes in 169 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 21 08:16:37.444841 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: [pid: 80245|app: 0|req: 145/292] 10.222.0.47 () {58 vars in 1282 bytes} [Thu Jul 21 08:16:36 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 455 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 21 08:16:37.450798 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68ff6d49-f4c5-490d-8097-9c371353e036 None admin] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:16:37.567751 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG dbcounter [-] [80244] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80244) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:16:37.586405 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68ff6d49-f4c5-490d-8097-9c371353e036 None admin] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:16:37.623421 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a490f9f-2ed8-4b6e-9ee3-cda768489444 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:16:37.624207 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a490f9f-2ed8-4b6e-9ee3-cda768489444 None admin] REQUEST_METHOD: `GET` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:16:37.624488 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a490f9f-2ed8-4b6e-9ee3-cda768489444 None admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:16:37.624761 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a490f9f-2ed8-4b6e-9ee3-cda768489444 None admin] PATH_INFO: `/v3/projects/d4fa4cba54f348b68df92b37fb5fd5d1` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:16:37.625942 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a490f9f-2ed8-4b6e-9ee3-cda768489444 None admin] RBAC: Authorizing `identity:get_project(project_id=d4fa4cba54f348b68df92b37fb5fd5d1)` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:16:37.627422 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a490f9f-2ed8-4b6e-9ee3-cda768489444 None admin] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:16:37.629195 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 148/293] 10.222.0.47 () {56 vars in 1186 bytes} [Thu Jul 21 08:16:37 2022] GET /identity/v3/projects/d4fa4cba54f348b68df92b37fb5fd5d1 => generated 302 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 08:16:37.635409 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf43e8a2-3101-470b-875f-ce318dad6b79 None admin] Authenticating user token {{(pid=80245) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:16:37.742206 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG dbcounter [-] [80245] Writing DB stats keystone:SELECT=100 {{(pid=80245) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:16:37.745822 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf43e8a2-3101-470b-875f-ce318dad6b79 None admin] Validating token access rules against request {{(pid=80245) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:16:37.789385 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-227b43c2-63e5-40a0-9e18-24a2bdfa1251 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80245) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:16:37.790049 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-227b43c2-63e5-40a0-9e18-24a2bdfa1251 None admin] REQUEST_METHOD: `GET` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:16:37.790331 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-227b43c2-63e5-40a0-9e18-24a2bdfa1251 None admin] SCRIPT_NAME: `/identity` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:16:37.790602 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-227b43c2-63e5-40a0-9e18-24a2bdfa1251 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:16:37.791486 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-227b43c2-63e5-40a0-9e18-24a2bdfa1251 None admin] RBAC: Adding query filter params (role.id=3a074738221e4601b5205715ba87d240, scope.project.id=d4fa4cba54f348b68df92b37fb5fd5d1, user.id=5a245876268b4f3797a25e891079b053) {{(pid=80245) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 08:16:37.791796 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-227b43c2-63e5-40a0-9e18-24a2bdfa1251 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:16:37.793226 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-227b43c2-63e5-40a0-9e18-24a2bdfa1251 None admin] RBAC: Authorization granted {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:16:37.804094 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: [pid: 80245|app: 0|req: 146/294] 10.222.0.47 () {56 vars in 1374 bytes} [Thu Jul 21 08:16:37 2022] GET /identity/v3/role_assignments?user.id=5a245876268b4f3797a25e891079b053&scope.project.id=d4fa4cba54f348b68df92b37fb5fd5d1&role.id=3a074738221e4601b5205715ba87d240 => generated 608 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 08:16:38.086417 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-848fc1a9-d305-46d2-96d0-86c4bf218d14 None None] REQUEST_METHOD: `GET` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:16:38.086865 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-848fc1a9-d305-46d2-96d0-86c4bf218d14 None None] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:16:38.087257 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-848fc1a9-d305-46d2-96d0-86c4bf218d14 None None] PATH_INFO: `/` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:16:38.088982 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 149/295] 10.222.0.47 () {54 vars in 926 bytes} [Thu Jul 21 08:16:38 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 08:16:38.094537 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d00ebf1b-d342-4553-bd44-5b7da48ace7e None None] REQUEST_METHOD: `POST` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:16:38.094825 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d00ebf1b-d342-4553-bd44-5b7da48ace7e None None] SCRIPT_NAME: `/identity` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:16:38.095102 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d00ebf1b-d342-4553-bd44-5b7da48ace7e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:16:38.118335 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.auth.core [None req-d00ebf1b-d342-4553-bd44-5b7da48ace7e None None] MFA Rules not processed for user `2dce40b8d74c41ec94cc36e6ff05d412`. Rule list: `[]` (Enabled: `True`). {{(pid=80245) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 08:16:38.119557 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.fernet_utils [None req-d00ebf1b-d342-4553-bd44-5b7da48ace7e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80245) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 08:16:38.197676 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: [pid: 80245|app: 0|req: 147/296] 10.222.0.47 () {58 vars in 1023 bytes} [Thu Jul 21 08:16:38 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 21 08:16:38.221006 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-848fc1a9-d305-46d2-96d0-86c4bf218d14 None None] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:16:38.311105 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d4382fe-dd09-4acb-afb0-a7fd78db61e5 None None] REQUEST_METHOD: `GET` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:16:38.311471 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d4382fe-dd09-4acb-afb0-a7fd78db61e5 None None] SCRIPT_NAME: `/identity` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:16:38.311702 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d4382fe-dd09-4acb-afb0-a7fd78db61e5 None None] PATH_INFO: `/` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:16:38.313219 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: [pid: 80245|app: 0|req: 148/297] 10.222.0.47 () {54 vars in 926 bytes} [Thu Jul 21 08:16:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 21 08:16:38.321194 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3e8793d-8481-458d-9590-5ad300ae77cb None None] REQUEST_METHOD: `POST` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:16:38.321486 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3e8793d-8481-458d-9590-5ad300ae77cb None None] SCRIPT_NAME: `/identity` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:16:38.321753 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3e8793d-8481-458d-9590-5ad300ae77cb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:16:38.343433 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.auth.core [None req-e3e8793d-8481-458d-9590-5ad300ae77cb None None] MFA Rules not processed for user `2dce40b8d74c41ec94cc36e6ff05d412`. Rule list: `[]` (Enabled: `True`). {{(pid=80245) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 08:16:38.344527 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.fernet_utils [None req-e3e8793d-8481-458d-9590-5ad300ae77cb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80245) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 08:16:38.345681 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-848fc1a9-d305-46d2-96d0-86c4bf218d14 None None] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:16:38.387404 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG dbcounter [-] [80244] Writing DB stats keystone:SELECT=100 {{(pid=80244) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:16:38.398581 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21082508-8e33-4a26-bfda-b82fff84c893 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:16:38.399401 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21082508-8e33-4a26-bfda-b82fff84c893 None admin] REQUEST_METHOD: `GET` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:16:38.399683 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21082508-8e33-4a26-bfda-b82fff84c893 None admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:16:38.399955 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21082508-8e33-4a26-bfda-b82fff84c893 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:16:38.415681 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21082508-8e33-4a26-bfda-b82fff84c893 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 21 08:16:38.425409 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application [None req-21082508-8e33-4a26-bfda-b82fff84c893 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 21 08:16:38.425409 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 08:16:38.425409 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 08:16:38.425409 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 08:16:38.425409 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 08:16:38.425409 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 08:16:38.425409 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 08:16:38.425409 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 08:16:38.425409 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 08:16:38.425409 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 08:16:38.425409 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 08:16:38.425409 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 08:16:38.425409 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 21 08:16:38.425409 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 21 08:16:38.425409 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 21 08:16:38.425409 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 21 08:16:38.426324 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 21 08:16:38.426324 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 21 08:16:38.426324 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 21 08:16:38.426324 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 21 08:16:38.426324 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 08:16:38.426324 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 08:16:38.426324 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 08:16:38.426324 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 08:16:38.426324 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 08:16:38.426324 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 08:16:38.426324 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 08:16:38.426324 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application with Lock( Jul 21 08:16:38.426324 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 08:16:38.426324 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self._enter() Jul 21 08:16:38.426324 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 08:16:38.426324 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 08:16:38.427091 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 08:16:38.427091 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self.creator() Jul 21 08:16:38.427091 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 08:16:38.427091 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application created_value = creator( Jul 21 08:16:38.427091 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 21 08:16:38.427091 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 21 08:16:38.427091 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 21 08:16:38.427091 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 21 08:16:38.427091 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 21 08:16:38.427091 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 21 08:16:38.427091 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 21 08:16:38.427091 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application Jul 21 08:16:38.427091 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 150/298] 10.222.0.47 () {56 vars in 1099 bytes} [Thu Jul 21 08:16:38 2022] GET /identity/v3/roles/member => generated 84 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 08:16:38.433328 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21082508-8e33-4a26-bfda-b82fff84c893 None admin] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:16:38.462987 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: [pid: 80245|app: 0|req: 149/299] 10.222.0.47 () {58 vars in 1023 bytes} [Thu Jul 21 08:16:38 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 143 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 21 08:16:38.475558 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-718f6de1-757b-4dbc-988f-2ecb60694238 None None] REQUEST_METHOD: `GET` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:16:38.476193 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-718f6de1-757b-4dbc-988f-2ecb60694238 None None] SCRIPT_NAME: `/identity` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:16:38.476477 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-718f6de1-757b-4dbc-988f-2ecb60694238 None None] PATH_INFO: `/` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:16:38.477980 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: [pid: 80245|app: 0|req: 150/300] 10.222.0.47 () {54 vars in 926 bytes} [Thu Jul 21 08:16:38 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 21 08:16:38.486048 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55217da1-7675-452a-9963-876082518343 None None] REQUEST_METHOD: `POST` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:16:38.486048 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55217da1-7675-452a-9963-876082518343 None None] SCRIPT_NAME: `/identity` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:16:38.486048 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55217da1-7675-452a-9963-876082518343 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:16:38.506160 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.auth.core [None req-55217da1-7675-452a-9963-876082518343 None None] MFA Rules not processed for user `2dce40b8d74c41ec94cc36e6ff05d412`. Rule list: `[]` (Enabled: `True`). {{(pid=80245) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 08:16:38.507850 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.fernet_utils [None req-55217da1-7675-452a-9963-876082518343 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80245) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 08:16:38.526233 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG dbcounter [-] [80245] Writing DB stats keystone:SELECT=100 {{(pid=80245) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:16:38.552181 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21082508-8e33-4a26-bfda-b82fff84c893 None admin] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:16:38.591400 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: [pid: 80245|app: 0|req: 151/301] 10.222.0.47 () {58 vars in 1023 bytes} [Thu Jul 21 08:16:38 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 21 08:16:38.591400 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55217da1-7675-452a-9963-876082518343 None None] Authenticating user token {{(pid=80245) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:16:38.591660 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b097500-50e7-41a0-a583-53bc4cc22943 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:16:38.591660 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b097500-50e7-41a0-a583-53bc4cc22943 None admin] REQUEST_METHOD: `GET` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:16:38.591660 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b097500-50e7-41a0-a583-53bc4cc22943 None admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:16:38.591660 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b097500-50e7-41a0-a583-53bc4cc22943 None admin] PATH_INFO: `/v3/roles` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:16:38.591992 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b097500-50e7-41a0-a583-53bc4cc22943 None admin] RBAC: Adding query filter params (name=member) {{(pid=80244) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 08:16:38.591992 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b097500-50e7-41a0-a583-53bc4cc22943 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:16:38.591992 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b097500-50e7-41a0-a583-53bc4cc22943 None admin] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:16:38.599913 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 151/302] 10.222.0.47 () {56 vars in 1101 bytes} [Thu Jul 21 08:16:38 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 08:16:38.606874 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b097500-50e7-41a0-a583-53bc4cc22943 None admin] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:16:38.667911 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG dbcounter [-] [80244] Writing DB stats keystone:SELECT=100 {{(pid=80244) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:16:38.699534 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55217da1-7675-452a-9963-876082518343 None None] Validating token access rules against request {{(pid=80245) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:16:38.717111 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b097500-50e7-41a0-a583-53bc4cc22943 None admin] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:16:38.736373 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04494a54-d31d-48cc-b065-5900cb999bf0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80245) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:16:38.737265 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04494a54-d31d-48cc-b065-5900cb999bf0 None admin] REQUEST_METHOD: `GET` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:16:38.737554 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04494a54-d31d-48cc-b065-5900cb999bf0 None admin] SCRIPT_NAME: `/identity` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:16:38.737830 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04494a54-d31d-48cc-b065-5900cb999bf0 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:16:38.739008 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04494a54-d31d-48cc-b065-5900cb999bf0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:16:38.740537 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04494a54-d31d-48cc-b065-5900cb999bf0 None admin] RBAC: Authorization granted {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:16:38.742156 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: [pid: 80245|app: 0|req: 152/303] 10.222.0.47 () {56 vars in 1108 bytes} [Thu Jul 21 08:16:38 2022] GET /identity/v3/domains/default => generated 199 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 08:16:38.742929 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04494a54-d31d-48cc-b065-5900cb999bf0 None admin] Authenticating user token {{(pid=80245) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:16:38.759180 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-684c92e7-7a59-4ca1-969f-57a0ae25fa94 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:16:38.760092 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-684c92e7-7a59-4ca1-969f-57a0ae25fa94 None admin] REQUEST_METHOD: `GET` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:16:38.760381 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-684c92e7-7a59-4ca1-969f-57a0ae25fa94 None admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:16:38.760638 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-684c92e7-7a59-4ca1-969f-57a0ae25fa94 None admin] PATH_INFO: `/v3/users/cbcab1143ff64eb89e9759367d95e744` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:16:38.762081 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-684c92e7-7a59-4ca1-969f-57a0ae25fa94 None admin] RBAC: Authorizing `identity:get_user(user_id=cbcab1143ff64eb89e9759367d95e744)` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:16:38.763549 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-684c92e7-7a59-4ca1-969f-57a0ae25fa94 None admin] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:16:38.769798 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 152/304] 10.222.0.47 () {56 vars in 1177 bytes} [Thu Jul 21 08:16:38 2022] GET /identity/v3/users/cbcab1143ff64eb89e9759367d95e744 => generated 293 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 08:16:38.769798 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-684c92e7-7a59-4ca1-969f-57a0ae25fa94 None admin] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:16:38.770508 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG dbcounter [-] [80245] Writing DB stats keystone:SELECT=100 {{(pid=80245) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:16:38.854091 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04494a54-d31d-48cc-b065-5900cb999bf0 None admin] Validating token access rules against request {{(pid=80245) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:16:38.880811 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-684c92e7-7a59-4ca1-969f-57a0ae25fa94 None admin] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:16:38.890964 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-649ce2d1-5fbc-4e10-b5e5-08339141a4ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80245) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:16:38.891804 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-649ce2d1-5fbc-4e10-b5e5-08339141a4ba None admin] REQUEST_METHOD: `GET` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:16:38.892095 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-649ce2d1-5fbc-4e10-b5e5-08339141a4ba None admin] SCRIPT_NAME: `/identity` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:16:38.892362 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-649ce2d1-5fbc-4e10-b5e5-08339141a4ba None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:16:38.904708 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG dbcounter [-] [80244] Writing DB stats keystone:SELECT=100 {{(pid=80244) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:16:38.905028 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-649ce2d1-5fbc-4e10-b5e5-08339141a4ba None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 21 08:16:38.913665 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application [None req-649ce2d1-5fbc-4e10-b5e5-08339141a4ba None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jul 21 08:16:38.913665 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 08:16:38.913665 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 08:16:38.913665 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 08:16:38.913665 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 08:16:38.913665 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 08:16:38.913665 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 08:16:38.913665 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 08:16:38.913665 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 08:16:38.913665 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 08:16:38.913665 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 08:16:38.913665 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 08:16:38.913665 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 21 08:16:38.913665 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 21 08:16:38.913665 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 21 08:16:38.913665 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 21 08:16:38.914556 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 21 08:16:38.914556 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 21 08:16:38.914556 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 21 08:16:38.914556 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 21 08:16:38.914556 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 08:16:38.914556 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 08:16:38.914556 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 08:16:38.914556 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 08:16:38.914556 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 08:16:38.914556 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 08:16:38.914556 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 08:16:38.914556 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application with Lock( Jul 21 08:16:38.914556 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 08:16:38.914556 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application return self._enter() Jul 21 08:16:38.914556 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 08:16:38.914556 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 08:16:38.920244 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 08:16:38.920244 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application return self.creator() Jul 21 08:16:38.920244 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 08:16:38.920244 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application created_value = creator( Jul 21 08:16:38.920244 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 21 08:16:38.920244 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 21 08:16:38.920244 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 21 08:16:38.920244 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 21 08:16:38.920244 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 21 08:16:38.920244 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 21 08:16:38.920244 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jul 21 08:16:38.920244 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application Jul 21 08:16:38.920244 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: [pid: 80245|app: 0|req: 153/305] 10.222.0.47 () {56 vars in 1099 bytes} [Thu Jul 21 08:16:38 2022] GET /identity/v3/roles/reader => generated 84 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 08:16:38.920244 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-649ce2d1-5fbc-4e10-b5e5-08339141a4ba None admin] Authenticating user token {{(pid=80245) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:16:38.928224 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c631e7ac-aea6-44d0-b1a1-e7bb3e962354 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:16:38.928984 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c631e7ac-aea6-44d0-b1a1-e7bb3e962354 None admin] REQUEST_METHOD: `POST` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:16:38.929344 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c631e7ac-aea6-44d0-b1a1-e7bb3e962354 None admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:16:38.929693 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c631e7ac-aea6-44d0-b1a1-e7bb3e962354 None admin] PATH_INFO: `/v3/groups` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:16:38.930481 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c631e7ac-aea6-44d0-b1a1-e7bb3e962354 None admin] RBAC: Authorizing `identity:create_group()` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:16:38.932112 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c631e7ac-aea6-44d0-b1a1-e7bb3e962354 None admin] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:16:38.948993 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 153/306] 10.222.0.47 () {60 vars in 1134 bytes} [Thu Jul 21 08:16:38 2022] POST /identity/v3/groups => generated 229 bytes in 183 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 08:16:38.949805 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c631e7ac-aea6-44d0-b1a1-e7bb3e962354 None admin] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:16:39.032975 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG dbcounter [-] [80245] Writing DB stats keystone:SELECT=100 {{(pid=80245) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:16:39.056011 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-649ce2d1-5fbc-4e10-b5e5-08339141a4ba None admin] Validating token access rules against request {{(pid=80245) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:16:39.069932 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c631e7ac-aea6-44d0-b1a1-e7bb3e962354 None admin] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:16:39.095491 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a95ecfe5-7a1f-4e18-af43-fe6e548d1c5d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80245) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:16:39.096754 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a95ecfe5-7a1f-4e18-af43-fe6e548d1c5d None admin] REQUEST_METHOD: `PUT` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:16:39.097268 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a95ecfe5-7a1f-4e18-af43-fe6e548d1c5d None admin] SCRIPT_NAME: `/identity` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:16:39.097777 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a95ecfe5-7a1f-4e18-af43-fe6e548d1c5d None admin] PATH_INFO: `/v3/system/users/cbcab1143ff64eb89e9759367d95e744/roles/3a074738221e4601b5205715ba87d240` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:16:39.100115 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a95ecfe5-7a1f-4e18-af43-fe6e548d1c5d None admin] RBAC: Authorizing `identity:create_system_grant_for_user(user_id=cbcab1143ff64eb89e9759367d95e744, role_id=3a074738221e4601b5205715ba87d240)` {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:16:39.101797 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a95ecfe5-7a1f-4e18-af43-fe6e548d1c5d None admin] RBAC: Authorization granted {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:16:39.110815 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ba19b3f-6ae9-457a-bdce-3f26743f7604 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:16:39.111526 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ba19b3f-6ae9-457a-bdce-3f26743f7604 None admin] REQUEST_METHOD: `GET` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:16:39.111868 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ba19b3f-6ae9-457a-bdce-3f26743f7604 None admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:16:39.112155 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ba19b3f-6ae9-457a-bdce-3f26743f7604 None admin] PATH_INFO: `/v3/roles` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:16:39.112690 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: [pid: 80245|app: 0|req: 154/307] 10.222.0.47 () {58 vars in 1334 bytes} [Thu Jul 21 08:16:38 2022] PUT /identity/v3/system/users/cbcab1143ff64eb89e9759367d95e744/roles/3a074738221e4601b5205715ba87d240 => generated 0 bytes in 197 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 08:16:39.113292 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ba19b3f-6ae9-457a-bdce-3f26743f7604 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80244) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 08:16:39.113724 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ba19b3f-6ae9-457a-bdce-3f26743f7604 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:16:39.115769 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ba19b3f-6ae9-457a-bdce-3f26743f7604 None admin] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:16:39.127109 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 154/308] 10.222.0.47 () {56 vars in 1101 bytes} [Thu Jul 21 08:16:38 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 08:16:39.134229 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a95ecfe5-7a1f-4e18-af43-fe6e548d1c5d None admin] Authenticating user token {{(pid=80245) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:16:39.262647 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a95ecfe5-7a1f-4e18-af43-fe6e548d1c5d None admin] Validating token access rules against request {{(pid=80245) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:16:39.302574 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-376a100a-7b65-4d94-9b51-e207bc4e64d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80245) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:16:39.303551 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-376a100a-7b65-4d94-9b51-e207bc4e64d3 None admin] REQUEST_METHOD: `GET` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:16:39.303834 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-376a100a-7b65-4d94-9b51-e207bc4e64d3 None admin] SCRIPT_NAME: `/identity` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:16:39.304106 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-376a100a-7b65-4d94-9b51-e207bc4e64d3 None admin] PATH_INFO: `/v3/users/83c8bdcbe1da4ac9b82d63af18922290` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:16:39.319451 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-376a100a-7b65-4d94-9b51-e207bc4e64d3 None admin] RBAC: Authorizing `identity:get_user(user_id=83c8bdcbe1da4ac9b82d63af18922290)` {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:16:39.321056 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-376a100a-7b65-4d94-9b51-e207bc4e64d3 None admin] RBAC: Authorization granted {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:16:39.323150 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: [pid: 80245|app: 0|req: 155/309] 10.222.0.47 () {56 vars in 1177 bytes} [Thu Jul 21 08:16:39 2022] GET /identity/v3/users/83c8bdcbe1da4ac9b82d63af18922290 => generated 293 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 08:16:39.328977 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ba19b3f-6ae9-457a-bdce-3f26743f7604 None admin] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:16:39.384412 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG dbcounter [-] [80244] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80244) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:16:39.447905 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ba19b3f-6ae9-457a-bdce-3f26743f7604 None admin] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:16:39.485037 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3a2a87d-3ffb-4f78-b38a-aba0e055691e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:16:39.485715 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3a2a87d-3ffb-4f78-b38a-aba0e055691e None admin] REQUEST_METHOD: `GET` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:16:39.486318 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3a2a87d-3ffb-4f78-b38a-aba0e055691e None admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:16:39.486595 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3a2a87d-3ffb-4f78-b38a-aba0e055691e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:16:39.487478 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3a2a87d-3ffb-4f78-b38a-aba0e055691e None admin] RBAC: Adding query filter params (role.id=9e5171884e9f401ab2337b74db49546e, user.id=83c8bdcbe1da4ac9b82d63af18922290, scope.system=all) {{(pid=80244) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 08:16:39.487793 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3a2a87d-3ffb-4f78-b38a-aba0e055691e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:16:39.489285 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3a2a87d-3ffb-4f78-b38a-aba0e055691e None admin] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:16:39.499490 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 155/310] 10.222.0.47 () {56 vars in 1308 bytes} [Thu Jul 21 08:16:39 2022] GET /identity/v3/role_assignments?user.id=83c8bdcbe1da4ac9b82d63af18922290&scope.system=all&role.id=9e5171884e9f401ab2337b74db49546e => generated 226 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 08:16:39.722260 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f37179b-58a5-42b5-b674-23a728277d6c None None] REQUEST_METHOD: `GET` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:16:39.722641 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f37179b-58a5-42b5-b674-23a728277d6c None None] SCRIPT_NAME: `/identity` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:16:39.722855 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f37179b-58a5-42b5-b674-23a728277d6c None None] PATH_INFO: `/` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:16:39.724969 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: [pid: 80245|app: 0|req: 156/311] 10.222.0.47 () {54 vars in 926 bytes} [Thu Jul 21 08:16:39 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 08:16:39.730517 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6542b339-d012-461e-9fa8-64998ec2617b None None] REQUEST_METHOD: `POST` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:16:39.730813 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6542b339-d012-461e-9fa8-64998ec2617b None None] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:16:39.731096 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6542b339-d012-461e-9fa8-64998ec2617b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:16:39.749676 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.auth.core [None req-6542b339-d012-461e-9fa8-64998ec2617b None None] MFA Rules not processed for user `2dce40b8d74c41ec94cc36e6ff05d412`. Rule list: `[]` (Enabled: `True`). {{(pid=80244) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 08:16:39.751153 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.fernet_utils [None req-6542b339-d012-461e-9fa8-64998ec2617b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80244) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 08:16:39.824437 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 156/312] 10.222.0.47 () {58 vars in 1023 bytes} [Thu Jul 21 08:16:39 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 21 08:16:39.847759 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f37179b-58a5-42b5-b674-23a728277d6c None None] Authenticating user token {{(pid=80245) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:16:39.851666 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG dbcounter [-] [80245] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80245) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:16:39.963410 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f37179b-58a5-42b5-b674-23a728277d6c None None] Validating token access rules against request {{(pid=80245) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:16:40.000362 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c92107f3-6a32-4dc3-9af5-388fa2f5a3d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80245) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:16:40.001135 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c92107f3-6a32-4dc3-9af5-388fa2f5a3d5 None admin] REQUEST_METHOD: `GET` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:16:40.001462 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c92107f3-6a32-4dc3-9af5-388fa2f5a3d5 None admin] SCRIPT_NAME: `/identity` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:16:40.001745 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c92107f3-6a32-4dc3-9af5-388fa2f5a3d5 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:16:40.012705 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c92107f3-6a32-4dc3-9af5-388fa2f5a3d5 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 21 08:16:40.020465 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application [None req-c92107f3-6a32-4dc3-9af5-388fa2f5a3d5 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jul 21 08:16:40.020465 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 08:16:40.020465 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 08:16:40.020465 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 08:16:40.020465 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 08:16:40.020465 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 08:16:40.020465 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 08:16:40.020465 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 08:16:40.020465 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 08:16:40.020465 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 08:16:40.020465 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 08:16:40.020465 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 08:16:40.020465 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 21 08:16:40.020465 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 21 08:16:40.020465 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 21 08:16:40.020465 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 21 08:16:40.021326 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 21 08:16:40.021326 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 21 08:16:40.021326 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 21 08:16:40.021326 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 21 08:16:40.021326 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 08:16:40.021326 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 08:16:40.021326 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 08:16:40.021326 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 08:16:40.021326 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 08:16:40.021326 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 08:16:40.021326 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 08:16:40.021326 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application with Lock( Jul 21 08:16:40.021326 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 08:16:40.021326 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application return self._enter() Jul 21 08:16:40.021326 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 08:16:40.021326 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 08:16:40.022099 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 08:16:40.022099 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application return self.creator() Jul 21 08:16:40.022099 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 08:16:40.022099 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application created_value = creator( Jul 21 08:16:40.022099 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 21 08:16:40.022099 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 21 08:16:40.022099 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 21 08:16:40.022099 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 21 08:16:40.022099 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 21 08:16:40.022099 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 21 08:16:40.022099 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jul 21 08:16:40.022099 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application Jul 21 08:16:40.022099 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: [pid: 80245|app: 0|req: 157/313] 10.222.0.47 () {56 vars in 1099 bytes} [Thu Jul 21 08:16:39 2022] GET /identity/v3/roles/reader => generated 84 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 08:16:40.028860 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6542b339-d012-461e-9fa8-64998ec2617b None None] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:16:40.071737 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG dbcounter [-] [80244] Writing DB stats keystone:SELECT=100 {{(pid=80244) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:16:40.141449 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6542b339-d012-461e-9fa8-64998ec2617b None None] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:16:40.177916 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56b7485a-7de3-4bf7-8b62-d0accecc33c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:16:40.178604 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56b7485a-7de3-4bf7-8b62-d0accecc33c9 None admin] REQUEST_METHOD: `GET` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:16:40.178890 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56b7485a-7de3-4bf7-8b62-d0accecc33c9 None admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:16:40.179165 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56b7485a-7de3-4bf7-8b62-d0accecc33c9 None admin] PATH_INFO: `/v3/roles` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:16:40.179953 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56b7485a-7de3-4bf7-8b62-d0accecc33c9 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80244) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 08:16:40.180261 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56b7485a-7de3-4bf7-8b62-d0accecc33c9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:16:40.181654 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56b7485a-7de3-4bf7-8b62-d0accecc33c9 None admin] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:16:40.195481 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 157/314] 10.222.0.47 () {56 vars in 1101 bytes} [Thu Jul 21 08:16:40 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 08:16:40.197287 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c92107f3-6a32-4dc3-9af5-388fa2f5a3d5 None admin] Authenticating user token {{(pid=80245) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:16:40.266355 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG dbcounter [-] [80245] Writing DB stats keystone:SELECT=100 {{(pid=80245) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:16:40.318065 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c92107f3-6a32-4dc3-9af5-388fa2f5a3d5 None admin] Validating token access rules against request {{(pid=80245) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:16:40.353464 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eda88e2-8c83-48f1-bef8-82eff04980b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80245) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:16:40.354343 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eda88e2-8c83-48f1-bef8-82eff04980b6 None admin] REQUEST_METHOD: `GET` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:16:40.354676 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eda88e2-8c83-48f1-bef8-82eff04980b6 None admin] SCRIPT_NAME: `/identity` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:16:40.354956 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eda88e2-8c83-48f1-bef8-82eff04980b6 None admin] PATH_INFO: `/v3/users/578bf32a7bab47b3829483e36917158c` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:16:40.356159 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5eda88e2-8c83-48f1-bef8-82eff04980b6 None admin] RBAC: Authorizing `identity:get_user(user_id=578bf32a7bab47b3829483e36917158c)` {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:16:40.357606 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5eda88e2-8c83-48f1-bef8-82eff04980b6 None admin] RBAC: Authorization granted {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:16:40.359454 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: [pid: 80245|app: 0|req: 158/315] 10.222.0.47 () {56 vars in 1177 bytes} [Thu Jul 21 08:16:40 2022] GET /identity/v3/users/578bf32a7bab47b3829483e36917158c => generated 297 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 08:16:40.364734 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56b7485a-7de3-4bf7-8b62-d0accecc33c9 None admin] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:16:40.462180 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG dbcounter [-] [80244] Writing DB stats keystone:SELECT=100 {{(pid=80244) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:16:40.465595 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56b7485a-7de3-4bf7-8b62-d0accecc33c9 None admin] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:16:40.515052 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2d1ae12-4b01-41ae-871a-c71c8adc3a5c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:16:40.516045 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2d1ae12-4b01-41ae-871a-c71c8adc3a5c None admin] REQUEST_METHOD: `GET` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:16:40.516590 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2d1ae12-4b01-41ae-871a-c71c8adc3a5c None admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:16:40.517086 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2d1ae12-4b01-41ae-871a-c71c8adc3a5c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:16:40.587883 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2d1ae12-4b01-41ae-871a-c71c8adc3a5c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:16:40.589754 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2d1ae12-4b01-41ae-871a-c71c8adc3a5c None admin] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:16:40.694315 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 158/316] 10.222.0.47 () {58 vars in 1282 bytes} [Thu Jul 21 08:16:40 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 330 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 21 08:16:40.694989 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG dbcounter [-] [80244] Writing DB stats keystone:SELECT=100 {{(pid=80244) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:16:40.699703 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eda88e2-8c83-48f1-bef8-82eff04980b6 None admin] Authenticating user token {{(pid=80245) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:16:40.811723 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eda88e2-8c83-48f1-bef8-82eff04980b6 None admin] Validating token access rules against request {{(pid=80245) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:16:40.841933 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG dbcounter [-] [80245] Writing DB stats keystone:SELECT=100 {{(pid=80245) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:16:40.848162 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e51b446-e4a0-430a-bd1c-8f8840b83bb1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80245) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:16:40.849034 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e51b446-e4a0-430a-bd1c-8f8840b83bb1 None admin] REQUEST_METHOD: `GET` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:16:40.849347 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e51b446-e4a0-430a-bd1c-8f8840b83bb1 None admin] SCRIPT_NAME: `/identity` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:16:40.849622 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e51b446-e4a0-430a-bd1c-8f8840b83bb1 None admin] PATH_INFO: `/v3/projects/d4fa4cba54f348b68df92b37fb5fd5d1` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:16:40.850800 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e51b446-e4a0-430a-bd1c-8f8840b83bb1 None admin] RBAC: Authorizing `identity:get_project(project_id=d4fa4cba54f348b68df92b37fb5fd5d1)` {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:16:40.852273 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e51b446-e4a0-430a-bd1c-8f8840b83bb1 None admin] RBAC: Authorization granted {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:16:40.854249 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: [pid: 80245|app: 0|req: 159/317] 10.222.0.47 () {56 vars in 1186 bytes} [Thu Jul 21 08:16:40 2022] GET /identity/v3/projects/d4fa4cba54f348b68df92b37fb5fd5d1 => generated 302 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 08:16:40.860039 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2d1ae12-4b01-41ae-871a-c71c8adc3a5c None admin] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:16:40.961766 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2d1ae12-4b01-41ae-871a-c71c8adc3a5c None admin] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:16:41.001218 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50da3b7b-8760-4d47-a5ed-aee946c28c98 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:16:41.001880 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50da3b7b-8760-4d47-a5ed-aee946c28c98 None admin] REQUEST_METHOD: `GET` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:16:41.002163 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50da3b7b-8760-4d47-a5ed-aee946c28c98 None admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:16:41.002439 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50da3b7b-8760-4d47-a5ed-aee946c28c98 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:16:41.003281 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50da3b7b-8760-4d47-a5ed-aee946c28c98 None admin] RBAC: Adding query filter params (role.id=9e5171884e9f401ab2337b74db49546e, scope.project.id=d4fa4cba54f348b68df92b37fb5fd5d1, user.id=578bf32a7bab47b3829483e36917158c) {{(pid=80244) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 08:16:41.003625 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50da3b7b-8760-4d47-a5ed-aee946c28c98 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:16:41.005064 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50da3b7b-8760-4d47-a5ed-aee946c28c98 None admin] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:16:41.012245 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 159/318] 10.222.0.47 () {56 vars in 1374 bytes} [Thu Jul 21 08:16:40 2022] GET /identity/v3/role_assignments?user.id=578bf32a7bab47b3829483e36917158c&scope.project.id=d4fa4cba54f348b68df92b37fb5fd5d1&role.id=9e5171884e9f401ab2337b74db49546e => generated 608 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 08:16:41.176391 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a955c3bc-4516-4a4b-8004-368549c6f990 None None] REQUEST_METHOD: `GET` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:16:41.176764 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a955c3bc-4516-4a4b-8004-368549c6f990 None None] SCRIPT_NAME: `/identity` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:16:41.176962 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a955c3bc-4516-4a4b-8004-368549c6f990 None None] PATH_INFO: `/` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:16:41.179140 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: [pid: 80245|app: 0|req: 160/319] 10.222.0.47 () {54 vars in 926 bytes} [Thu Jul 21 08:16:41 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 08:16:41.184236 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8c3abf0-ad4a-4606-833f-c122f89d1c3c None None] REQUEST_METHOD: `POST` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:16:41.184524 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8c3abf0-ad4a-4606-833f-c122f89d1c3c None None] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:16:41.184799 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8c3abf0-ad4a-4606-833f-c122f89d1c3c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:16:41.203246 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.auth.core [None req-f8c3abf0-ad4a-4606-833f-c122f89d1c3c None None] MFA Rules not processed for user `2dce40b8d74c41ec94cc36e6ff05d412`. Rule list: `[]` (Enabled: `True`). {{(pid=80244) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 08:16:41.204515 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.fernet_utils [None req-f8c3abf0-ad4a-4606-833f-c122f89d1c3c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80244) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 08:16:41.265522 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6fe512b-3235-4bbb-9add-209890be98b2 None None] REQUEST_METHOD: `GET` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:16:41.265826 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6fe512b-3235-4bbb-9add-209890be98b2 None None] SCRIPT_NAME: `/identity` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:16:41.266075 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6fe512b-3235-4bbb-9add-209890be98b2 None None] PATH_INFO: `/` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:16:41.266866 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG dbcounter [-] [80244] Writing DB stats keystone:SELECT=100 {{(pid=80244) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:16:41.267896 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: [pid: 80245|app: 0|req: 161/320] 10.222.0.47 () {54 vars in 926 bytes} [Thu Jul 21 08:16:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 21 08:16:41.273502 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27c94bfc-3ff1-4eab-8ebb-36c38d9623e4 None None] REQUEST_METHOD: `POST` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:16:41.273672 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27c94bfc-3ff1-4eab-8ebb-36c38d9623e4 None None] SCRIPT_NAME: `/identity` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:16:41.273935 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27c94bfc-3ff1-4eab-8ebb-36c38d9623e4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:16:41.280999 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 160/321] 10.222.0.47 () {58 vars in 1023 bytes} [Thu Jul 21 08:16:41 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 21 08:16:41.292331 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.auth.core [None req-27c94bfc-3ff1-4eab-8ebb-36c38d9623e4 None None] MFA Rules not processed for user `2dce40b8d74c41ec94cc36e6ff05d412`. Rule list: `[]` (Enabled: `True`). {{(pid=80245) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 08:16:41.293529 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.fernet_utils [None req-27c94bfc-3ff1-4eab-8ebb-36c38d9623e4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80245) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 08:16:41.304852 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8c3abf0-ad4a-4606-833f-c122f89d1c3c None None] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:16:41.371467 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: [pid: 80245|app: 0|req: 162/322] 10.222.0.47 () {58 vars in 1023 bytes} [Thu Jul 21 08:16:41 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 21 08:16:41.398575 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27c94bfc-3ff1-4eab-8ebb-36c38d9623e4 None None] Authenticating user token {{(pid=80245) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:16:41.416388 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8c3abf0-ad4a-4606-833f-c122f89d1c3c None None] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:16:41.453529 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55fe4491-5ff0-4b59-b1d3-7f5aea6a5636 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:16:41.454275 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55fe4491-5ff0-4b59-b1d3-7f5aea6a5636 None admin] REQUEST_METHOD: `GET` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:16:41.454559 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55fe4491-5ff0-4b59-b1d3-7f5aea6a5636 None admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:16:41.454835 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55fe4491-5ff0-4b59-b1d3-7f5aea6a5636 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:16:41.456170 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55fe4491-5ff0-4b59-b1d3-7f5aea6a5636 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:16:41.457693 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55fe4491-5ff0-4b59-b1d3-7f5aea6a5636 None admin] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:16:41.459902 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 161/323] 10.222.0.47 () {56 vars in 1108 bytes} [Thu Jul 21 08:16:41 2022] GET /identity/v3/domains/default => generated 199 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 21 08:16:41.465343 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55fe4491-5ff0-4b59-b1d3-7f5aea6a5636 None admin] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:16:41.526141 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27c94bfc-3ff1-4eab-8ebb-36c38d9623e4 None None] Validating token access rules against request {{(pid=80245) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:16:41.530817 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG dbcounter [-] [80244] Writing DB stats keystone:SELECT=100 {{(pid=80244) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:16:41.555553 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG dbcounter [-] [80245] Writing DB stats keystone:SELECT=100 {{(pid=80245) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:16:41.565855 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35b4e404-1e4d-4b71-9984-4116018a299a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80245) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:16:41.566665 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35b4e404-1e4d-4b71-9984-4116018a299a None admin] REQUEST_METHOD: `GET` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:16:41.567018 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35b4e404-1e4d-4b71-9984-4116018a299a None admin] SCRIPT_NAME: `/identity` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:16:41.567247 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35b4e404-1e4d-4b71-9984-4116018a299a None admin] PATH_INFO: `/v3/roles/member` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:16:41.577579 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55fe4491-5ff0-4b59-b1d3-7f5aea6a5636 None admin] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:16:41.578384 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35b4e404-1e4d-4b71-9984-4116018a299a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 21 08:16:41.586363 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application [None req-35b4e404-1e4d-4b71-9984-4116018a299a None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 21 08:16:41.586363 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 08:16:41.586363 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 08:16:41.586363 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 08:16:41.586363 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 08:16:41.586363 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 08:16:41.586363 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 08:16:41.586363 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 08:16:41.586363 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 08:16:41.586363 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 08:16:41.586363 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 08:16:41.586363 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 08:16:41.586363 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 21 08:16:41.586363 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 21 08:16:41.586363 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 21 08:16:41.586363 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 21 08:16:41.587214 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 21 08:16:41.587214 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 21 08:16:41.587214 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 21 08:16:41.587214 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 21 08:16:41.587214 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 08:16:41.587214 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 08:16:41.587214 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 08:16:41.587214 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 08:16:41.587214 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 08:16:41.587214 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 08:16:41.587214 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 08:16:41.587214 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application with Lock( Jul 21 08:16:41.587214 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 08:16:41.587214 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application return self._enter() Jul 21 08:16:41.587214 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 08:16:41.587214 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 08:16:41.588039 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 08:16:41.588039 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application return self.creator() Jul 21 08:16:41.588039 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 08:16:41.588039 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application created_value = creator( Jul 21 08:16:41.588039 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 21 08:16:41.588039 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 21 08:16:41.588039 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 21 08:16:41.588039 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 21 08:16:41.588039 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 21 08:16:41.588039 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 21 08:16:41.588039 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 21 08:16:41.588039 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application Jul 21 08:16:41.588039 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: [pid: 80245|app: 0|req: 163/324] 10.222.0.47 () {56 vars in 1099 bytes} [Thu Jul 21 08:16:41 2022] GET /identity/v3/roles/member => generated 84 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 21 08:16:41.594074 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35b4e404-1e4d-4b71-9984-4116018a299a None admin] Authenticating user token {{(pid=80245) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:16:41.617748 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a398cde-a808-4b0e-a2f5-0362c3215e6b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:16:41.618418 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a398cde-a808-4b0e-a2f5-0362c3215e6b None admin] REQUEST_METHOD: `POST` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:16:41.618711 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a398cde-a808-4b0e-a2f5-0362c3215e6b None admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:16:41.618994 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a398cde-a808-4b0e-a2f5-0362c3215e6b None admin] PATH_INFO: `/v3/groups` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:16:41.619718 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a398cde-a808-4b0e-a2f5-0362c3215e6b None admin] RBAC: Authorizing `identity:create_group()` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:16:41.621192 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a398cde-a808-4b0e-a2f5-0362c3215e6b None admin] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:16:41.647354 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 162/325] 10.222.0.47 () {60 vars in 1134 bytes} [Thu Jul 21 08:16:41 2022] POST /identity/v3/groups => generated 226 bytes in 174 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 21 08:16:41.700266 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a90a3fc-da0f-4348-a2a7-fe349e101c2e None None] REQUEST_METHOD: `GET` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:16:41.700560 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a90a3fc-da0f-4348-a2a7-fe349e101c2e None None] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:16:41.700879 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a90a3fc-da0f-4348-a2a7-fe349e101c2e None None] PATH_INFO: `/` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:16:41.702327 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 163/326] 10.222.0.47 () {54 vars in 926 bytes} [Thu Jul 21 08:16:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 08:16:41.707544 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35b4e404-1e4d-4b71-9984-4116018a299a None admin] Validating token access rules against request {{(pid=80245) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:16:41.707997 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4616149-94b0-4557-af8d-ac8012c1416a None None] REQUEST_METHOD: `POST` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:16:41.708295 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4616149-94b0-4557-af8d-ac8012c1416a None None] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:16:41.709171 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4616149-94b0-4557-af8d-ac8012c1416a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:16:41.732369 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.auth.core [None req-c4616149-94b0-4557-af8d-ac8012c1416a None None] MFA Rules not processed for user `2dce40b8d74c41ec94cc36e6ff05d412`. Rule list: `[]` (Enabled: `True`). {{(pid=80244) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 08:16:41.733728 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.fernet_utils [None req-c4616149-94b0-4557-af8d-ac8012c1416a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80244) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 08:16:41.754069 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d151b0c-6ce2-445a-ac32-e0c298798161 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80245) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:16:41.754768 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d151b0c-6ce2-445a-ac32-e0c298798161 None admin] REQUEST_METHOD: `GET` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:16:41.755053 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d151b0c-6ce2-445a-ac32-e0c298798161 None admin] SCRIPT_NAME: `/identity` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:16:41.755352 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d151b0c-6ce2-445a-ac32-e0c298798161 None admin] PATH_INFO: `/v3/roles` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:16:41.756263 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d151b0c-6ce2-445a-ac32-e0c298798161 None admin] RBAC: Adding query filter params (name=member) {{(pid=80245) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 08:16:41.756582 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d151b0c-6ce2-445a-ac32-e0c298798161 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:16:41.758815 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d151b0c-6ce2-445a-ac32-e0c298798161 None admin] RBAC: Authorization granted {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:16:41.768389 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: [pid: 80245|app: 0|req: 164/327] 10.222.0.47 () {56 vars in 1101 bytes} [Thu Jul 21 08:16:41 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 08:16:41.774447 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d151b0c-6ce2-445a-ac32-e0c298798161 None admin] Authenticating user token {{(pid=80245) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:16:41.809054 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 164/328] 10.222.0.47 () {58 vars in 1023 bytes} [Thu Jul 21 08:16:41 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 21 08:16:41.830810 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG dbcounter [-] [80245] Writing DB stats keystone:SELECT=100 {{(pid=80245) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:16:41.830989 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4616149-94b0-4557-af8d-ac8012c1416a None None] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:16:41.888227 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d151b0c-6ce2-445a-ac32-e0c298798161 None admin] Validating token access rules against request {{(pid=80245) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:16:41.897418 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG dbcounter [-] [80244] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80244) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:16:41.939094 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be7f59cd-dd06-47e5-a6dc-e615a827bcc1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80245) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:16:41.940025 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be7f59cd-dd06-47e5-a6dc-e615a827bcc1 None admin] REQUEST_METHOD: `GET` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:16:41.940309 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be7f59cd-dd06-47e5-a6dc-e615a827bcc1 None admin] SCRIPT_NAME: `/identity` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:16:41.940588 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be7f59cd-dd06-47e5-a6dc-e615a827bcc1 None admin] PATH_INFO: `/v3/users/cbcab1143ff64eb89e9759367d95e744` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:16:41.941803 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be7f59cd-dd06-47e5-a6dc-e615a827bcc1 None admin] RBAC: Authorizing `identity:get_user(user_id=cbcab1143ff64eb89e9759367d95e744)` {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:16:41.943969 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be7f59cd-dd06-47e5-a6dc-e615a827bcc1 None admin] RBAC: Authorization granted {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:16:41.945638 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: [pid: 80245|app: 0|req: 165/329] 10.222.0.47 () {56 vars in 1177 bytes} [Thu Jul 21 08:16:41 2022] GET /identity/v3/users/cbcab1143ff64eb89e9759367d95e744 => generated 293 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 21 08:16:41.951513 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be7f59cd-dd06-47e5-a6dc-e615a827bcc1 None admin] Authenticating user token {{(pid=80245) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:16:41.966148 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4616149-94b0-4557-af8d-ac8012c1416a None None] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:16:42.011784 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ff60910-1d60-4fd7-97c8-82715d10aebc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:16:42.012754 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ff60910-1d60-4fd7-97c8-82715d10aebc None admin] REQUEST_METHOD: `GET` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:16:42.013038 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ff60910-1d60-4fd7-97c8-82715d10aebc None admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:16:42.013869 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ff60910-1d60-4fd7-97c8-82715d10aebc None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:16:42.026699 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ff60910-1d60-4fd7-97c8-82715d10aebc None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 21 08:16:42.034599 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application [None req-2ff60910-1d60-4fd7-97c8-82715d10aebc None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jul 21 08:16:42.034599 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 08:16:42.034599 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 08:16:42.034599 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 08:16:42.034599 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 08:16:42.034599 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 08:16:42.034599 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 08:16:42.034599 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 08:16:42.034599 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 08:16:42.034599 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 08:16:42.034599 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 08:16:42.034599 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 08:16:42.034599 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 21 08:16:42.034599 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 21 08:16:42.034599 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 21 08:16:42.034599 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 21 08:16:42.035519 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 21 08:16:42.035519 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 21 08:16:42.035519 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 21 08:16:42.035519 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 21 08:16:42.035519 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 08:16:42.035519 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 08:16:42.035519 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 08:16:42.035519 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 08:16:42.035519 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 08:16:42.035519 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 08:16:42.035519 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 08:16:42.035519 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application with Lock( Jul 21 08:16:42.035519 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 08:16:42.035519 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self._enter() Jul 21 08:16:42.035519 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 08:16:42.035519 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 08:16:42.036293 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 08:16:42.036293 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self.creator() Jul 21 08:16:42.036293 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 08:16:42.036293 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application created_value = creator( Jul 21 08:16:42.036293 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 21 08:16:42.036293 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 21 08:16:42.036293 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 21 08:16:42.036293 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 21 08:16:42.036293 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 21 08:16:42.036293 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 21 08:16:42.036293 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jul 21 08:16:42.036293 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application Jul 21 08:16:42.036293 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 165/330] 10.222.0.47 () {56 vars in 1099 bytes} [Thu Jul 21 08:16:41 2022] GET /identity/v3/roles/reader => generated 84 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 21 08:16:42.044248 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ff60910-1d60-4fd7-97c8-82715d10aebc None admin] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:16:42.068900 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be7f59cd-dd06-47e5-a6dc-e615a827bcc1 None admin] Validating token access rules against request {{(pid=80245) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:16:42.089918 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG dbcounter [-] [80245] Writing DB stats keystone:SELECT=100 {{(pid=80245) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:16:42.116057 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90ad1ab5-c50a-40a6-90cb-16ce7b674319 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80245) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:16:42.117015 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90ad1ab5-c50a-40a6-90cb-16ce7b674319 None admin] REQUEST_METHOD: `GET` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:16:42.117503 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90ad1ab5-c50a-40a6-90cb-16ce7b674319 None admin] SCRIPT_NAME: `/identity` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:16:42.117955 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90ad1ab5-c50a-40a6-90cb-16ce7b674319 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:16:42.118968 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90ad1ab5-c50a-40a6-90cb-16ce7b674319 None admin] RBAC: Adding query filter params (role.id=3a074738221e4601b5205715ba87d240, user.id=cbcab1143ff64eb89e9759367d95e744, scope.system=all) {{(pid=80245) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 08:16:42.119482 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90ad1ab5-c50a-40a6-90cb-16ce7b674319 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:16:42.121104 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90ad1ab5-c50a-40a6-90cb-16ce7b674319 None admin] RBAC: Authorization granted {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:16:42.136448 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: [pid: 80245|app: 0|req: 166/331] 10.222.0.47 () {56 vars in 1308 bytes} [Thu Jul 21 08:16:41 2022] GET /identity/v3/role_assignments?user.id=cbcab1143ff64eb89e9759367d95e744&scope.system=all&role.id=3a074738221e4601b5205715ba87d240 => generated 510 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 08:16:42.190804 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ff60910-1d60-4fd7-97c8-82715d10aebc None admin] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:16:42.204454 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG dbcounter [-] [80244] Writing DB stats keystone:SELECT=100 {{(pid=80244) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:16:42.235473 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61985351-5f8c-4cf1-aff0-d1f4df67b9b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:16:42.236470 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61985351-5f8c-4cf1-aff0-d1f4df67b9b7 None admin] REQUEST_METHOD: `GET` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:16:42.236950 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61985351-5f8c-4cf1-aff0-d1f4df67b9b7 None admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:16:42.237397 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61985351-5f8c-4cf1-aff0-d1f4df67b9b7 None admin] PATH_INFO: `/v3/roles` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:16:42.238342 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61985351-5f8c-4cf1-aff0-d1f4df67b9b7 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80244) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 08:16:42.238827 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61985351-5f8c-4cf1-aff0-d1f4df67b9b7 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:16:42.241166 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61985351-5f8c-4cf1-aff0-d1f4df67b9b7 None admin] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:16:42.251366 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 166/332] 10.222.0.47 () {56 vars in 1101 bytes} [Thu Jul 21 08:16:42 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 08:16:42.257464 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90ad1ab5-c50a-40a6-90cb-16ce7b674319 None admin] Authenticating user token {{(pid=80245) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:16:42.375287 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90ad1ab5-c50a-40a6-90cb-16ce7b674319 None admin] Validating token access rules against request {{(pid=80245) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:16:42.417023 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec667724-7296-4c5e-a313-1c45fe80b058 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80245) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:16:42.417930 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec667724-7296-4c5e-a313-1c45fe80b058 None admin] REQUEST_METHOD: `GET` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:16:42.418221 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec667724-7296-4c5e-a313-1c45fe80b058 None admin] SCRIPT_NAME: `/identity` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:16:42.418503 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec667724-7296-4c5e-a313-1c45fe80b058 None admin] PATH_INFO: `/v3/users/83c8bdcbe1da4ac9b82d63af18922290` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:16:42.420337 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec667724-7296-4c5e-a313-1c45fe80b058 None admin] RBAC: Authorizing `identity:get_user(user_id=83c8bdcbe1da4ac9b82d63af18922290)` {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:16:42.421839 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec667724-7296-4c5e-a313-1c45fe80b058 None admin] RBAC: Authorization granted {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:16:42.423853 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: [pid: 80245|app: 0|req: 167/333] 10.222.0.47 () {56 vars in 1177 bytes} [Thu Jul 21 08:16:42 2022] GET /identity/v3/users/83c8bdcbe1da4ac9b82d63af18922290 => generated 293 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 08:16:42.429661 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61985351-5f8c-4cf1-aff0-d1f4df67b9b7 None admin] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:16:42.559628 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61985351-5f8c-4cf1-aff0-d1f4df67b9b7 None admin] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:16:42.594750 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f742dfa3-5acb-40e1-946e-594bcd2aaad8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:16:42.595710 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f742dfa3-5acb-40e1-946e-594bcd2aaad8 None admin] REQUEST_METHOD: `PUT` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:16:42.596001 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f742dfa3-5acb-40e1-946e-594bcd2aaad8 None admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:16:42.596277 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f742dfa3-5acb-40e1-946e-594bcd2aaad8 None admin] PATH_INFO: `/v3/system/users/83c8bdcbe1da4ac9b82d63af18922290/roles/9e5171884e9f401ab2337b74db49546e` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:16:42.597767 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f742dfa3-5acb-40e1-946e-594bcd2aaad8 None admin] RBAC: Authorizing `identity:create_system_grant_for_user(user_id=83c8bdcbe1da4ac9b82d63af18922290, role_id=9e5171884e9f401ab2337b74db49546e)` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:16:42.599782 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f742dfa3-5acb-40e1-946e-594bcd2aaad8 None admin] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:16:42.612219 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 167/334] 10.222.0.47 () {58 vars in 1334 bytes} [Thu Jul 21 08:16:42 2022] PUT /identity/v3/system/users/83c8bdcbe1da4ac9b82d63af18922290/roles/9e5171884e9f401ab2337b74db49546e => generated 0 bytes in 183 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 08:16:44.378383 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e66fcd79-a934-447f-8c8a-8161cc8c312f None None] REQUEST_METHOD: `GET` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:16:44.379389 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e66fcd79-a934-447f-8c8a-8161cc8c312f None None] SCRIPT_NAME: `/identity` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:16:44.379886 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e66fcd79-a934-447f-8c8a-8161cc8c312f None None] PATH_INFO: `/` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:16:44.382000 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: [pid: 80245|app: 0|req: 168/335] 10.222.0.47 () {54 vars in 926 bytes} [Thu Jul 21 08:16:44 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 08:16:44.397700 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-245023b1-247c-46ff-b691-07e86b032822 None None] REQUEST_METHOD: `POST` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:16:44.398618 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-245023b1-247c-46ff-b691-07e86b032822 None None] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:16:44.399089 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-245023b1-247c-46ff-b691-07e86b032822 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:16:44.424411 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.auth.core [None req-245023b1-247c-46ff-b691-07e86b032822 None None] MFA Rules not processed for user `2dce40b8d74c41ec94cc36e6ff05d412`. Rule list: `[]` (Enabled: `True`). {{(pid=80244) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 08:16:44.425956 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.fernet_utils [None req-245023b1-247c-46ff-b691-07e86b032822 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80244) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 08:16:44.459365 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG dbcounter [-] [80244] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80244) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:16:44.510068 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 168/336] 10.222.0.47 () {58 vars in 1023 bytes} [Thu Jul 21 08:16:44 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 21 08:16:44.560320 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e66fcd79-a934-447f-8c8a-8161cc8c312f None None] Authenticating user token {{(pid=80245) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:16:44.597984 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02532afc-3f16-4657-bbb2-7cf3fc015751 None None] REQUEST_METHOD: `GET` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:16:44.598637 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02532afc-3f16-4657-bbb2-7cf3fc015751 None None] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:16:44.599706 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02532afc-3f16-4657-bbb2-7cf3fc015751 None None] PATH_INFO: `/` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:16:44.601947 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 169/337] 10.222.0.47 () {54 vars in 926 bytes} [Thu Jul 21 08:16:44 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 08:16:44.607541 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1e9f592-f76a-493c-a923-1ebe84e8712c None None] REQUEST_METHOD: `POST` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:16:44.608086 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1e9f592-f76a-493c-a923-1ebe84e8712c None None] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:16:44.608559 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1e9f592-f76a-493c-a923-1ebe84e8712c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:16:44.616099 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG dbcounter [-] [80245] Writing DB stats keystone:SELECT=100 {{(pid=80245) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:16:44.629054 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.auth.core [None req-f1e9f592-f76a-493c-a923-1ebe84e8712c None None] MFA Rules not processed for user `2dce40b8d74c41ec94cc36e6ff05d412`. Rule list: `[]` (Enabled: `True`). {{(pid=80244) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 08:16:44.630985 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.fernet_utils [None req-f1e9f592-f76a-493c-a923-1ebe84e8712c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80244) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 08:16:44.680946 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e66fcd79-a934-447f-8c8a-8161cc8c312f None None] Validating token access rules against request {{(pid=80245) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:16:44.718646 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-436c46cb-f743-4b6f-956c-c5b3fa481da3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80245) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:16:44.719846 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-436c46cb-f743-4b6f-956c-c5b3fa481da3 None admin] REQUEST_METHOD: `GET` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:16:44.720294 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-436c46cb-f743-4b6f-956c-c5b3fa481da3 None admin] SCRIPT_NAME: `/identity` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:16:44.720705 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-436c46cb-f743-4b6f-956c-c5b3fa481da3 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:16:44.733185 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-436c46cb-f743-4b6f-956c-c5b3fa481da3 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 21 08:16:44.744066 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application [None req-436c46cb-f743-4b6f-956c-c5b3fa481da3 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 21 08:16:44.744066 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 08:16:44.744066 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 08:16:44.744066 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 08:16:44.744066 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 08:16:44.744066 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 08:16:44.744066 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 08:16:44.744066 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 08:16:44.744066 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 08:16:44.744066 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 08:16:44.744066 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 08:16:44.744066 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 08:16:44.744066 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 21 08:16:44.744066 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 21 08:16:44.744066 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 21 08:16:44.744066 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 21 08:16:44.745217 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 21 08:16:44.745217 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 21 08:16:44.745217 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 21 08:16:44.745217 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 21 08:16:44.745217 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 08:16:44.745217 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 08:16:44.745217 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 08:16:44.745217 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 08:16:44.745217 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 08:16:44.745217 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 08:16:44.745217 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 08:16:44.745217 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application with Lock( Jul 21 08:16:44.745217 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 08:16:44.745217 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application return self._enter() Jul 21 08:16:44.745217 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 08:16:44.745217 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 08:16:44.746151 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 08:16:44.746151 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application return self.creator() Jul 21 08:16:44.746151 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 08:16:44.746151 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application created_value = creator( Jul 21 08:16:44.746151 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 21 08:16:44.746151 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 21 08:16:44.746151 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 21 08:16:44.746151 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 21 08:16:44.746151 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 21 08:16:44.746151 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 21 08:16:44.746151 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 21 08:16:44.746151 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application Jul 21 08:16:44.748384 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: [pid: 80245|app: 0|req: 169/338] 10.222.0.47 () {56 vars in 1099 bytes} [Thu Jul 21 08:16:44 2022] GET /identity/v3/roles/member => generated 84 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 08:16:44.756296 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-436c46cb-f743-4b6f-956c-c5b3fa481da3 None admin] Authenticating user token {{(pid=80245) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:16:44.796397 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 170/339] 10.222.0.47 () {58 vars in 1023 bytes} [Thu Jul 21 08:16:44 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 189 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 21 08:16:44.831354 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1e9f592-f76a-493c-a923-1ebe84e8712c None None] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:16:44.903247 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-436c46cb-f743-4b6f-956c-c5b3fa481da3 None admin] Validating token access rules against request {{(pid=80245) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:16:44.907652 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG dbcounter [-] [80245] Writing DB stats keystone:SELECT=100 {{(pid=80245) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:16:44.955597 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce1ba98b-070d-47fe-9b26-16d49b3666dc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80245) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:16:44.956628 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce1ba98b-070d-47fe-9b26-16d49b3666dc None admin] REQUEST_METHOD: `GET` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:16:44.957050 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce1ba98b-070d-47fe-9b26-16d49b3666dc None admin] SCRIPT_NAME: `/identity` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:16:44.957463 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce1ba98b-070d-47fe-9b26-16d49b3666dc None admin] PATH_INFO: `/v3/roles` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:16:44.958375 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce1ba98b-070d-47fe-9b26-16d49b3666dc None admin] RBAC: Adding query filter params (name=member) {{(pid=80245) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 08:16:44.959372 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce1ba98b-070d-47fe-9b26-16d49b3666dc None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:16:44.960886 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce1ba98b-070d-47fe-9b26-16d49b3666dc None admin] RBAC: Authorization granted {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:16:44.968103 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG dbcounter [-] [80244] Writing DB stats keystone:SELECT=100 {{(pid=80244) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:16:44.974859 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: [pid: 80245|app: 0|req: 170/340] 10.222.0.47 () {56 vars in 1101 bytes} [Thu Jul 21 08:16:44 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 219 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 08:16:44.976627 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1e9f592-f76a-493c-a923-1ebe84e8712c None None] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:16:44.981058 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce1ba98b-070d-47fe-9b26-16d49b3666dc None admin] Authenticating user token {{(pid=80245) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:16:45.015387 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07429288-d959-4685-8fe3-e25bfdfc7fab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:16:45.017013 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07429288-d959-4685-8fe3-e25bfdfc7fab None admin] REQUEST_METHOD: `GET` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:16:45.017498 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07429288-d959-4685-8fe3-e25bfdfc7fab None admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:16:45.017952 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07429288-d959-4685-8fe3-e25bfdfc7fab None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:16:45.031069 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07429288-d959-4685-8fe3-e25bfdfc7fab None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 21 08:16:45.040493 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application [None req-07429288-d959-4685-8fe3-e25bfdfc7fab None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 21 08:16:45.040493 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 08:16:45.040493 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 08:16:45.040493 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 08:16:45.040493 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 08:16:45.040493 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 08:16:45.040493 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 08:16:45.040493 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 08:16:45.040493 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 08:16:45.040493 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 08:16:45.040493 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 08:16:45.040493 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 08:16:45.040493 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 21 08:16:45.040493 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 21 08:16:45.040493 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 21 08:16:45.040493 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 21 08:16:45.041381 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 21 08:16:45.041381 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 21 08:16:45.041381 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 21 08:16:45.041381 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 21 08:16:45.041381 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 08:16:45.041381 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 08:16:45.041381 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 08:16:45.041381 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 08:16:45.041381 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 08:16:45.041381 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 08:16:45.041381 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 08:16:45.041381 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application with Lock( Jul 21 08:16:45.041381 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 08:16:45.041381 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self._enter() Jul 21 08:16:45.041381 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 08:16:45.041381 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 08:16:45.042149 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 08:16:45.042149 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self.creator() Jul 21 08:16:45.042149 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 08:16:45.042149 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application created_value = creator( Jul 21 08:16:45.042149 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 21 08:16:45.042149 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 21 08:16:45.042149 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 21 08:16:45.042149 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 21 08:16:45.042149 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 21 08:16:45.042149 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 21 08:16:45.042149 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 21 08:16:45.042149 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application Jul 21 08:16:45.044584 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 171/341] 10.222.0.47 () {56 vars in 1114 bytes} [Thu Jul 21 08:16:44 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 218 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 21 08:16:45.050239 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07429288-d959-4685-8fe3-e25bfdfc7fab None admin] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:16:45.100635 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce1ba98b-070d-47fe-9b26-16d49b3666dc None admin] Validating token access rules against request {{(pid=80245) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:16:45.140687 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a73ff417-e858-4593-94c9-95648f4f1c38 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80245) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:16:45.141582 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a73ff417-e858-4593-94c9-95648f4f1c38 None admin] REQUEST_METHOD: `GET` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:16:45.142006 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a73ff417-e858-4593-94c9-95648f4f1c38 None admin] SCRIPT_NAME: `/identity` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:16:45.142419 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a73ff417-e858-4593-94c9-95648f4f1c38 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:16:45.166553 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07429288-d959-4685-8fe3-e25bfdfc7fab None admin] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:16:45.195717 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG dbcounter [-] [80245] Writing DB stats keystone:SELECT=100 {{(pid=80245) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:16:45.212757 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a38f581-451b-446c-8326-bda1633db7e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:16:45.212757 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a38f581-451b-446c-8326-bda1633db7e2 None admin] REQUEST_METHOD: `GET` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:16:45.212757 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a38f581-451b-446c-8326-bda1633db7e2 None admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:16:45.212757 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a38f581-451b-446c-8326-bda1633db7e2 None admin] PATH_INFO: `/v3/roles` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:16:45.215949 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a38f581-451b-446c-8326-bda1633db7e2 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80244) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 08:16:45.216370 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a38f581-451b-446c-8326-bda1633db7e2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:16:45.217875 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a38f581-451b-446c-8326-bda1633db7e2 None admin] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:16:45.223915 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a73ff417-e858-4593-94c9-95648f4f1c38 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:16:45.225684 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a73ff417-e858-4593-94c9-95648f4f1c38 None admin] RBAC: Authorization granted {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:16:45.228279 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 172/342] 10.222.0.47 () {56 vars in 1111 bytes} [Thu Jul 21 08:16:45 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 08:16:45.234758 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a38f581-451b-446c-8326-bda1633db7e2 None admin] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:16:45.287930 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG dbcounter [-] [80244] Writing DB stats keystone:SELECT=100 {{(pid=80244) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:16:45.358972 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: [pid: 80245|app: 0|req: 171/343] 10.222.0.47 () {58 vars in 1282 bytes} [Thu Jul 21 08:16:44 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 378 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Jul 21 08:16:45.361297 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a68b80e6-61c8-4343-ba11-d614d553ce2d None None] REQUEST_METHOD: `GET` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:16:45.361765 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a68b80e6-61c8-4343-ba11-d614d553ce2d None None] SCRIPT_NAME: `/identity` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:16:45.362178 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a68b80e6-61c8-4343-ba11-d614d553ce2d None None] PATH_INFO: `/` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:16:45.364184 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: [pid: 80245|app: 0|req: 172/344] 10.222.0.47 () {54 vars in 926 bytes} [Thu Jul 21 08:16:45 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 08:16:45.366045 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a68b80e6-61c8-4343-ba11-d614d553ce2d None None] Authenticating user token {{(pid=80245) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:16:45.400059 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a38f581-451b-446c-8326-bda1633db7e2 None admin] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:16:45.450887 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeb6305f-ad37-43c7-b4c8-3d181654ff78 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:16:45.451909 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeb6305f-ad37-43c7-b4c8-3d181654ff78 None admin] REQUEST_METHOD: `GET` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:16:45.453007 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeb6305f-ad37-43c7-b4c8-3d181654ff78 None admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:16:45.453464 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeb6305f-ad37-43c7-b4c8-3d181654ff78 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:16:45.484209 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG dbcounter [-] [80245] Writing DB stats keystone:SELECT=100 {{(pid=80245) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:16:45.533283 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a68b80e6-61c8-4343-ba11-d614d553ce2d None None] Validating token access rules against request {{(pid=80245) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:16:45.546541 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eeb6305f-ad37-43c7-b4c8-3d181654ff78 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:16:45.548828 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eeb6305f-ad37-43c7-b4c8-3d181654ff78 None admin] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:16:45.575593 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38f85622-43b9-4c9d-a691-680f5ecbdf56 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80245) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:16:45.576758 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38f85622-43b9-4c9d-a691-680f5ecbdf56 None admin] REQUEST_METHOD: `GET` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:16:45.577242 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38f85622-43b9-4c9d-a691-680f5ecbdf56 None admin] SCRIPT_NAME: `/identity` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:16:45.577661 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38f85622-43b9-4c9d-a691-680f5ecbdf56 None admin] PATH_INFO: `/v3/projects/eab7e5ec54c94574ad68c89747ad5bcc` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:16:45.579071 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38f85622-43b9-4c9d-a691-680f5ecbdf56 None admin] RBAC: Authorizing `identity:get_project(project_id=eab7e5ec54c94574ad68c89747ad5bcc)` {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:16:45.580726 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38f85622-43b9-4c9d-a691-680f5ecbdf56 None admin] RBAC: Authorization granted {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:16:45.582718 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: [pid: 80245|app: 0|req: 173/345] 10.222.0.47 () {56 vars in 1186 bytes} [Thu Jul 21 08:16:45 2022] GET /identity/v3/projects/eab7e5ec54c94574ad68c89747ad5bcc => generated 298 bytes in 218 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 08:16:45.583208 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG dbcounter [-] [80244] Writing DB stats keystone:SELECT=100 {{(pid=80244) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:16:45.586504 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28daf55b-151f-4705-8eb6-d5f9fb4c80c4 None None] REQUEST_METHOD: `POST` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:16:45.587029 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28daf55b-151f-4705-8eb6-d5f9fb4c80c4 None None] SCRIPT_NAME: `/identity` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:16:45.587829 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28daf55b-151f-4705-8eb6-d5f9fb4c80c4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:16:45.606853 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.auth.core [None req-28daf55b-151f-4705-8eb6-d5f9fb4c80c4 None None] MFA Rules not processed for user `2dce40b8d74c41ec94cc36e6ff05d412`. Rule list: `[]` (Enabled: `True`). {{(pid=80245) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 08:16:45.608511 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.fernet_utils [None req-28daf55b-151f-4705-8eb6-d5f9fb4c80c4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80245) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 08:16:45.689589 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 173/346] 10.222.0.47 () {58 vars in 1282 bytes} [Thu Jul 21 08:16:45 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 452 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Jul 21 08:16:45.689589 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce67b28e-32d6-4458-8585-4a7cea0161ca None None] REQUEST_METHOD: `GET` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:16:45.689589 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce67b28e-32d6-4458-8585-4a7cea0161ca None None] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:16:45.689589 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce67b28e-32d6-4458-8585-4a7cea0161ca None None] PATH_INFO: `/` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:16:45.690002 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 174/347] 10.222.0.47 () {54 vars in 926 bytes} [Thu Jul 21 08:16:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 08:16:45.690886 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce67b28e-32d6-4458-8585-4a7cea0161ca None None] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:16:45.702221 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: [pid: 80245|app: 0|req: 174/348] 10.222.0.47 () {58 vars in 1023 bytes} [Thu Jul 21 08:16:45 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 21 08:16:45.703470 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28daf55b-151f-4705-8eb6-d5f9fb4c80c4 None None] Authenticating user token {{(pid=80245) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:16:45.814161 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG dbcounter [-] [80245] Writing DB stats keystone:SELECT=100 {{(pid=80245) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:16:45.838125 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce67b28e-32d6-4458-8585-4a7cea0161ca None None] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:16:45.854106 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28daf55b-151f-4705-8eb6-d5f9fb4c80c4 None None] Validating token access rules against request {{(pid=80245) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:16:45.858795 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG dbcounter [-] [80244] Writing DB stats keystone:SELECT=100 {{(pid=80244) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:16:45.878046 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9733a194-38f4-4f2c-8d7e-9848e0211e91 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:16:45.879006 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9733a194-38f4-4f2c-8d7e-9848e0211e91 None admin] REQUEST_METHOD: `GET` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:16:45.879477 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9733a194-38f4-4f2c-8d7e-9848e0211e91 None admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:16:45.879924 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9733a194-38f4-4f2c-8d7e-9848e0211e91 None admin] PATH_INFO: `/v3/groups/f75bd62425144927b6fd534595218714` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:16:45.887203 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9733a194-38f4-4f2c-8d7e-9848e0211e91 None admin] RBAC: Authorizing `identity:get_group(group_id=f75bd62425144927b6fd534595218714)` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:16:45.889025 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9733a194-38f4-4f2c-8d7e-9848e0211e91 None admin] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:16:45.891196 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 175/349] 10.222.0.47 () {56 vars in 1180 bytes} [Thu Jul 21 08:16:45 2022] GET /identity/v3/groups/f75bd62425144927b6fd534595218714 => generated 226 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 08:16:45.892841 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d8ad7d4-1604-41f2-a47c-2613da3453e2 None None] REQUEST_METHOD: `POST` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:16:45.893405 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d8ad7d4-1604-41f2-a47c-2613da3453e2 None None] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:16:45.893926 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d8ad7d4-1604-41f2-a47c-2613da3453e2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:16:45.896506 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb034e4e-9606-41da-a22e-04ab22584382 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80245) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:16:45.898576 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb034e4e-9606-41da-a22e-04ab22584382 None admin] REQUEST_METHOD: `GET` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:16:45.898961 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb034e4e-9606-41da-a22e-04ab22584382 None admin] SCRIPT_NAME: `/identity` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:16:45.899357 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb034e4e-9606-41da-a22e-04ab22584382 None admin] PATH_INFO: `/v3/projects/eab7e5ec54c94574ad68c89747ad5bcc` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:16:45.900730 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb034e4e-9606-41da-a22e-04ab22584382 None admin] RBAC: Authorizing `identity:get_project(project_id=eab7e5ec54c94574ad68c89747ad5bcc)` {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:16:45.903136 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb034e4e-9606-41da-a22e-04ab22584382 None admin] RBAC: Authorization granted {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:16:45.904189 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: [pid: 80245|app: 0|req: 175/350] 10.222.0.47 () {56 vars in 1186 bytes} [Thu Jul 21 08:16:45 2022] GET /identity/v3/projects/eab7e5ec54c94574ad68c89747ad5bcc => generated 298 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 08:16:45.905052 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb034e4e-9606-41da-a22e-04ab22584382 None admin] Authenticating user token {{(pid=80245) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:16:45.915579 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.auth.core [None req-2d8ad7d4-1604-41f2-a47c-2613da3453e2 None None] MFA Rules not processed for user `2dce40b8d74c41ec94cc36e6ff05d412`. Rule list: `[]` (Enabled: `True`). {{(pid=80244) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 08:16:45.916982 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.fernet_utils [None req-2d8ad7d4-1604-41f2-a47c-2613da3453e2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80244) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 08:16:45.996017 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 176/351] 10.222.0.47 () {58 vars in 1023 bytes} [Thu Jul 21 08:16:45 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 21 08:16:45.996777 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d8ad7d4-1604-41f2-a47c-2613da3453e2 None None] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:16:46.033860 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb034e4e-9606-41da-a22e-04ab22584382 None admin] Validating token access rules against request {{(pid=80245) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:16:46.071738 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG dbcounter [-] [80245] Writing DB stats keystone:SELECT=100 {{(pid=80245) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:16:46.080050 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27e9552a-7cdd-44d9-9989-976fb6920753 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80245) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:16:46.080981 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27e9552a-7cdd-44d9-9989-976fb6920753 None admin] REQUEST_METHOD: `GET` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:16:46.081266 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27e9552a-7cdd-44d9-9989-976fb6920753 None admin] SCRIPT_NAME: `/identity` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:16:46.081542 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27e9552a-7cdd-44d9-9989-976fb6920753 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:16:46.096091 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27e9552a-7cdd-44d9-9989-976fb6920753 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 21 08:16:46.104130 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application [None req-27e9552a-7cdd-44d9-9989-976fb6920753 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jul 21 08:16:46.104130 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 08:16:46.104130 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 08:16:46.104130 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 08:16:46.104130 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 08:16:46.104130 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 08:16:46.104130 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 08:16:46.104130 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 08:16:46.104130 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 08:16:46.104130 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 08:16:46.104130 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 08:16:46.104130 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 08:16:46.104130 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 21 08:16:46.104130 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 21 08:16:46.104130 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 21 08:16:46.104130 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 21 08:16:46.105175 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 21 08:16:46.105175 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 21 08:16:46.105175 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 21 08:16:46.105175 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 21 08:16:46.105175 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 08:16:46.105175 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 08:16:46.105175 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 08:16:46.105175 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 08:16:46.105175 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 08:16:46.105175 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 08:16:46.105175 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 08:16:46.105175 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application with Lock( Jul 21 08:16:46.105175 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 08:16:46.105175 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application return self._enter() Jul 21 08:16:46.105175 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 08:16:46.105175 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 08:16:46.105977 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 08:16:46.105977 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application return self.creator() Jul 21 08:16:46.105977 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 08:16:46.105977 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application created_value = creator( Jul 21 08:16:46.105977 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 21 08:16:46.105977 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 21 08:16:46.105977 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 21 08:16:46.105977 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 21 08:16:46.105977 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 21 08:16:46.105977 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 21 08:16:46.105977 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jul 21 08:16:46.105977 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application Jul 21 08:16:46.106573 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: [pid: 80245|app: 0|req: 176/352] 10.222.0.47 () {56 vars in 1099 bytes} [Thu Jul 21 08:16:45 2022] GET /identity/v3/roles/reader => generated 84 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 08:16:46.107507 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27e9552a-7cdd-44d9-9989-976fb6920753 None admin] Authenticating user token {{(pid=80245) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:16:46.150518 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d8ad7d4-1604-41f2-a47c-2613da3453e2 None None] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:16:46.167274 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG dbcounter [-] [80244] Writing DB stats keystone:SELECT=100 {{(pid=80244) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:16:46.202801 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98f300c2-8a99-4fd3-9021-7dc615af01d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:16:46.203531 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98f300c2-8a99-4fd3-9021-7dc615af01d5 None admin] REQUEST_METHOD: `GET` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:16:46.203819 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98f300c2-8a99-4fd3-9021-7dc615af01d5 None admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:16:46.204094 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98f300c2-8a99-4fd3-9021-7dc615af01d5 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:16:46.204947 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98f300c2-8a99-4fd3-9021-7dc615af01d5 None admin] RBAC: Adding query filter params (group.id=f75bd62425144927b6fd534595218714, role.id=3a074738221e4601b5205715ba87d240, scope.project.id=eab7e5ec54c94574ad68c89747ad5bcc) {{(pid=80244) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 08:16:46.205313 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98f300c2-8a99-4fd3-9021-7dc615af01d5 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:16:46.206799 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98f300c2-8a99-4fd3-9021-7dc615af01d5 None admin] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:16:46.223462 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 177/353] 10.222.0.47 () {56 vars in 1376 bytes} [Thu Jul 21 08:16:45 2022] GET /identity/v3/role_assignments?group.id=f75bd62425144927b6fd534595218714&scope.project.id=eab7e5ec54c94574ad68c89747ad5bcc&role.id=3a074738221e4601b5205715ba87d240 => generated 260 bytes in 219 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 08:16:46.225069 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bb5368d-c41e-4674-8859-cb81d544bef1 None None] REQUEST_METHOD: `GET` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:16:46.225361 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bb5368d-c41e-4674-8859-cb81d544bef1 None None] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:16:46.225640 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bb5368d-c41e-4674-8859-cb81d544bef1 None None] PATH_INFO: `/` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:16:46.227418 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 178/354] 10.222.0.47 () {54 vars in 926 bytes} [Thu Jul 21 08:16:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 08:16:46.228172 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bb5368d-c41e-4674-8859-cb81d544bef1 None None] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:16:46.258804 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27e9552a-7cdd-44d9-9989-976fb6920753 None admin] Validating token access rules against request {{(pid=80245) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:16:46.312029 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c62c7623-ff7c-44c6-819d-ca6f32cfce5f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80245) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:16:46.313058 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c62c7623-ff7c-44c6-819d-ca6f32cfce5f None admin] REQUEST_METHOD: `GET` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:16:46.313423 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c62c7623-ff7c-44c6-819d-ca6f32cfce5f None admin] SCRIPT_NAME: `/identity` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:16:46.313625 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c62c7623-ff7c-44c6-819d-ca6f32cfce5f None admin] PATH_INFO: `/v3/groups/f75bd62425144927b6fd534595218714` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:16:46.314954 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c62c7623-ff7c-44c6-819d-ca6f32cfce5f None admin] RBAC: Authorizing `identity:get_group(group_id=f75bd62425144927b6fd534595218714)` {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:16:46.316633 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c62c7623-ff7c-44c6-819d-ca6f32cfce5f None admin] RBAC: Authorization granted {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:16:46.318811 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: [pid: 80245|app: 0|req: 177/355] 10.222.0.47 () {56 vars in 1180 bytes} [Thu Jul 21 08:16:46 2022] GET /identity/v3/groups/f75bd62425144927b6fd534595218714 => generated 226 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 08:16:46.321303 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c62c7623-ff7c-44c6-819d-ca6f32cfce5f None admin] Authenticating user token {{(pid=80245) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:16:46.384359 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bb5368d-c41e-4674-8859-cb81d544bef1 None None] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:16:46.398002 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG dbcounter [-] [80245] Writing DB stats keystone:SELECT=100 {{(pid=80245) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:16:46.437889 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57970566-efba-47ca-af45-d974484c5a57 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:16:46.438814 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57970566-efba-47ca-af45-d974484c5a57 None admin] REQUEST_METHOD: `GET` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:16:46.439101 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57970566-efba-47ca-af45-d974484c5a57 None admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:16:46.439418 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57970566-efba-47ca-af45-d974484c5a57 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:16:46.452676 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c62c7623-ff7c-44c6-819d-ca6f32cfce5f None admin] Validating token access rules against request {{(pid=80245) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:16:46.459188 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57970566-efba-47ca-af45-d974484c5a57 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 21 08:16:46.479147 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application [None req-57970566-efba-47ca-af45-d974484c5a57 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 21 08:16:46.479147 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 08:16:46.479147 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 08:16:46.479147 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 08:16:46.479147 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 08:16:46.479147 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 08:16:46.479147 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 08:16:46.479147 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 08:16:46.479147 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 08:16:46.479147 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 08:16:46.479147 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 08:16:46.479147 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 08:16:46.479147 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 21 08:16:46.479147 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 21 08:16:46.479147 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 21 08:16:46.479147 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 21 08:16:46.480210 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 21 08:16:46.480210 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 21 08:16:46.480210 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 21 08:16:46.480210 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 21 08:16:46.480210 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 08:16:46.480210 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 08:16:46.480210 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 08:16:46.480210 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 08:16:46.480210 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 08:16:46.480210 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 08:16:46.480210 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 08:16:46.480210 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application with Lock( Jul 21 08:16:46.480210 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 08:16:46.480210 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self._enter() Jul 21 08:16:46.480210 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 08:16:46.480210 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 08:16:46.481455 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 08:16:46.481455 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self.creator() Jul 21 08:16:46.481455 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 08:16:46.481455 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application created_value = creator( Jul 21 08:16:46.481455 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 21 08:16:46.481455 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 21 08:16:46.481455 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 21 08:16:46.481455 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 21 08:16:46.481455 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 21 08:16:46.481455 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 21 08:16:46.481455 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 21 08:16:46.481455 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application Jul 21 08:16:46.481455 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 179/356] 10.222.0.47 () {56 vars in 1099 bytes} [Thu Jul 21 08:16:46 2022] GET /identity/v3/roles/member => generated 84 bytes in 253 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 08:16:46.482152 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8b54ea3-d2a0-4f5a-8ad3-f9a942cf516e None None] REQUEST_METHOD: `POST` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:16:46.482446 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8b54ea3-d2a0-4f5a-8ad3-f9a942cf516e None None] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:16:46.484389 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8b54ea3-d2a0-4f5a-8ad3-f9a942cf516e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:16:46.505478 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0614a7e-6965-4913-9bb3-f6e6f697ff20 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80245) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:16:46.506167 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0614a7e-6965-4913-9bb3-f6e6f697ff20 None admin] REQUEST_METHOD: `GET` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:16:46.509152 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0614a7e-6965-4913-9bb3-f6e6f697ff20 None admin] SCRIPT_NAME: `/identity` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:16:46.509152 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0614a7e-6965-4913-9bb3-f6e6f697ff20 None admin] PATH_INFO: `/v3/roles` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:16:46.509152 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0614a7e-6965-4913-9bb3-f6e6f697ff20 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80245) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 08:16:46.509152 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0614a7e-6965-4913-9bb3-f6e6f697ff20 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:16:46.509773 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0614a7e-6965-4913-9bb3-f6e6f697ff20 None admin] RBAC: Authorization granted {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:16:46.510608 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.auth.core [None req-f8b54ea3-d2a0-4f5a-8ad3-f9a942cf516e None None] MFA Rules not processed for user `2dce40b8d74c41ec94cc36e6ff05d412`. Rule list: `[]` (Enabled: `True`). {{(pid=80244) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 08:16:46.511927 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.fernet_utils [None req-f8b54ea3-d2a0-4f5a-8ad3-f9a942cf516e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80244) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 08:16:46.518716 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: [pid: 80245|app: 0|req: 178/357] 10.222.0.47 () {56 vars in 1101 bytes} [Thu Jul 21 08:16:46 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 08:16:46.519589 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0614a7e-6965-4913-9bb3-f6e6f697ff20 None admin] Authenticating user token {{(pid=80245) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:16:46.551556 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG dbcounter [-] [80244] Writing DB stats keystone:SELECT=100 {{(pid=80244) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:16:46.607140 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 180/358] 10.222.0.47 () {58 vars in 1023 bytes} [Thu Jul 21 08:16:46 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 21 08:16:46.607842 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8b54ea3-d2a0-4f5a-8ad3-f9a942cf516e None None] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:16:46.634034 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0614a7e-6965-4913-9bb3-f6e6f697ff20 None admin] Validating token access rules against request {{(pid=80245) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:16:46.659458 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG dbcounter [-] [80245] Writing DB stats keystone:SELECT=100 {{(pid=80245) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:16:46.673252 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34c1df1f-fd28-4c14-b6c4-d359fb5e6ad4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80245) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:16:46.673915 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34c1df1f-fd28-4c14-b6c4-d359fb5e6ad4 None admin] REQUEST_METHOD: `GET` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:16:46.674195 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34c1df1f-fd28-4c14-b6c4-d359fb5e6ad4 None admin] SCRIPT_NAME: `/identity` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:16:46.674467 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34c1df1f-fd28-4c14-b6c4-d359fb5e6ad4 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:16:46.675292 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34c1df1f-fd28-4c14-b6c4-d359fb5e6ad4 None admin] RBAC: Adding query filter params (group.id=f75bd62425144927b6fd534595218714, role.id=99a638c0aac44b79a1f2858574724495, scope.project.id=eab7e5ec54c94574ad68c89747ad5bcc) {{(pid=80245) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 08:16:46.675648 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34c1df1f-fd28-4c14-b6c4-d359fb5e6ad4 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:16:46.677089 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34c1df1f-fd28-4c14-b6c4-d359fb5e6ad4 None admin] RBAC: Authorization granted {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:16:46.683761 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: [pid: 80245|app: 0|req: 179/359] 10.222.0.47 () {56 vars in 1376 bytes} [Thu Jul 21 08:16:46 2022] GET /identity/v3/role_assignments?group.id=f75bd62425144927b6fd534595218714&scope.project.id=eab7e5ec54c94574ad68c89747ad5bcc&role.id=99a638c0aac44b79a1f2858574724495 => generated 260 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 08:16:46.684795 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34c1df1f-fd28-4c14-b6c4-d359fb5e6ad4 None admin] Authenticating user token {{(pid=80245) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:16:46.727359 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8b54ea3-d2a0-4f5a-8ad3-f9a942cf516e None None] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:16:46.779003 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da5b8156-4a00-4561-9911-ab9bbf635cd1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:16:46.779974 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da5b8156-4a00-4561-9911-ab9bbf635cd1 None admin] REQUEST_METHOD: `GET` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:16:46.780421 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da5b8156-4a00-4561-9911-ab9bbf635cd1 None admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:16:46.780852 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da5b8156-4a00-4561-9911-ab9bbf635cd1 None admin] PATH_INFO: `/v3/roles` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:16:46.781795 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da5b8156-4a00-4561-9911-ab9bbf635cd1 None admin] RBAC: Adding query filter params (name=member) {{(pid=80244) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 08:16:46.782253 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da5b8156-4a00-4561-9911-ab9bbf635cd1 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:16:46.784184 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da5b8156-4a00-4561-9911-ab9bbf635cd1 None admin] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:16:46.803210 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 181/360] 10.222.0.47 () {56 vars in 1101 bytes} [Thu Jul 21 08:16:46 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 08:16:46.803210 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1b86cce-e14c-4a64-85f9-07c4f7851d08 None None] REQUEST_METHOD: `GET` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:16:46.803210 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1b86cce-e14c-4a64-85f9-07c4f7851d08 None None] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:16:46.803210 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1b86cce-e14c-4a64-85f9-07c4f7851d08 None None] PATH_INFO: `/` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:16:46.804895 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 182/361] 10.222.0.47 () {54 vars in 926 bytes} [Thu Jul 21 08:16:46 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 08:16:46.805703 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1b86cce-e14c-4a64-85f9-07c4f7851d08 None None] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:16:46.830185 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34c1df1f-fd28-4c14-b6c4-d359fb5e6ad4 None admin] Validating token access rules against request {{(pid=80245) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:16:46.857515 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG dbcounter [-] [80244] Writing DB stats keystone:SELECT=100 {{(pid=80244) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:16:46.875706 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bb3d7e2-5cf7-462e-a95d-51f7824e20bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80245) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:16:46.877022 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bb3d7e2-5cf7-462e-a95d-51f7824e20bf None admin] REQUEST_METHOD: `GET` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:16:46.877507 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bb3d7e2-5cf7-462e-a95d-51f7824e20bf None admin] SCRIPT_NAME: `/identity` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:16:46.877996 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bb3d7e2-5cf7-462e-a95d-51f7824e20bf None admin] PATH_INFO: `/v3/users/83c8bdcbe1da4ac9b82d63af18922290` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:16:46.879362 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bb3d7e2-5cf7-462e-a95d-51f7824e20bf None admin] RBAC: Authorizing `identity:get_user(user_id=83c8bdcbe1da4ac9b82d63af18922290)` {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:16:46.880978 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bb3d7e2-5cf7-462e-a95d-51f7824e20bf None admin] RBAC: Authorization granted {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:16:46.884064 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: [pid: 80245|app: 0|req: 180/362] 10.222.0.47 () {56 vars in 1177 bytes} [Thu Jul 21 08:16:46 2022] GET /identity/v3/users/83c8bdcbe1da4ac9b82d63af18922290 => generated 293 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 08:16:46.884207 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bb3d7e2-5cf7-462e-a95d-51f7824e20bf None admin] Authenticating user token {{(pid=80245) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:16:46.954279 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1b86cce-e14c-4a64-85f9-07c4f7851d08 None None] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:16:46.965409 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG dbcounter [-] [80245] Writing DB stats keystone:SELECT=100 {{(pid=80245) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:16:46.998509 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b982fd98-51f8-499e-aa60-a9cdc7c063dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:16:46.999730 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b982fd98-51f8-499e-aa60-a9cdc7c063dd None admin] REQUEST_METHOD: `GET` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:16:47.000269 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b982fd98-51f8-499e-aa60-a9cdc7c063dd None admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:16:47.000739 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b982fd98-51f8-499e-aa60-a9cdc7c063dd None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:16:47.013017 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b982fd98-51f8-499e-aa60-a9cdc7c063dd None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 21 08:16:47.021615 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bb3d7e2-5cf7-462e-a95d-51f7824e20bf None admin] Validating token access rules against request {{(pid=80245) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:16:47.022736 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application [None req-b982fd98-51f8-499e-aa60-a9cdc7c063dd None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 21 08:16:47.022736 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 08:16:47.022736 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 08:16:47.022736 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 08:16:47.022736 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 08:16:47.022736 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 08:16:47.022736 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 08:16:47.022736 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 08:16:47.022736 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 08:16:47.022736 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 08:16:47.022736 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 08:16:47.022736 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 08:16:47.022736 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 21 08:16:47.022736 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 21 08:16:47.022736 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 21 08:16:47.022736 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 21 08:16:47.023921 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 21 08:16:47.023921 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 21 08:16:47.023921 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 21 08:16:47.023921 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 21 08:16:47.023921 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 08:16:47.023921 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 08:16:47.023921 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 08:16:47.023921 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 08:16:47.023921 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 08:16:47.023921 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 08:16:47.023921 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 08:16:47.023921 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application with Lock( Jul 21 08:16:47.023921 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 08:16:47.023921 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self._enter() Jul 21 08:16:47.023921 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 08:16:47.023921 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 08:16:47.024695 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 08:16:47.024695 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self.creator() Jul 21 08:16:47.024695 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 08:16:47.024695 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application created_value = creator( Jul 21 08:16:47.024695 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 21 08:16:47.024695 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 21 08:16:47.024695 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 21 08:16:47.024695 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 21 08:16:47.024695 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 21 08:16:47.024695 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 21 08:16:47.024695 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 21 08:16:47.024695 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application Jul 21 08:16:47.027095 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 183/363] 10.222.0.47 () {56 vars in 1114 bytes} [Thu Jul 21 08:16:46 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 222 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 08:16:47.028595 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-836b249e-489e-4f80-8aec-97cc60943590 None None] REQUEST_METHOD: `POST` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:16:47.029163 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-836b249e-489e-4f80-8aec-97cc60943590 None None] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:16:47.029593 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-836b249e-489e-4f80-8aec-97cc60943590 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:16:47.048950 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.auth.core [None req-836b249e-489e-4f80-8aec-97cc60943590 None None] MFA Rules not processed for user `2dce40b8d74c41ec94cc36e6ff05d412`. Rule list: `[]` (Enabled: `True`). {{(pid=80244) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 08:16:47.050474 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.fernet_utils [None req-836b249e-489e-4f80-8aec-97cc60943590 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80244) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 08:16:47.061197 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad073fc1-943f-4a0e-b9c6-c35058232525 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80245) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:16:47.061865 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad073fc1-943f-4a0e-b9c6-c35058232525 None admin] REQUEST_METHOD: `GET` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:16:47.062148 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad073fc1-943f-4a0e-b9c6-c35058232525 None admin] SCRIPT_NAME: `/identity` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:16:47.062424 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad073fc1-943f-4a0e-b9c6-c35058232525 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:16:47.141661 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 184/364] 10.222.0.47 () {58 vars in 1023 bytes} [Thu Jul 21 08:16:47 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 21 08:16:47.143020 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-836b249e-489e-4f80-8aec-97cc60943590 None None] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:16:47.162920 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad073fc1-943f-4a0e-b9c6-c35058232525 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:16:47.164955 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad073fc1-943f-4a0e-b9c6-c35058232525 None admin] RBAC: Authorization granted {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:16:47.180872 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG dbcounter [-] [80244] Writing DB stats keystone:SELECT=100 {{(pid=80244) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:16:47.236456 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG dbcounter [-] [80245] Writing DB stats keystone:SELECT=100 {{(pid=80245) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:16:47.291457 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-836b249e-489e-4f80-8aec-97cc60943590 None None] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:16:47.295560 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: [pid: 80245|app: 0|req: 181/365] 10.222.0.47 () {58 vars in 1282 bytes} [Thu Jul 21 08:16:46 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 410 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 21 08:16:47.295560 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad073fc1-943f-4a0e-b9c6-c35058232525 None admin] Authenticating user token {{(pid=80245) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:16:47.343063 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7636c42c-d287-402c-ad08-256e61819018 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:16:47.344011 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7636c42c-d287-402c-ad08-256e61819018 None admin] REQUEST_METHOD: `GET` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:16:47.344433 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7636c42c-d287-402c-ad08-256e61819018 None admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:16:47.344846 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7636c42c-d287-402c-ad08-256e61819018 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:16:47.345829 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7636c42c-d287-402c-ad08-256e61819018 None admin] RBAC: Adding query filter params (role.id=9e5171884e9f401ab2337b74db49546e, user.id=83c8bdcbe1da4ac9b82d63af18922290, scope.system=all) {{(pid=80244) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 08:16:47.346136 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7636c42c-d287-402c-ad08-256e61819018 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:16:47.347614 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7636c42c-d287-402c-ad08-256e61819018 None admin] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:16:47.361269 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 185/366] 10.222.0.47 () {56 vars in 1308 bytes} [Thu Jul 21 08:16:47 2022] GET /identity/v3/role_assignments?user.id=83c8bdcbe1da4ac9b82d63af18922290&scope.system=all&role.id=9e5171884e9f401ab2337b74db49546e => generated 510 bytes in 219 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 08:16:47.362118 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7636c42c-d287-402c-ad08-256e61819018 None admin] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:16:47.425287 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad073fc1-943f-4a0e-b9c6-c35058232525 None admin] Validating token access rules against request {{(pid=80245) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:16:47.453608 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG dbcounter [-] [80244] Writing DB stats keystone:SELECT=100 {{(pid=80244) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:16:47.466389 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6938b02-11fd-4e56-8181-229b7b92731c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80245) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:16:47.468221 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6938b02-11fd-4e56-8181-229b7b92731c None admin] REQUEST_METHOD: `GET` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:16:47.468933 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6938b02-11fd-4e56-8181-229b7b92731c None admin] SCRIPT_NAME: `/identity` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:16:47.469617 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6938b02-11fd-4e56-8181-229b7b92731c None admin] PATH_INFO: `/v3/roles` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:16:47.472101 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6938b02-11fd-4e56-8181-229b7b92731c None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80245) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 08:16:47.472869 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6938b02-11fd-4e56-8181-229b7b92731c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:16:47.474788 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6938b02-11fd-4e56-8181-229b7b92731c None admin] RBAC: Authorization granted {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:16:47.481896 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7636c42c-d287-402c-ad08-256e61819018 None admin] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:16:47.485957 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: [pid: 80245|app: 0|req: 182/367] 10.222.0.47 () {56 vars in 1111 bytes} [Thu Jul 21 08:16:47 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 08:16:47.487144 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6938b02-11fd-4e56-8181-229b7b92731c None admin] Authenticating user token {{(pid=80245) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:16:47.510355 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG dbcounter [-] [80245] Writing DB stats keystone:SELECT=100 {{(pid=80245) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:16:47.524108 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0db41ccf-5151-43de-af18-c491f296133f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:16:47.524862 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0db41ccf-5151-43de-af18-c491f296133f None admin] REQUEST_METHOD: `GET` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:16:47.525142 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0db41ccf-5151-43de-af18-c491f296133f None admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:16:47.525412 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0db41ccf-5151-43de-af18-c491f296133f None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:16:47.537703 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0db41ccf-5151-43de-af18-c491f296133f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 21 08:16:47.548885 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application [None req-0db41ccf-5151-43de-af18-c491f296133f None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 21 08:16:47.548885 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 08:16:47.548885 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 08:16:47.548885 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 08:16:47.548885 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 08:16:47.548885 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 08:16:47.548885 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 08:16:47.548885 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 08:16:47.548885 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 08:16:47.548885 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 08:16:47.548885 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 08:16:47.548885 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 08:16:47.548885 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 21 08:16:47.548885 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 21 08:16:47.548885 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 21 08:16:47.548885 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 21 08:16:47.549803 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 21 08:16:47.549803 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 21 08:16:47.549803 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 21 08:16:47.549803 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 21 08:16:47.549803 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 08:16:47.549803 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 08:16:47.549803 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 08:16:47.549803 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 08:16:47.549803 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 08:16:47.549803 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 08:16:47.549803 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 08:16:47.549803 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application with Lock( Jul 21 08:16:47.549803 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 08:16:47.549803 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self._enter() Jul 21 08:16:47.549803 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 08:16:47.549803 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 08:16:47.550577 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 08:16:47.550577 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self.creator() Jul 21 08:16:47.550577 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 08:16:47.550577 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application created_value = creator( Jul 21 08:16:47.550577 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 21 08:16:47.550577 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 21 08:16:47.550577 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 21 08:16:47.550577 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 21 08:16:47.550577 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 21 08:16:47.550577 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 21 08:16:47.550577 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 21 08:16:47.550577 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application Jul 21 08:16:47.550577 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 186/368] 10.222.0.47 () {56 vars in 1096 bytes} [Thu Jul 21 08:16:47 2022] GET /identity/v3/roles/admin => generated 83 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 08:16:47.551226 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0db41ccf-5151-43de-af18-c491f296133f None admin] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:16:47.619277 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6938b02-11fd-4e56-8181-229b7b92731c None admin] Validating token access rules against request {{(pid=80245) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:16:47.658735 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b62acb3-8d1b-4c53-a947-8b7f144cb520 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80245) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:16:47.660114 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b62acb3-8d1b-4c53-a947-8b7f144cb520 None admin] REQUEST_METHOD: `GET` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:16:47.660600 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b62acb3-8d1b-4c53-a947-8b7f144cb520 None admin] SCRIPT_NAME: `/identity` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:16:47.661087 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b62acb3-8d1b-4c53-a947-8b7f144cb520 None admin] PATH_INFO: `/v3/projects/d4fa4cba54f348b68df92b37fb5fd5d1` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:16:47.664460 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b62acb3-8d1b-4c53-a947-8b7f144cb520 None admin] RBAC: Authorizing `identity:get_project(project_id=d4fa4cba54f348b68df92b37fb5fd5d1)` {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:16:47.667455 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b62acb3-8d1b-4c53-a947-8b7f144cb520 None admin] RBAC: Authorization granted {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:16:47.670754 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: [pid: 80245|app: 0|req: 183/369] 10.222.0.47 () {56 vars in 1186 bytes} [Thu Jul 21 08:16:47 2022] GET /identity/v3/projects/d4fa4cba54f348b68df92b37fb5fd5d1 => generated 302 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 08:16:47.671881 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b62acb3-8d1b-4c53-a947-8b7f144cb520 None admin] Authenticating user token {{(pid=80245) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:16:47.686058 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0db41ccf-5151-43de-af18-c491f296133f None admin] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:16:47.727388 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02697f05-14aa-456a-9313-d13c317efb41 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:16:47.728981 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02697f05-14aa-456a-9313-d13c317efb41 None admin] REQUEST_METHOD: `GET` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:16:47.729510 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02697f05-14aa-456a-9313-d13c317efb41 None admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:16:47.730158 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02697f05-14aa-456a-9313-d13c317efb41 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:16:47.735051 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG dbcounter [-] [80244] Writing DB stats keystone:SELECT=100 {{(pid=80244) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:16:47.780878 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG dbcounter [-] [80245] Writing DB stats keystone:SELECT=100 {{(pid=80245) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:16:47.804920 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b62acb3-8d1b-4c53-a947-8b7f144cb520 None admin] Validating token access rules against request {{(pid=80245) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:16:47.831971 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02697f05-14aa-456a-9313-d13c317efb41 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:16:47.834177 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02697f05-14aa-456a-9313-d13c317efb41 None admin] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:16:47.842211 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7dc883d-afc3-4943-a12d-4c5972f429c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80245) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:16:47.843250 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7dc883d-afc3-4943-a12d-4c5972f429c5 None admin] REQUEST_METHOD: `GET` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:16:47.843771 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7dc883d-afc3-4943-a12d-4c5972f429c5 None admin] SCRIPT_NAME: `/identity` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:16:47.844255 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7dc883d-afc3-4943-a12d-4c5972f429c5 None admin] PATH_INFO: `/v3/roles` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:16:47.845250 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7dc883d-afc3-4943-a12d-4c5972f429c5 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80245) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 08:16:47.845777 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7dc883d-afc3-4943-a12d-4c5972f429c5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:16:47.847414 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7dc883d-afc3-4943-a12d-4c5972f429c5 None admin] RBAC: Authorization granted {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:16:47.860896 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: [pid: 80245|app: 0|req: 184/370] 10.222.0.47 () {56 vars in 1099 bytes} [Thu Jul 21 08:16:47 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 08:16:47.862185 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7dc883d-afc3-4943-a12d-4c5972f429c5 None admin] Authenticating user token {{(pid=80245) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:16:47.952657 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 187/371] 10.222.0.47 () {58 vars in 1282 bytes} [Thu Jul 21 08:16:47 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 402 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 21 08:16:47.953345 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02697f05-14aa-456a-9313-d13c317efb41 None admin] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:16:47.987279 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7dc883d-afc3-4943-a12d-4c5972f429c5 None admin] Validating token access rules against request {{(pid=80245) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:16:47.989731 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG dbcounter [-] [80244] Writing DB stats keystone:SELECT=100 {{(pid=80244) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:16:48.025927 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d34e33f-bce8-49c4-a67d-af8531eed172 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80245) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:16:48.027200 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d34e33f-bce8-49c4-a67d-af8531eed172 None admin] REQUEST_METHOD: `GET` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:16:48.027712 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d34e33f-bce8-49c4-a67d-af8531eed172 None admin] SCRIPT_NAME: `/identity` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:16:48.028195 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d34e33f-bce8-49c4-a67d-af8531eed172 None admin] PATH_INFO: `/v3/groups/f75bd62425144927b6fd534595218714` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:16:48.029581 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d34e33f-bce8-49c4-a67d-af8531eed172 None admin] RBAC: Authorizing `identity:get_group(group_id=f75bd62425144927b6fd534595218714)` {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:16:48.031231 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d34e33f-bce8-49c4-a67d-af8531eed172 None admin] RBAC: Authorization granted {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:16:48.033445 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: [pid: 80245|app: 0|req: 185/372] 10.222.0.47 () {56 vars in 1180 bytes} [Thu Jul 21 08:16:47 2022] GET /identity/v3/groups/f75bd62425144927b6fd534595218714 => generated 226 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 08:16:48.034517 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d34e33f-bce8-49c4-a67d-af8531eed172 None admin] Authenticating user token {{(pid=80245) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:16:48.051350 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG dbcounter [-] [80245] Writing DB stats keystone:SELECT=100 {{(pid=80245) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:16:48.061201 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02697f05-14aa-456a-9313-d13c317efb41 None admin] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:16:48.097914 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91b24f13-4b44-436e-9701-37857fbbc0bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:16:48.098577 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91b24f13-4b44-436e-9701-37857fbbc0bb None admin] REQUEST_METHOD: `GET` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:16:48.098861 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91b24f13-4b44-436e-9701-37857fbbc0bb None admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:16:48.099176 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91b24f13-4b44-436e-9701-37857fbbc0bb None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:16:48.145343 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d34e33f-bce8-49c4-a67d-af8531eed172 None admin] Validating token access rules against request {{(pid=80245) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:16:48.172675 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91b24f13-4b44-436e-9701-37857fbbc0bb None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:16:48.174184 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91b24f13-4b44-436e-9701-37857fbbc0bb None admin] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:16:48.183516 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1c8e6ba-9949-4823-beca-5523660c3bbc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80245) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:16:48.184815 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1c8e6ba-9949-4823-beca-5523660c3bbc None admin] REQUEST_METHOD: `GET` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:16:48.185297 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1c8e6ba-9949-4823-beca-5523660c3bbc None admin] SCRIPT_NAME: `/identity` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:16:48.185774 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1c8e6ba-9949-4823-beca-5523660c3bbc None admin] PATH_INFO: `/v3/projects/d4fa4cba54f348b68df92b37fb5fd5d1` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:16:48.187505 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1c8e6ba-9949-4823-beca-5523660c3bbc None admin] RBAC: Authorizing `identity:get_project(project_id=d4fa4cba54f348b68df92b37fb5fd5d1)` {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:16:48.189162 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1c8e6ba-9949-4823-beca-5523660c3bbc None admin] RBAC: Authorization granted {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:16:48.191405 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: [pid: 80245|app: 0|req: 186/373] 10.222.0.47 () {56 vars in 1186 bytes} [Thu Jul 21 08:16:48 2022] GET /identity/v3/projects/d4fa4cba54f348b68df92b37fb5fd5d1 => generated 302 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 08:16:48.192411 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1c8e6ba-9949-4823-beca-5523660c3bbc None admin] Authenticating user token {{(pid=80245) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:16:48.214977 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG dbcounter [-] [80244] Writing DB stats keystone:SELECT=100 {{(pid=80244) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:16:48.288330 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 188/374] 10.222.0.47 () {58 vars in 1282 bytes} [Thu Jul 21 08:16:47 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 334 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 21 08:16:48.289568 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91b24f13-4b44-436e-9701-37857fbbc0bb None admin] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:16:48.293869 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG dbcounter [-] [80245] Writing DB stats keystone:SELECT=100 {{(pid=80245) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:16:48.315497 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1c8e6ba-9949-4823-beca-5523660c3bbc None admin] Validating token access rules against request {{(pid=80245) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:16:48.354086 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d67ff235-9e55-4130-8f6d-6bb26d0a9116 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80245) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:16:48.354754 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d67ff235-9e55-4130-8f6d-6bb26d0a9116 None admin] REQUEST_METHOD: `GET` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:16:48.355030 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d67ff235-9e55-4130-8f6d-6bb26d0a9116 None admin] SCRIPT_NAME: `/identity` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:16:48.355394 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d67ff235-9e55-4130-8f6d-6bb26d0a9116 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:16:48.356157 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d67ff235-9e55-4130-8f6d-6bb26d0a9116 None admin] RBAC: Adding query filter params (group.id=f75bd62425144927b6fd534595218714, role.id=3a074738221e4601b5205715ba87d240, scope.project.id=d4fa4cba54f348b68df92b37fb5fd5d1) {{(pid=80245) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 08:16:48.356531 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d67ff235-9e55-4130-8f6d-6bb26d0a9116 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:16:48.358272 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d67ff235-9e55-4130-8f6d-6bb26d0a9116 None admin] RBAC: Authorization granted {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:16:48.365120 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: [pid: 80245|app: 0|req: 187/375] 10.222.0.47 () {56 vars in 1376 bytes} [Thu Jul 21 08:16:48 2022] GET /identity/v3/role_assignments?group.id=f75bd62425144927b6fd534595218714&scope.project.id=d4fa4cba54f348b68df92b37fb5fd5d1&role.id=3a074738221e4601b5205715ba87d240 => generated 260 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 08:16:48.365957 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d67ff235-9e55-4130-8f6d-6bb26d0a9116 None admin] Authenticating user token {{(pid=80245) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:16:48.400703 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91b24f13-4b44-436e-9701-37857fbbc0bb None admin] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:16:48.438924 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f06bb88e-00bc-45ae-8e90-890d1350886d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:16:48.440181 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f06bb88e-00bc-45ae-8e90-890d1350886d None admin] REQUEST_METHOD: `GET` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:16:48.440683 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f06bb88e-00bc-45ae-8e90-890d1350886d None admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:16:48.441169 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f06bb88e-00bc-45ae-8e90-890d1350886d None admin] PATH_INFO: `/v3/groups/f75bd62425144927b6fd534595218714` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:16:48.442559 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f06bb88e-00bc-45ae-8e90-890d1350886d None admin] RBAC: Authorizing `identity:get_group(group_id=f75bd62425144927b6fd534595218714)` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:16:48.444918 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f06bb88e-00bc-45ae-8e90-890d1350886d None admin] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:16:48.447117 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 189/376] 10.222.0.47 () {56 vars in 1180 bytes} [Thu Jul 21 08:16:48 2022] GET /identity/v3/groups/f75bd62425144927b6fd534595218714 => generated 226 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 08:16:48.454069 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f06bb88e-00bc-45ae-8e90-890d1350886d None admin] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:16:48.477250 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG dbcounter [-] [80244] Writing DB stats keystone:SELECT=100 {{(pid=80244) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:16:48.533384 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d67ff235-9e55-4130-8f6d-6bb26d0a9116 None admin] Validating token access rules against request {{(pid=80245) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:16:48.614280 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c893103-0284-424b-8306-40b9c085b78d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80245) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:16:48.615765 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c893103-0284-424b-8306-40b9c085b78d None admin] REQUEST_METHOD: `GET` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:16:48.616107 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c893103-0284-424b-8306-40b9c085b78d None admin] SCRIPT_NAME: `/identity` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:16:48.616350 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c893103-0284-424b-8306-40b9c085b78d None admin] PATH_INFO: `/v3/projects/42d15fc0515945acacbbffcf2356c7a5` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:16:48.618951 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c893103-0284-424b-8306-40b9c085b78d None admin] RBAC: Authorizing `identity:get_project(project_id=42d15fc0515945acacbbffcf2356c7a5)` {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:16:48.620539 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c893103-0284-424b-8306-40b9c085b78d None admin] RBAC: Authorization granted {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:16:48.622786 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: [pid: 80245|app: 0|req: 188/377] 10.222.0.47 () {56 vars in 1186 bytes} [Thu Jul 21 08:16:48 2022] GET /identity/v3/projects/42d15fc0515945acacbbffcf2356c7a5 => generated 344 bytes in 257 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 08:16:48.628270 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c893103-0284-424b-8306-40b9c085b78d None admin] Authenticating user token {{(pid=80245) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:16:48.652555 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG dbcounter [-] [80245] Writing DB stats keystone:SELECT=100 {{(pid=80245) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:16:48.654513 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f06bb88e-00bc-45ae-8e90-890d1350886d None admin] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:16:48.698748 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33200ace-1408-4be6-abeb-7e2b99e92bb5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:16:48.702130 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33200ace-1408-4be6-abeb-7e2b99e92bb5 None admin] REQUEST_METHOD: `GET` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:16:48.702130 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33200ace-1408-4be6-abeb-7e2b99e92bb5 None admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:16:48.702130 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33200ace-1408-4be6-abeb-7e2b99e92bb5 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:16:48.702130 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33200ace-1408-4be6-abeb-7e2b99e92bb5 None admin] RBAC: Adding query filter params (group.id=f75bd62425144927b6fd534595218714, role.id=99a638c0aac44b79a1f2858574724495, scope.project.id=d4fa4cba54f348b68df92b37fb5fd5d1) {{(pid=80244) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 08:16:48.702130 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33200ace-1408-4be6-abeb-7e2b99e92bb5 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:16:48.703081 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33200ace-1408-4be6-abeb-7e2b99e92bb5 None admin] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:16:48.711113 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 190/378] 10.222.0.47 () {56 vars in 1376 bytes} [Thu Jul 21 08:16:48 2022] GET /identity/v3/role_assignments?group.id=f75bd62425144927b6fd534595218714&scope.project.id=d4fa4cba54f348b68df92b37fb5fd5d1&role.id=99a638c0aac44b79a1f2858574724495 => generated 260 bytes in 256 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 08:16:48.712745 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0324614-ae4c-46f2-b019-c9dd51347e91 None None] REQUEST_METHOD: `GET` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:16:48.713946 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0324614-ae4c-46f2-b019-c9dd51347e91 None None] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:16:48.714537 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0324614-ae4c-46f2-b019-c9dd51347e91 None None] PATH_INFO: `/` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:16:48.717436 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 191/379] 10.222.0.47 () {54 vars in 926 bytes} [Thu Jul 21 08:16:48 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 08:16:48.736230 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e785872-d9cc-49a1-8900-867a90f7b443 None None] REQUEST_METHOD: `POST` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:16:48.736772 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e785872-d9cc-49a1-8900-867a90f7b443 None None] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:16:48.737188 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e785872-d9cc-49a1-8900-867a90f7b443 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:16:48.771382 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.auth.core [None req-3e785872-d9cc-49a1-8900-867a90f7b443 None None] MFA Rules not processed for user `2dce40b8d74c41ec94cc36e6ff05d412`. Rule list: `[]` (Enabled: `True`). {{(pid=80244) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 08:16:48.773069 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.fernet_utils [None req-3e785872-d9cc-49a1-8900-867a90f7b443 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80244) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 08:16:48.817247 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c893103-0284-424b-8306-40b9c085b78d None admin] Validating token access rules against request {{(pid=80245) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:16:48.867633 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af3f5861-d887-4869-836f-77b79d55abaa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80245) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:16:48.869285 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af3f5861-d887-4869-836f-77b79d55abaa None admin] REQUEST_METHOD: `GET` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:16:48.869729 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af3f5861-d887-4869-836f-77b79d55abaa None admin] SCRIPT_NAME: `/identity` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:16:48.870145 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af3f5861-d887-4869-836f-77b79d55abaa None admin] PATH_INFO: `/v3/groups/569a366732204da3a5ab12f694275b62` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:16:48.890668 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af3f5861-d887-4869-836f-77b79d55abaa None admin] RBAC: Authorizing `identity:get_group(group_id=569a366732204da3a5ab12f694275b62)` {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:16:48.892498 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af3f5861-d887-4869-836f-77b79d55abaa None admin] RBAC: Authorization granted {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:16:48.895626 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: [pid: 80245|app: 0|req: 189/380] 10.222.0.47 () {56 vars in 1180 bytes} [Thu Jul 21 08:16:48 2022] GET /identity/v3/groups/569a366732204da3a5ab12f694275b62 => generated 229 bytes in 268 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 08:16:48.901748 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af3f5861-d887-4869-836f-77b79d55abaa None admin] Authenticating user token {{(pid=80245) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:16:48.923397 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG dbcounter [-] [80244] Writing DB stats keystone:SELECT=100 {{(pid=80244) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:16:48.932234 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 192/381] 10.222.0.47 () {58 vars in 1023 bytes} [Thu Jul 21 08:16:48 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 197 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 21 08:16:48.958816 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e785872-d9cc-49a1-8900-867a90f7b443 None None] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:16:49.002560 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG dbcounter [-] [80245] Writing DB stats keystone:SELECT=100 {{(pid=80245) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:16:49.042404 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af3f5861-d887-4869-836f-77b79d55abaa None admin] Validating token access rules against request {{(pid=80245) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:16:49.104697 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8448a569-73d0-4b17-b041-60c1b159bd1e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80245) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:16:49.105386 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8448a569-73d0-4b17-b041-60c1b159bd1e None admin] REQUEST_METHOD: `GET` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:16:49.105673 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8448a569-73d0-4b17-b041-60c1b159bd1e None admin] SCRIPT_NAME: `/identity` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:16:49.105952 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8448a569-73d0-4b17-b041-60c1b159bd1e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:16:49.106794 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8448a569-73d0-4b17-b041-60c1b159bd1e None admin] RBAC: Adding query filter params (group.id=569a366732204da3a5ab12f694275b62, role.id=e29870f6e9e2411a9db0d5bf45f32ca5, scope.project.id=42d15fc0515945acacbbffcf2356c7a5) {{(pid=80245) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 08:16:49.107110 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8448a569-73d0-4b17-b041-60c1b159bd1e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:16:49.108592 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8448a569-73d0-4b17-b041-60c1b159bd1e None admin] RBAC: Authorization granted {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:16:49.122905 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: [pid: 80245|app: 0|req: 190/382] 10.222.0.47 () {56 vars in 1376 bytes} [Thu Jul 21 08:16:48 2022] GET /identity/v3/role_assignments?group.id=569a366732204da3a5ab12f694275b62&scope.project.id=42d15fc0515945acacbbffcf2356c7a5&role.id=e29870f6e9e2411a9db0d5bf45f32ca5 => generated 260 bytes in 222 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 21 08:16:49.125352 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e785872-d9cc-49a1-8900-867a90f7b443 None None] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:16:49.164208 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3c9b670-2a62-4746-badb-d21322825ad2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:16:49.165412 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3c9b670-2a62-4746-badb-d21322825ad2 None admin] REQUEST_METHOD: `GET` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:16:49.165904 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3c9b670-2a62-4746-badb-d21322825ad2 None admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:16:49.166400 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3c9b670-2a62-4746-badb-d21322825ad2 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:16:49.178826 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3c9b670-2a62-4746-badb-d21322825ad2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 21 08:16:49.187373 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application [None req-a3c9b670-2a62-4746-badb-d21322825ad2 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 21 08:16:49.187373 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 08:16:49.187373 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 08:16:49.187373 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 08:16:49.187373 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 08:16:49.187373 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 08:16:49.187373 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 08:16:49.187373 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 08:16:49.187373 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 08:16:49.187373 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 08:16:49.187373 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 08:16:49.187373 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 08:16:49.187373 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 21 08:16:49.187373 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 21 08:16:49.187373 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 21 08:16:49.187373 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 21 08:16:49.188302 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 21 08:16:49.188302 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 21 08:16:49.188302 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 21 08:16:49.188302 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 21 08:16:49.188302 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 08:16:49.188302 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 08:16:49.188302 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 08:16:49.188302 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 08:16:49.188302 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 08:16:49.188302 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 08:16:49.188302 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 08:16:49.188302 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application with Lock( Jul 21 08:16:49.188302 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 08:16:49.188302 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self._enter() Jul 21 08:16:49.188302 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 08:16:49.188302 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 08:16:49.189062 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 08:16:49.189062 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self.creator() Jul 21 08:16:49.189062 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 08:16:49.189062 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application created_value = creator( Jul 21 08:16:49.189062 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 21 08:16:49.189062 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 21 08:16:49.189062 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 21 08:16:49.189062 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 21 08:16:49.189062 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 21 08:16:49.189062 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 21 08:16:49.189062 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 21 08:16:49.189062 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application Jul 21 08:16:49.191931 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 193/383] 10.222.0.47 () {56 vars in 1099 bytes} [Thu Jul 21 08:16:48 2022] GET /identity/v3/roles/member => generated 84 bytes in 234 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 08:16:49.200436 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8448a569-73d0-4b17-b041-60c1b159bd1e None admin] Authenticating user token {{(pid=80245) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:16:49.330478 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8448a569-73d0-4b17-b041-60c1b159bd1e None admin] Validating token access rules against request {{(pid=80245) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:16:49.374158 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b776534-0df0-46e2-9570-36877730d895 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80245) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:16:49.374817 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b776534-0df0-46e2-9570-36877730d895 None admin] REQUEST_METHOD: `GET` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:16:49.375108 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b776534-0df0-46e2-9570-36877730d895 None admin] SCRIPT_NAME: `/identity` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:16:49.375410 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b776534-0df0-46e2-9570-36877730d895 None admin] PATH_INFO: `/v3/roles` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:16:49.376178 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b776534-0df0-46e2-9570-36877730d895 None admin] RBAC: Adding query filter params (name=member) {{(pid=80245) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 08:16:49.376489 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b776534-0df0-46e2-9570-36877730d895 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:16:49.377898 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b776534-0df0-46e2-9570-36877730d895 None admin] RBAC: Authorization granted {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:16:49.392514 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: [pid: 80245|app: 0|req: 191/384] 10.222.0.47 () {56 vars in 1101 bytes} [Thu Jul 21 08:16:49 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 08:16:49.400480 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3c9b670-2a62-4746-badb-d21322825ad2 None admin] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:16:49.438457 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd7c6c78-ad21-4b58-ba6b-4fd50e84daf8 None None] REQUEST_METHOD: `GET` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:16:49.438816 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd7c6c78-ad21-4b58-ba6b-4fd50e84daf8 None None] SCRIPT_NAME: `/identity` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:16:49.439083 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd7c6c78-ad21-4b58-ba6b-4fd50e84daf8 None None] PATH_INFO: `/` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:16:49.441127 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: [pid: 80245|app: 0|req: 192/385] 10.222.0.47 () {54 vars in 926 bytes} [Thu Jul 21 08:16:49 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 21 08:16:49.447616 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba4ac028-d55b-4fd9-921c-68ff7e9b9dce None None] REQUEST_METHOD: `POST` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:16:49.447897 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba4ac028-d55b-4fd9-921c-68ff7e9b9dce None None] SCRIPT_NAME: `/identity` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:16:49.448186 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba4ac028-d55b-4fd9-921c-68ff7e9b9dce None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:16:49.454674 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG dbcounter [-] [80245] Writing DB stats keystone:SELECT=100 {{(pid=80245) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:16:49.485731 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.auth.core [None req-ba4ac028-d55b-4fd9-921c-68ff7e9b9dce None None] MFA Rules not processed for user `2dce40b8d74c41ec94cc36e6ff05d412`. Rule list: `[]` (Enabled: `True`). {{(pid=80245) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 08:16:49.486994 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.fernet_utils [None req-ba4ac028-d55b-4fd9-921c-68ff7e9b9dce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80245) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 08:16:49.519288 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG dbcounter [-] [80244] Writing DB stats keystone:SELECT=100 {{(pid=80244) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:16:49.602127 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3c9b670-2a62-4746-badb-d21322825ad2 None admin] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:16:49.610783 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: [pid: 80245|app: 0|req: 193/386] 10.222.0.47 () {58 vars in 1023 bytes} [Thu Jul 21 08:16:49 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 21 08:16:49.644480 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba4ac028-d55b-4fd9-921c-68ff7e9b9dce None None] Authenticating user token {{(pid=80245) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:16:49.652979 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c10130fd-48a8-402b-b10d-c404092f98ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:16:49.654173 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c10130fd-48a8-402b-b10d-c404092f98ac None admin] REQUEST_METHOD: `GET` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:16:49.654652 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c10130fd-48a8-402b-b10d-c404092f98ac None admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:16:49.655185 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c10130fd-48a8-402b-b10d-c404092f98ac None admin] PATH_INFO: `/v3/groups/f75bd62425144927b6fd534595218714` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:16:49.656615 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c10130fd-48a8-402b-b10d-c404092f98ac None admin] RBAC: Authorizing `identity:get_group(group_id=f75bd62425144927b6fd534595218714)` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:16:49.658292 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c10130fd-48a8-402b-b10d-c404092f98ac None admin] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:16:49.660487 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 194/387] 10.222.0.47 () {56 vars in 1180 bytes} [Thu Jul 21 08:16:49 2022] GET /identity/v3/groups/f75bd62425144927b6fd534595218714 => generated 226 bytes in 260 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 08:16:49.665899 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c10130fd-48a8-402b-b10d-c404092f98ac None admin] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:16:49.827940 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba4ac028-d55b-4fd9-921c-68ff7e9b9dce None None] Validating token access rules against request {{(pid=80245) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:16:49.830110 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c10130fd-48a8-402b-b10d-c404092f98ac None admin] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:16:49.869682 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG dbcounter [-] [80244] Writing DB stats keystone:SELECT=100 {{(pid=80244) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:16:49.883833 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f13409f-437a-4725-b482-7df6b2f84c9c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:16:49.884701 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f13409f-437a-4725-b482-7df6b2f84c9c None admin] REQUEST_METHOD: `GET` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:16:49.884987 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f13409f-437a-4725-b482-7df6b2f84c9c None admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:16:49.885261 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f13409f-437a-4725-b482-7df6b2f84c9c None admin] PATH_INFO: `/v3/projects/eab7e5ec54c94574ad68c89747ad5bcc` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:16:49.886940 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f13409f-437a-4725-b482-7df6b2f84c9c None admin] RBAC: Authorizing `identity:get_project(project_id=eab7e5ec54c94574ad68c89747ad5bcc)` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:16:49.888461 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f13409f-437a-4725-b482-7df6b2f84c9c None admin] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:16:49.890969 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e70cbdf3-4a4d-4172-926d-6765c2d61e45 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80245) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:16:49.891907 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e70cbdf3-4a4d-4172-926d-6765c2d61e45 None admin] REQUEST_METHOD: `GET` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:16:49.892192 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e70cbdf3-4a4d-4172-926d-6765c2d61e45 None admin] SCRIPT_NAME: `/identity` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:16:49.892462 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e70cbdf3-4a4d-4172-926d-6765c2d61e45 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:16:49.895623 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 195/388] 10.222.0.47 () {56 vars in 1186 bytes} [Thu Jul 21 08:16:49 2022] GET /identity/v3/projects/eab7e5ec54c94574ad68c89747ad5bcc => generated 298 bytes in 228 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 21 08:16:49.896167 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG dbcounter [-] [80245] Writing DB stats keystone:SELECT=100 {{(pid=80245) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:16:49.896503 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f13409f-437a-4725-b482-7df6b2f84c9c None admin] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:16:49.914081 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e70cbdf3-4a4d-4172-926d-6765c2d61e45 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 21 08:16:49.922432 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application [None req-e70cbdf3-4a4d-4172-926d-6765c2d61e45 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 21 08:16:49.922432 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 08:16:49.922432 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 08:16:49.922432 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 08:16:49.922432 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 08:16:49.922432 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 08:16:49.922432 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 08:16:49.922432 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 08:16:49.922432 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 08:16:49.922432 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 08:16:49.922432 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 08:16:49.922432 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 08:16:49.922432 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 21 08:16:49.922432 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 21 08:16:49.922432 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 21 08:16:49.922432 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 21 08:16:49.923375 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 21 08:16:49.923375 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 21 08:16:49.923375 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 21 08:16:49.923375 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 21 08:16:49.923375 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 08:16:49.923375 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 08:16:49.923375 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 08:16:49.923375 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 08:16:49.923375 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 08:16:49.923375 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 08:16:49.923375 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 08:16:49.923375 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application with Lock( Jul 21 08:16:49.923375 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 08:16:49.923375 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application return self._enter() Jul 21 08:16:49.923375 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 08:16:49.923375 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 08:16:49.924162 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 08:16:49.924162 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application return self.creator() Jul 21 08:16:49.924162 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 08:16:49.924162 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application created_value = creator( Jul 21 08:16:49.924162 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 21 08:16:49.924162 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 21 08:16:49.924162 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 21 08:16:49.924162 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 21 08:16:49.924162 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 21 08:16:49.924162 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 21 08:16:49.924162 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 21 08:16:49.924162 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application Jul 21 08:16:49.924162 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: [pid: 80245|app: 0|req: 194/389] 10.222.0.47 () {56 vars in 1114 bytes} [Thu Jul 21 08:16:49 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 280 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 21 08:16:49.937545 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e70cbdf3-4a4d-4172-926d-6765c2d61e45 None admin] Authenticating user token {{(pid=80245) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:16:50.065584 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e70cbdf3-4a4d-4172-926d-6765c2d61e45 None admin] Validating token access rules against request {{(pid=80245) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:16:50.099477 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f13409f-437a-4725-b482-7df6b2f84c9c None admin] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:16:50.110056 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5507fd93-f2e9-4c80-8adb-3dbc1faf388f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80245) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:16:50.111569 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5507fd93-f2e9-4c80-8adb-3dbc1faf388f None admin] REQUEST_METHOD: `GET` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:16:50.112064 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5507fd93-f2e9-4c80-8adb-3dbc1faf388f None admin] SCRIPT_NAME: `/identity` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:16:50.112925 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5507fd93-f2e9-4c80-8adb-3dbc1faf388f None admin] PATH_INFO: `/v3/roles` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:16:50.114184 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5507fd93-f2e9-4c80-8adb-3dbc1faf388f None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80245) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 08:16:50.115230 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5507fd93-f2e9-4c80-8adb-3dbc1faf388f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:16:50.117744 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5507fd93-f2e9-4c80-8adb-3dbc1faf388f None admin] RBAC: Authorization granted {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:16:50.127961 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: [pid: 80245|app: 0|req: 195/390] 10.222.0.47 () {56 vars in 1111 bytes} [Thu Jul 21 08:16:49 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 08:16:50.134747 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5507fd93-f2e9-4c80-8adb-3dbc1faf388f None admin] Authenticating user token {{(pid=80245) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:16:50.145285 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1da88251-b1b4-49d7-881e-c306c847afa6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:16:50.146391 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1da88251-b1b4-49d7-881e-c306c847afa6 None admin] REQUEST_METHOD: `PUT` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:16:50.146620 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1da88251-b1b4-49d7-881e-c306c847afa6 None admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:16:50.146894 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1da88251-b1b4-49d7-881e-c306c847afa6 None admin] PATH_INFO: `/v3/projects/eab7e5ec54c94574ad68c89747ad5bcc/groups/f75bd62425144927b6fd534595218714/roles/3a074738221e4601b5205715ba87d240` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:16:50.151000 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1da88251-b1b4-49d7-881e-c306c847afa6 None admin] RBAC: Authorizing `identity:create_grant(project_id=eab7e5ec54c94574ad68c89747ad5bcc, group_id=f75bd62425144927b6fd534595218714, role_id=3a074738221e4601b5205715ba87d240)` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:16:50.151000 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1da88251-b1b4-49d7-881e-c306c847afa6 None admin] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:16:50.168012 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 196/391] 10.222.0.47 () {58 vars in 1442 bytes} [Thu Jul 21 08:16:49 2022] PUT /identity/v3/projects/eab7e5ec54c94574ad68c89747ad5bcc/groups/f75bd62425144927b6fd534595218714/roles/3a074738221e4601b5205715ba87d240 => generated 0 bytes in 272 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 08:16:50.212063 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG dbcounter [-] [80245] Writing DB stats keystone:SELECT=100 {{(pid=80245) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:16:50.268955 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5507fd93-f2e9-4c80-8adb-3dbc1faf388f None admin] Validating token access rules against request {{(pid=80245) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:16:50.309601 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10936d1d-406b-4c5a-bcd6-6a9a9f0cf11f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80245) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:16:50.310993 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10936d1d-406b-4c5a-bcd6-6a9a9f0cf11f None admin] REQUEST_METHOD: `GET` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:16:50.311528 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10936d1d-406b-4c5a-bcd6-6a9a9f0cf11f None admin] SCRIPT_NAME: `/identity` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:16:50.312002 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10936d1d-406b-4c5a-bcd6-6a9a9f0cf11f None admin] PATH_INFO: `/v3/groups/f75bd62425144927b6fd534595218714` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:16:50.313496 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10936d1d-406b-4c5a-bcd6-6a9a9f0cf11f None admin] RBAC: Authorizing `identity:get_group(group_id=f75bd62425144927b6fd534595218714)` {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:16:50.315163 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10936d1d-406b-4c5a-bcd6-6a9a9f0cf11f None admin] RBAC: Authorization granted {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:16:50.317421 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: [pid: 80245|app: 0|req: 196/392] 10.222.0.47 () {56 vars in 1180 bytes} [Thu Jul 21 08:16:50 2022] GET /identity/v3/groups/f75bd62425144927b6fd534595218714 => generated 226 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 21 08:16:50.325436 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1da88251-b1b4-49d7-881e-c306c847afa6 None admin] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:16:50.395120 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG dbcounter [-] [80244] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80244) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:16:50.447559 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1da88251-b1b4-49d7-881e-c306c847afa6 None admin] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:16:50.484748 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8bc6dd4-4125-4897-8272-8607a6f50244 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:16:50.485956 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8bc6dd4-4125-4897-8272-8607a6f50244 None admin] REQUEST_METHOD: `GET` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:16:50.486429 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8bc6dd4-4125-4897-8272-8607a6f50244 None admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:16:50.486960 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8bc6dd4-4125-4897-8272-8607a6f50244 None admin] PATH_INFO: `/v3/projects/eab7e5ec54c94574ad68c89747ad5bcc` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:16:50.488456 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8bc6dd4-4125-4897-8272-8607a6f50244 None admin] RBAC: Authorizing `identity:get_project(project_id=eab7e5ec54c94574ad68c89747ad5bcc)` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:16:50.490695 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8bc6dd4-4125-4897-8272-8607a6f50244 None admin] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:16:50.493089 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 197/393] 10.222.0.47 () {56 vars in 1186 bytes} [Thu Jul 21 08:16:50 2022] GET /identity/v3/projects/eab7e5ec54c94574ad68c89747ad5bcc => generated 298 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 08:16:50.498958 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10936d1d-406b-4c5a-bcd6-6a9a9f0cf11f None admin] Authenticating user token {{(pid=80245) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:16:50.606858 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10936d1d-406b-4c5a-bcd6-6a9a9f0cf11f None admin] Validating token access rules against request {{(pid=80245) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:16:50.631864 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG dbcounter [-] [80245] Writing DB stats keystone:SELECT=100 {{(pid=80245) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:16:50.644695 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9627f306-ce9d-407b-9a2c-a93ce20f1533 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80245) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:16:50.645591 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9627f306-ce9d-407b-9a2c-a93ce20f1533 None admin] REQUEST_METHOD: `PUT` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:16:50.646461 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9627f306-ce9d-407b-9a2c-a93ce20f1533 None admin] SCRIPT_NAME: `/identity` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:16:50.646736 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9627f306-ce9d-407b-9a2c-a93ce20f1533 None admin] PATH_INFO: `/v3/projects/eab7e5ec54c94574ad68c89747ad5bcc/groups/f75bd62425144927b6fd534595218714/roles/99a638c0aac44b79a1f2858574724495` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:16:50.649033 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9627f306-ce9d-407b-9a2c-a93ce20f1533 None admin] RBAC: Authorizing `identity:create_grant(project_id=eab7e5ec54c94574ad68c89747ad5bcc, group_id=f75bd62425144927b6fd534595218714, role_id=99a638c0aac44b79a1f2858574724495)` {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:16:50.650474 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9627f306-ce9d-407b-9a2c-a93ce20f1533 None admin] RBAC: Authorization granted {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:16:50.672936 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: [pid: 80245|app: 0|req: 197/394] 10.222.0.47 () {58 vars in 1442 bytes} [Thu Jul 21 08:16:50 2022] PUT /identity/v3/projects/eab7e5ec54c94574ad68c89747ad5bcc/groups/f75bd62425144927b6fd534595218714/roles/99a638c0aac44b79a1f2858574724495 => generated 0 bytes in 174 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 08:16:51.019981 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d33ebd38-55f1-4bf1-aa77-cbbae713de24 None None] REQUEST_METHOD: `GET` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:16:51.020662 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d33ebd38-55f1-4bf1-aa77-cbbae713de24 None None] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:16:51.020662 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d33ebd38-55f1-4bf1-aa77-cbbae713de24 None None] PATH_INFO: `/` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:16:51.022631 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 198/395] 10.222.0.47 () {54 vars in 926 bytes} [Thu Jul 21 08:16:51 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 08:16:51.028228 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73ba7fe8-748a-46b0-9c2b-647b9730fd90 None None] REQUEST_METHOD: `POST` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:16:51.028531 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73ba7fe8-748a-46b0-9c2b-647b9730fd90 None None] SCRIPT_NAME: `/identity` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:16:51.028816 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73ba7fe8-748a-46b0-9c2b-647b9730fd90 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:16:51.049327 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.auth.core [None req-73ba7fe8-748a-46b0-9c2b-647b9730fd90 None None] MFA Rules not processed for user `2dce40b8d74c41ec94cc36e6ff05d412`. Rule list: `[]` (Enabled: `True`). {{(pid=80245) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 08:16:51.050525 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.fernet_utils [None req-73ba7fe8-748a-46b0-9c2b-647b9730fd90 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80245) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 08:16:51.135822 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: [pid: 80245|app: 0|req: 198/396] 10.222.0.47 () {58 vars in 1023 bytes} [Thu Jul 21 08:16:51 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 21 08:16:51.177219 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d33ebd38-55f1-4bf1-aa77-cbbae713de24 None None] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:16:51.360717 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d33ebd38-55f1-4bf1-aa77-cbbae713de24 None None] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:16:51.411385 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG dbcounter [-] [80244] Writing DB stats keystone:SELECT=100 {{(pid=80244) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:16:51.429542 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b96f149-4af2-4616-8885-e8753abf6c79 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:16:51.430361 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b96f149-4af2-4616-8885-e8753abf6c79 None admin] REQUEST_METHOD: `GET` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:16:51.430644 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b96f149-4af2-4616-8885-e8753abf6c79 None admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:16:51.430918 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b96f149-4af2-4616-8885-e8753abf6c79 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:16:51.448281 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b96f149-4af2-4616-8885-e8753abf6c79 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 21 08:16:51.456177 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application [None req-3b96f149-4af2-4616-8885-e8753abf6c79 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 21 08:16:51.456177 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 08:16:51.456177 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 08:16:51.456177 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 08:16:51.456177 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 08:16:51.456177 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 08:16:51.456177 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 08:16:51.456177 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 08:16:51.456177 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 08:16:51.456177 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 08:16:51.456177 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 08:16:51.456177 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 08:16:51.456177 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 21 08:16:51.456177 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 21 08:16:51.456177 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 21 08:16:51.456177 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 21 08:16:51.457092 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 21 08:16:51.457092 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 21 08:16:51.457092 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 21 08:16:51.457092 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 21 08:16:51.457092 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 08:16:51.457092 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 08:16:51.457092 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 08:16:51.457092 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 08:16:51.457092 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 08:16:51.457092 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 08:16:51.457092 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 08:16:51.457092 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application with Lock( Jul 21 08:16:51.457092 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 08:16:51.457092 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self._enter() Jul 21 08:16:51.457092 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 08:16:51.457092 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 08:16:51.457844 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 08:16:51.457844 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self.creator() Jul 21 08:16:51.457844 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 08:16:51.457844 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application created_value = creator( Jul 21 08:16:51.457844 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 21 08:16:51.457844 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 21 08:16:51.457844 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 21 08:16:51.457844 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 21 08:16:51.457844 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 21 08:16:51.457844 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 21 08:16:51.457844 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 21 08:16:51.457844 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application Jul 21 08:16:51.463353 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73ba7fe8-748a-46b0-9c2b-647b9730fd90 None None] Authenticating user token {{(pid=80245) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:16:51.463719 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 199/397] 10.222.0.47 () {56 vars in 1099 bytes} [Thu Jul 21 08:16:51 2022] GET /identity/v3/roles/member => generated 84 bytes in 287 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 08:16:51.620563 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73ba7fe8-748a-46b0-9c2b-647b9730fd90 None None] Validating token access rules against request {{(pid=80245) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:16:51.642496 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG dbcounter [-] [80245] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80245) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:16:51.672113 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28ab827a-1a86-4fb1-b117-b72d64cb7ac0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80245) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:16:51.672798 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28ab827a-1a86-4fb1-b117-b72d64cb7ac0 None admin] REQUEST_METHOD: `GET` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:16:51.673082 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28ab827a-1a86-4fb1-b117-b72d64cb7ac0 None admin] SCRIPT_NAME: `/identity` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:16:51.673356 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28ab827a-1a86-4fb1-b117-b72d64cb7ac0 None admin] PATH_INFO: `/v3/roles` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:16:51.674109 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28ab827a-1a86-4fb1-b117-b72d64cb7ac0 None admin] RBAC: Adding query filter params (name=member) {{(pid=80245) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 08:16:51.674416 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28ab827a-1a86-4fb1-b117-b72d64cb7ac0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:16:51.675824 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28ab827a-1a86-4fb1-b117-b72d64cb7ac0 None admin] RBAC: Authorization granted {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:16:51.686980 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: [pid: 80245|app: 0|req: 199/398] 10.222.0.47 () {56 vars in 1101 bytes} [Thu Jul 21 08:16:51 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 224 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 08:16:51.694097 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b96f149-4af2-4616-8885-e8753abf6c79 None admin] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:16:51.762868 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28303b76-c382-47f7-b467-b37557baa584 None None] REQUEST_METHOD: `GET` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:16:51.763250 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28303b76-c382-47f7-b467-b37557baa584 None None] SCRIPT_NAME: `/identity` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:16:51.763672 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28303b76-c382-47f7-b467-b37557baa584 None None] PATH_INFO: `/` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:16:51.765227 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: [pid: 80245|app: 0|req: 200/399] 10.222.0.47 () {54 vars in 926 bytes} [Thu Jul 21 08:16:51 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 21 08:16:51.771395 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93bc99e8-6812-4f58-adf9-a8e0b5fd27a2 None None] REQUEST_METHOD: `POST` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:16:51.771819 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93bc99e8-6812-4f58-adf9-a8e0b5fd27a2 None None] SCRIPT_NAME: `/identity` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:16:51.772209 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93bc99e8-6812-4f58-adf9-a8e0b5fd27a2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:16:51.794935 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.auth.core [None req-93bc99e8-6812-4f58-adf9-a8e0b5fd27a2 None None] MFA Rules not processed for user `2dce40b8d74c41ec94cc36e6ff05d412`. Rule list: `[]` (Enabled: `True`). {{(pid=80245) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 08:16:51.796350 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.fernet_utils [None req-93bc99e8-6812-4f58-adf9-a8e0b5fd27a2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80245) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 08:16:51.816856 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b96f149-4af2-4616-8885-e8753abf6c79 None admin] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:16:51.853208 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1525bcb2-91cd-4082-bb72-4bdaecd25471 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:16:51.854378 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1525bcb2-91cd-4082-bb72-4bdaecd25471 None admin] REQUEST_METHOD: `GET` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:16:51.854942 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1525bcb2-91cd-4082-bb72-4bdaecd25471 None admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:16:51.855416 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1525bcb2-91cd-4082-bb72-4bdaecd25471 None admin] PATH_INFO: `/v3/groups/f75bd62425144927b6fd534595218714` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:16:51.856776 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1525bcb2-91cd-4082-bb72-4bdaecd25471 None admin] RBAC: Authorizing `identity:get_group(group_id=f75bd62425144927b6fd534595218714)` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:16:51.858410 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1525bcb2-91cd-4082-bb72-4bdaecd25471 None admin] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:16:51.860277 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 200/400] 10.222.0.47 () {56 vars in 1180 bytes} [Thu Jul 21 08:16:51 2022] GET /identity/v3/groups/f75bd62425144927b6fd534595218714 => generated 226 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 08:16:51.866521 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1525bcb2-91cd-4082-bb72-4bdaecd25471 None admin] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:16:51.884308 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: [pid: 80245|app: 0|req: 201/401] 10.222.0.47 () {58 vars in 1023 bytes} [Thu Jul 21 08:16:51 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 21 08:16:51.908082 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93bc99e8-6812-4f58-adf9-a8e0b5fd27a2 None None] Authenticating user token {{(pid=80245) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:16:51.928357 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG dbcounter [-] [80244] Writing DB stats keystone:SELECT=100 {{(pid=80244) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:16:52.003133 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1525bcb2-91cd-4082-bb72-4bdaecd25471 None admin] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:16:52.019755 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93bc99e8-6812-4f58-adf9-a8e0b5fd27a2 None None] Validating token access rules against request {{(pid=80245) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:16:52.028082 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG dbcounter [-] [80245] Writing DB stats keystone:SELECT=100 {{(pid=80245) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:16:52.059663 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d87e6d92-a6f6-4e90-8e4b-a9b04b781511 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80245) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:16:52.060603 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d87e6d92-a6f6-4e90-8e4b-a9b04b781511 None admin] REQUEST_METHOD: `GET` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:16:52.060883 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d87e6d92-a6f6-4e90-8e4b-a9b04b781511 None admin] SCRIPT_NAME: `/identity` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:16:52.061156 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d87e6d92-a6f6-4e90-8e4b-a9b04b781511 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:16:52.072402 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d87e6d92-a6f6-4e90-8e4b-a9b04b781511 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 21 08:16:52.080763 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application [None req-d87e6d92-a6f6-4e90-8e4b-a9b04b781511 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 21 08:16:52.080763 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 08:16:52.080763 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 08:16:52.080763 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 08:16:52.080763 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 08:16:52.080763 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 08:16:52.080763 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 08:16:52.080763 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 08:16:52.080763 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 08:16:52.080763 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 08:16:52.080763 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 08:16:52.080763 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 08:16:52.080763 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 21 08:16:52.080763 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 21 08:16:52.080763 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 21 08:16:52.080763 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 21 08:16:52.081639 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 21 08:16:52.081639 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 21 08:16:52.081639 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 21 08:16:52.081639 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 21 08:16:52.081639 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 08:16:52.081639 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 08:16:52.081639 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 08:16:52.081639 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 08:16:52.081639 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 08:16:52.081639 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 08:16:52.081639 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 08:16:52.081639 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application with Lock( Jul 21 08:16:52.081639 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 08:16:52.081639 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application return self._enter() Jul 21 08:16:52.081639 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 08:16:52.081639 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 08:16:52.082430 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 08:16:52.082430 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application return self.creator() Jul 21 08:16:52.082430 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 08:16:52.082430 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application created_value = creator( Jul 21 08:16:52.082430 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 21 08:16:52.082430 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 21 08:16:52.082430 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 21 08:16:52.082430 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 21 08:16:52.082430 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 21 08:16:52.082430 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 21 08:16:52.082430 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 21 08:16:52.082430 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application Jul 21 08:16:52.082430 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: [pid: 80245|app: 0|req: 202/402] 10.222.0.47 () {56 vars in 1114 bytes} [Thu Jul 21 08:16:51 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 08:16:52.087783 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d87e6d92-a6f6-4e90-8e4b-a9b04b781511 None admin] Authenticating user token {{(pid=80245) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:16:52.092028 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5bf9e4f-5fe0-4b44-aa54-bf8177608dd5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:16:52.092916 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5bf9e4f-5fe0-4b44-aa54-bf8177608dd5 None admin] REQUEST_METHOD: `GET` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:16:52.093201 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5bf9e4f-5fe0-4b44-aa54-bf8177608dd5 None admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:16:52.093476 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5bf9e4f-5fe0-4b44-aa54-bf8177608dd5 None admin] PATH_INFO: `/v3/projects/d4fa4cba54f348b68df92b37fb5fd5d1` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:16:52.094730 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5bf9e4f-5fe0-4b44-aa54-bf8177608dd5 None admin] RBAC: Authorizing `identity:get_project(project_id=d4fa4cba54f348b68df92b37fb5fd5d1)` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:16:52.096228 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5bf9e4f-5fe0-4b44-aa54-bf8177608dd5 None admin] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:16:52.099454 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 201/403] 10.222.0.47 () {56 vars in 1186 bytes} [Thu Jul 21 08:16:51 2022] GET /identity/v3/projects/d4fa4cba54f348b68df92b37fb5fd5d1 => generated 302 bytes in 232 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 21 08:16:52.104275 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5bf9e4f-5fe0-4b44-aa54-bf8177608dd5 None admin] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:16:52.199773 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d87e6d92-a6f6-4e90-8e4b-a9b04b781511 None admin] Validating token access rules against request {{(pid=80245) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:16:52.236630 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c002d95-f52f-4e27-89fd-f0b385e14c9f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80245) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:16:52.237305 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c002d95-f52f-4e27-89fd-f0b385e14c9f None admin] REQUEST_METHOD: `GET` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:16:52.237585 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c002d95-f52f-4e27-89fd-f0b385e14c9f None admin] SCRIPT_NAME: `/identity` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:16:52.237856 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c002d95-f52f-4e27-89fd-f0b385e14c9f None admin] PATH_INFO: `/v3/roles` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:16:52.238596 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c002d95-f52f-4e27-89fd-f0b385e14c9f None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80245) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 08:16:52.238897 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c002d95-f52f-4e27-89fd-f0b385e14c9f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:16:52.240543 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c002d95-f52f-4e27-89fd-f0b385e14c9f None admin] RBAC: Authorization granted {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:16:52.249548 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: [pid: 80245|app: 0|req: 203/404] 10.222.0.47 () {56 vars in 1111 bytes} [Thu Jul 21 08:16:52 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 21 08:16:52.255433 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-959589d1-84ef-416d-a39d-84a6654769e6 None None] REQUEST_METHOD: `GET` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:16:52.255725 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-959589d1-84ef-416d-a39d-84a6654769e6 None None] SCRIPT_NAME: `/identity` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:16:52.256001 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-959589d1-84ef-416d-a39d-84a6654769e6 None None] PATH_INFO: `/` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:16:52.261709 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: [pid: 80245|app: 0|req: 204/405] 10.222.0.47 () {54 vars in 926 bytes} [Thu Jul 21 08:16:52 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 08:16:52.261709 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-959589d1-84ef-416d-a39d-84a6654769e6 None None] Authenticating user token {{(pid=80245) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:16:52.277677 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5bf9e4f-5fe0-4b44-aa54-bf8177608dd5 None admin] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:16:52.295146 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG dbcounter [-] [80245] Writing DB stats keystone:SELECT=100 {{(pid=80245) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:16:52.297994 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG dbcounter [-] [80244] Writing DB stats keystone:SELECT=100 {{(pid=80244) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:16:52.317918 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da048274-51bd-405c-9bb5-c6e50abbc945 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:16:52.319202 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da048274-51bd-405c-9bb5-c6e50abbc945 None admin] REQUEST_METHOD: `PUT` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:16:52.319705 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da048274-51bd-405c-9bb5-c6e50abbc945 None admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:16:52.320166 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da048274-51bd-405c-9bb5-c6e50abbc945 None admin] PATH_INFO: `/v3/projects/d4fa4cba54f348b68df92b37fb5fd5d1/groups/f75bd62425144927b6fd534595218714/roles/3a074738221e4601b5205715ba87d240` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:16:52.322282 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da048274-51bd-405c-9bb5-c6e50abbc945 None admin] RBAC: Authorizing `identity:create_grant(project_id=d4fa4cba54f348b68df92b37fb5fd5d1, group_id=f75bd62425144927b6fd534595218714, role_id=3a074738221e4601b5205715ba87d240)` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:16:52.324044 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da048274-51bd-405c-9bb5-c6e50abbc945 None admin] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:16:52.347356 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 202/406] 10.222.0.47 () {58 vars in 1442 bytes} [Thu Jul 21 08:16:52 2022] PUT /identity/v3/projects/d4fa4cba54f348b68df92b37fb5fd5d1/groups/f75bd62425144927b6fd534595218714/roles/3a074738221e4601b5205715ba87d240 => generated 0 bytes in 238 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 08:16:52.350385 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59d6c87d-596f-4a63-acb0-58cdff1d304b None None] REQUEST_METHOD: `POST` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:16:52.350676 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59d6c87d-596f-4a63-acb0-58cdff1d304b None None] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:16:52.350951 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59d6c87d-596f-4a63-acb0-58cdff1d304b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:16:52.382449 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.auth.core [None req-59d6c87d-596f-4a63-acb0-58cdff1d304b None None] MFA Rules not processed for user `2dce40b8d74c41ec94cc36e6ff05d412`. Rule list: `[]` (Enabled: `True`). {{(pid=80244) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 08:16:52.383956 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.fernet_utils [None req-59d6c87d-596f-4a63-acb0-58cdff1d304b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80244) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 08:16:52.429547 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-959589d1-84ef-416d-a39d-84a6654769e6 None None] Validating token access rules against request {{(pid=80245) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:16:52.517349 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 203/407] 10.222.0.47 () {58 vars in 1023 bytes} [Thu Jul 21 08:16:52 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 21 08:16:52.518283 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a409191f-ab5d-402c-b83c-1783fbf496db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80245) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:16:52.519251 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a409191f-ab5d-402c-b83c-1783fbf496db None admin] REQUEST_METHOD: `GET` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:16:52.519592 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a409191f-ab5d-402c-b83c-1783fbf496db None admin] SCRIPT_NAME: `/identity` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:16:52.519864 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a409191f-ab5d-402c-b83c-1783fbf496db None admin] PATH_INFO: `/v3/groups/f75bd62425144927b6fd534595218714` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:16:52.524077 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a409191f-ab5d-402c-b83c-1783fbf496db None admin] RBAC: Authorizing `identity:get_group(group_id=f75bd62425144927b6fd534595218714)` {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:16:52.525781 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a409191f-ab5d-402c-b83c-1783fbf496db None admin] RBAC: Authorization granted {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:16:52.527939 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: [pid: 80245|app: 0|req: 205/408] 10.222.0.47 () {56 vars in 1180 bytes} [Thu Jul 21 08:16:52 2022] GET /identity/v3/groups/f75bd62425144927b6fd534595218714 => generated 226 bytes in 270 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 08:16:52.533755 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59d6c87d-596f-4a63-acb0-58cdff1d304b None None] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:16:52.544279 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a409191f-ab5d-402c-b83c-1783fbf496db None admin] Authenticating user token {{(pid=80245) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:16:52.643193 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59d6c87d-596f-4a63-acb0-58cdff1d304b None None] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:16:52.644967 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG dbcounter [-] [80245] Writing DB stats keystone:SELECT=100 {{(pid=80245) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:16:52.654126 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a409191f-ab5d-402c-b83c-1783fbf496db None admin] Validating token access rules against request {{(pid=80245) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:16:52.654554 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG dbcounter [-] [80244] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80244) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:16:52.684020 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a985546e-5cef-4b41-916e-24046481c4d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:16:52.684854 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a985546e-5cef-4b41-916e-24046481c4d8 None admin] REQUEST_METHOD: `GET` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:16:52.685139 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a985546e-5cef-4b41-916e-24046481c4d8 None admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:16:52.685416 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a985546e-5cef-4b41-916e-24046481c4d8 None admin] PATH_INFO: `/v3/projects/d4fa4cba54f348b68df92b37fb5fd5d1` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:16:52.686607 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a985546e-5cef-4b41-916e-24046481c4d8 None admin] RBAC: Authorizing `identity:get_project(project_id=d4fa4cba54f348b68df92b37fb5fd5d1)` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:16:52.688114 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a985546e-5cef-4b41-916e-24046481c4d8 None admin] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:16:52.694972 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 204/409] 10.222.0.47 () {56 vars in 1186 bytes} [Thu Jul 21 08:16:52 2022] GET /identity/v3/projects/d4fa4cba54f348b68df92b37fb5fd5d1 => generated 302 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 08:16:52.696631 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a985546e-5cef-4b41-916e-24046481c4d8 None admin] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:16:52.700086 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-207f321c-a092-4704-badb-11540943e7cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80245) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:16:52.700911 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-207f321c-a092-4704-badb-11540943e7cc None admin] REQUEST_METHOD: `GET` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:16:52.701190 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-207f321c-a092-4704-badb-11540943e7cc None admin] SCRIPT_NAME: `/identity` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:16:52.701464 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-207f321c-a092-4704-badb-11540943e7cc None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:16:52.715254 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-207f321c-a092-4704-badb-11540943e7cc None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 21 08:16:52.723908 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application [None req-207f321c-a092-4704-badb-11540943e7cc None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 21 08:16:52.723908 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 08:16:52.723908 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 08:16:52.723908 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 08:16:52.723908 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 08:16:52.723908 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 08:16:52.723908 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 08:16:52.723908 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 08:16:52.723908 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 08:16:52.723908 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 08:16:52.723908 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 08:16:52.723908 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 08:16:52.723908 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 21 08:16:52.723908 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 21 08:16:52.723908 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 21 08:16:52.723908 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 21 08:16:52.724779 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 21 08:16:52.724779 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 21 08:16:52.724779 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 21 08:16:52.724779 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 21 08:16:52.724779 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 08:16:52.724779 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 08:16:52.724779 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 08:16:52.724779 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 08:16:52.724779 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 08:16:52.724779 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 08:16:52.724779 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 08:16:52.724779 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application with Lock( Jul 21 08:16:52.724779 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 08:16:52.724779 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application return self._enter() Jul 21 08:16:52.724779 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 08:16:52.724779 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 08:16:52.725553 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 08:16:52.725553 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application return self.creator() Jul 21 08:16:52.725553 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 08:16:52.725553 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application created_value = creator( Jul 21 08:16:52.725553 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 21 08:16:52.725553 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 21 08:16:52.725553 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 21 08:16:52.725553 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 21 08:16:52.725553 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 21 08:16:52.725553 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 21 08:16:52.725553 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 21 08:16:52.725553 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application Jul 21 08:16:52.728214 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: [pid: 80245|app: 0|req: 206/410] 10.222.0.47 () {56 vars in 1096 bytes} [Thu Jul 21 08:16:52 2022] GET /identity/v3/roles/admin => generated 83 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 21 08:16:52.734183 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-207f321c-a092-4704-badb-11540943e7cc None admin] Authenticating user token {{(pid=80245) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:16:52.841323 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a985546e-5cef-4b41-916e-24046481c4d8 None admin] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:16:52.845353 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-207f321c-a092-4704-badb-11540943e7cc None admin] Validating token access rules against request {{(pid=80245) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:16:52.877414 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41ca3e95-e087-482b-b055-703527a52920 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:16:52.878302 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41ca3e95-e087-482b-b055-703527a52920 None admin] REQUEST_METHOD: `PUT` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:16:52.878615 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41ca3e95-e087-482b-b055-703527a52920 None admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:16:52.878906 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41ca3e95-e087-482b-b055-703527a52920 None admin] PATH_INFO: `/v3/projects/d4fa4cba54f348b68df92b37fb5fd5d1/groups/f75bd62425144927b6fd534595218714/roles/99a638c0aac44b79a1f2858574724495` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:16:52.882205 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41ca3e95-e087-482b-b055-703527a52920 None admin] RBAC: Authorizing `identity:create_grant(project_id=d4fa4cba54f348b68df92b37fb5fd5d1, group_id=f75bd62425144927b6fd534595218714, role_id=99a638c0aac44b79a1f2858574724495)` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:16:52.883985 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41ca3e95-e087-482b-b055-703527a52920 None admin] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:16:52.886454 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e37a8ae-9086-433c-847e-15715fd0b35b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80245) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:16:52.887514 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e37a8ae-9086-433c-847e-15715fd0b35b None admin] REQUEST_METHOD: `GET` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:16:52.888049 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e37a8ae-9086-433c-847e-15715fd0b35b None admin] SCRIPT_NAME: `/identity` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:16:52.888544 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e37a8ae-9086-433c-847e-15715fd0b35b None admin] PATH_INFO: `/v3/roles` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:16:52.889516 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e37a8ae-9086-433c-847e-15715fd0b35b None admin] RBAC: Adding query filter params (name=admin) {{(pid=80245) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 08:16:52.889943 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e37a8ae-9086-433c-847e-15715fd0b35b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:16:52.891528 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e37a8ae-9086-433c-847e-15715fd0b35b None admin] RBAC: Authorization granted {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:16:52.899467 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 205/411] 10.222.0.47 () {58 vars in 1442 bytes} [Thu Jul 21 08:16:52 2022] PUT /identity/v3/projects/d4fa4cba54f348b68df92b37fb5fd5d1/groups/f75bd62425144927b6fd534595218714/roles/99a638c0aac44b79a1f2858574724495 => generated 0 bytes in 203 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 08:16:52.901324 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79b16419-dda4-41a0-9943-0a40c64ea3ef None None] REQUEST_METHOD: `GET` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:16:52.901865 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79b16419-dda4-41a0-9943-0a40c64ea3ef None None] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:16:52.902370 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79b16419-dda4-41a0-9943-0a40c64ea3ef None None] PATH_INFO: `/` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:16:52.904174 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 206/412] 10.222.0.47 () {54 vars in 926 bytes} [Thu Jul 21 08:16:52 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 08:16:52.910555 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eee7951-70b9-49d8-b782-bfad827ddcc1 None None] REQUEST_METHOD: `POST` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:16:52.912413 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eee7951-70b9-49d8-b782-bfad827ddcc1 None None] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:16:52.912975 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eee7951-70b9-49d8-b782-bfad827ddcc1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:16:52.914623 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: [pid: 80245|app: 0|req: 207/413] 10.222.0.47 () {56 vars in 1099 bytes} [Thu Jul 21 08:16:52 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 08:16:52.921311 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e37a8ae-9086-433c-847e-15715fd0b35b None admin] Authenticating user token {{(pid=80245) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:16:52.964590 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.auth.core [None req-9eee7951-70b9-49d8-b782-bfad827ddcc1 None None] MFA Rules not processed for user `2dce40b8d74c41ec94cc36e6ff05d412`. Rule list: `[]` (Enabled: `True`). {{(pid=80244) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 08:16:52.968842 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.fernet_utils [None req-9eee7951-70b9-49d8-b782-bfad827ddcc1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80244) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 08:16:52.978853 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG dbcounter [-] [80245] Writing DB stats keystone:SELECT=100 {{(pid=80245) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:16:53.035834 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG dbcounter [-] [80244] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80244) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:16:53.106762 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 207/414] 10.222.0.47 () {58 vars in 1023 bytes} [Thu Jul 21 08:16:52 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 197 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 21 08:16:53.121408 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e37a8ae-9086-433c-847e-15715fd0b35b None admin] Validating token access rules against request {{(pid=80245) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:16:53.131719 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9eee7951-70b9-49d8-b782-bfad827ddcc1 None None] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:16:53.160651 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-803f9260-3e96-47ca-a2c8-728f069d18cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80245) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:16:53.162481 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-803f9260-3e96-47ca-a2c8-728f069d18cf None admin] REQUEST_METHOD: `GET` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:16:53.162990 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-803f9260-3e96-47ca-a2c8-728f069d18cf None admin] SCRIPT_NAME: `/identity` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:16:53.163490 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-803f9260-3e96-47ca-a2c8-728f069d18cf None admin] PATH_INFO: `/v3/groups/569a366732204da3a5ab12f694275b62` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:16:53.164919 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-803f9260-3e96-47ca-a2c8-728f069d18cf None admin] RBAC: Authorizing `identity:get_group(group_id=569a366732204da3a5ab12f694275b62)` {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:16:53.166785 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-803f9260-3e96-47ca-a2c8-728f069d18cf None admin] RBAC: Authorization granted {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:16:53.169005 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: [pid: 80245|app: 0|req: 208/415] 10.222.0.47 () {56 vars in 1180 bytes} [Thu Jul 21 08:16:52 2022] GET /identity/v3/groups/569a366732204da3a5ab12f694275b62 => generated 229 bytes in 249 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 21 08:16:53.174642 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-803f9260-3e96-47ca-a2c8-728f069d18cf None admin] Authenticating user token {{(pid=80245) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:16:53.244180 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9eee7951-70b9-49d8-b782-bfad827ddcc1 None None] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:16:53.266857 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG dbcounter [-] [80245] Writing DB stats keystone:SELECT=100 {{(pid=80245) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:16:53.287951 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b3dd04a-8d18-4203-bdee-7e7672e9d638 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:16:53.288762 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b3dd04a-8d18-4203-bdee-7e7672e9d638 None admin] REQUEST_METHOD: `GET` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:16:53.289047 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b3dd04a-8d18-4203-bdee-7e7672e9d638 None admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:16:53.289324 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b3dd04a-8d18-4203-bdee-7e7672e9d638 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:16:53.294537 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-803f9260-3e96-47ca-a2c8-728f069d18cf None admin] Validating token access rules against request {{(pid=80245) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:16:53.302534 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b3dd04a-8d18-4203-bdee-7e7672e9d638 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 21 08:16:53.311040 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application [None req-1b3dd04a-8d18-4203-bdee-7e7672e9d638 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 21 08:16:53.311040 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 08:16:53.311040 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 08:16:53.311040 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 08:16:53.311040 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 08:16:53.311040 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 08:16:53.311040 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 08:16:53.311040 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 08:16:53.311040 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 08:16:53.311040 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 08:16:53.311040 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 08:16:53.311040 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 08:16:53.311040 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 21 08:16:53.311040 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 21 08:16:53.311040 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 21 08:16:53.311040 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 21 08:16:53.311961 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 21 08:16:53.311961 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 21 08:16:53.311961 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 21 08:16:53.311961 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 21 08:16:53.311961 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 08:16:53.311961 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 08:16:53.311961 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 08:16:53.311961 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 08:16:53.311961 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 08:16:53.311961 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 08:16:53.311961 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 08:16:53.311961 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application with Lock( Jul 21 08:16:53.311961 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 08:16:53.311961 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self._enter() Jul 21 08:16:53.311961 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 08:16:53.311961 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 08:16:53.312724 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 08:16:53.312724 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self.creator() Jul 21 08:16:53.312724 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 08:16:53.312724 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application created_value = creator( Jul 21 08:16:53.312724 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 21 08:16:53.312724 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 21 08:16:53.312724 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 21 08:16:53.312724 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 21 08:16:53.312724 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 21 08:16:53.312724 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 21 08:16:53.312724 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 21 08:16:53.312724 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application Jul 21 08:16:53.313349 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 208/416] 10.222.0.47 () {56 vars in 1099 bytes} [Thu Jul 21 08:16:53 2022] GET /identity/v3/roles/member => generated 84 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 21 08:16:53.319433 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b3dd04a-8d18-4203-bdee-7e7672e9d638 None admin] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:16:53.350783 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG dbcounter [-] [80244] Writing DB stats keystone:SELECT=100 {{(pid=80244) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:16:53.392047 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2071e0f7-b0b5-4faa-b031-1a997d5840c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80245) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:16:53.392938 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2071e0f7-b0b5-4faa-b031-1a997d5840c7 None admin] REQUEST_METHOD: `GET` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:16:53.393217 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2071e0f7-b0b5-4faa-b031-1a997d5840c7 None admin] SCRIPT_NAME: `/identity` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:16:53.393489 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2071e0f7-b0b5-4faa-b031-1a997d5840c7 None admin] PATH_INFO: `/v3/projects/42d15fc0515945acacbbffcf2356c7a5` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:16:53.394731 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2071e0f7-b0b5-4faa-b031-1a997d5840c7 None admin] RBAC: Authorizing `identity:get_project(project_id=42d15fc0515945acacbbffcf2356c7a5)` {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:16:53.396786 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2071e0f7-b0b5-4faa-b031-1a997d5840c7 None admin] RBAC: Authorization granted {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:16:53.398809 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: [pid: 80245|app: 0|req: 209/417] 10.222.0.47 () {56 vars in 1186 bytes} [Thu Jul 21 08:16:53 2022] GET /identity/v3/projects/42d15fc0515945acacbbffcf2356c7a5 => generated 344 bytes in 224 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 08:16:53.404592 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2071e0f7-b0b5-4faa-b031-1a997d5840c7 None admin] Authenticating user token {{(pid=80245) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:16:53.436437 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b3dd04a-8d18-4203-bdee-7e7672e9d638 None admin] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:16:53.491618 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d470d24-8902-4d23-bd89-22b647948d7e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:16:53.493039 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d470d24-8902-4d23-bd89-22b647948d7e None admin] REQUEST_METHOD: `GET` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:16:53.493039 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d470d24-8902-4d23-bd89-22b647948d7e None admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:16:53.493039 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d470d24-8902-4d23-bd89-22b647948d7e None admin] PATH_INFO: `/v3/roles` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:16:53.493967 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d470d24-8902-4d23-bd89-22b647948d7e None admin] RBAC: Adding query filter params (name=member) {{(pid=80244) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 08:16:53.494364 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d470d24-8902-4d23-bd89-22b647948d7e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:16:53.495864 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d470d24-8902-4d23-bd89-22b647948d7e None admin] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:16:53.505931 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 209/418] 10.222.0.47 () {56 vars in 1101 bytes} [Thu Jul 21 08:16:53 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 08:16:53.511564 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a447c35d-7b54-4b35-ba1e-de53545ede3c None None] REQUEST_METHOD: `GET` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:16:53.511564 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a447c35d-7b54-4b35-ba1e-de53545ede3c None None] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:16:53.511564 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a447c35d-7b54-4b35-ba1e-de53545ede3c None None] PATH_INFO: `/` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:16:53.513825 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 210/419] 10.222.0.47 () {54 vars in 926 bytes} [Thu Jul 21 08:16:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 08:16:53.515555 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1401019b-062c-4f60-9039-ae8678f5be1d None None] REQUEST_METHOD: `POST` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:16:53.515848 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1401019b-062c-4f60-9039-ae8678f5be1d None None] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:16:53.516174 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1401019b-062c-4f60-9039-ae8678f5be1d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:16:53.535265 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.auth.core [None req-1401019b-062c-4f60-9039-ae8678f5be1d None None] MFA Rules not processed for user `2dce40b8d74c41ec94cc36e6ff05d412`. Rule list: `[]` (Enabled: `True`). {{(pid=80244) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 08:16:53.536597 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.fernet_utils [None req-1401019b-062c-4f60-9039-ae8678f5be1d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80244) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 08:16:53.581245 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2071e0f7-b0b5-4faa-b031-1a997d5840c7 None admin] Validating token access rules against request {{(pid=80245) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:16:53.616808 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e69e955-65a5-4f3b-bd28-e1751f8fd391 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80245) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:16:53.617769 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e69e955-65a5-4f3b-bd28-e1751f8fd391 None admin] REQUEST_METHOD: `PUT` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:16:53.618023 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e69e955-65a5-4f3b-bd28-e1751f8fd391 None admin] SCRIPT_NAME: `/identity` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:16:53.618350 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e69e955-65a5-4f3b-bd28-e1751f8fd391 None admin] PATH_INFO: `/v3/projects/42d15fc0515945acacbbffcf2356c7a5/groups/569a366732204da3a5ab12f694275b62/roles/e29870f6e9e2411a9db0d5bf45f32ca5` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:16:53.620485 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e69e955-65a5-4f3b-bd28-e1751f8fd391 None admin] RBAC: Authorizing `identity:create_grant(project_id=42d15fc0515945acacbbffcf2356c7a5, group_id=569a366732204da3a5ab12f694275b62, role_id=e29870f6e9e2411a9db0d5bf45f32ca5)` {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:16:53.621936 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e69e955-65a5-4f3b-bd28-e1751f8fd391 None admin] RBAC: Authorization granted {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:16:53.624449 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 211/420] 10.222.0.47 () {58 vars in 1023 bytes} [Thu Jul 21 08:16:53 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 21 08:16:53.625508 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1401019b-062c-4f60-9039-ae8678f5be1d None None] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:16:53.647432 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG dbcounter [-] [80244] Writing DB stats keystone:SELECT=100 {{(pid=80244) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:16:53.650261 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: [pid: 80245|app: 0|req: 210/421] 10.222.0.47 () {58 vars in 1442 bytes} [Thu Jul 21 08:16:53 2022] PUT /identity/v3/projects/42d15fc0515945acacbbffcf2356c7a5/groups/569a366732204da3a5ab12f694275b62/roles/e29870f6e9e2411a9db0d5bf45f32ca5 => generated 0 bytes in 246 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 21 08:16:53.661273 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e69e955-65a5-4f3b-bd28-e1751f8fd391 None admin] Authenticating user token {{(pid=80245) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:16:53.679107 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG dbcounter [-] [80245] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80245) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:16:53.752080 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1401019b-062c-4f60-9039-ae8678f5be1d None None] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:16:53.801597 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eff7ecbc-e35a-4a3c-9766-6ecf80d75b47 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:16:53.803224 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eff7ecbc-e35a-4a3c-9766-6ecf80d75b47 None admin] REQUEST_METHOD: `GET` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:16:53.803764 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eff7ecbc-e35a-4a3c-9766-6ecf80d75b47 None admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:16:53.804251 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eff7ecbc-e35a-4a3c-9766-6ecf80d75b47 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:16:53.845232 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e69e955-65a5-4f3b-bd28-e1751f8fd391 None admin] Validating token access rules against request {{(pid=80245) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:16:53.891759 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-288b85df-976b-44d8-ba30-ab92162c59e8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80245) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:16:53.892610 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-288b85df-976b-44d8-ba30-ab92162c59e8 None admin] REQUEST_METHOD: `GET` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:16:53.892892 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-288b85df-976b-44d8-ba30-ab92162c59e8 None admin] SCRIPT_NAME: `/identity` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:16:53.893164 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-288b85df-976b-44d8-ba30-ab92162c59e8 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:16:53.898116 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eff7ecbc-e35a-4a3c-9766-6ecf80d75b47 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:16:53.898597 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eff7ecbc-e35a-4a3c-9766-6ecf80d75b47 None admin] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:16:53.914151 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-288b85df-976b-44d8-ba30-ab92162c59e8 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 21 08:16:53.923963 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG dbcounter [-] [80244] Writing DB stats keystone:SELECT=100 {{(pid=80244) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:16:53.926275 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application [None req-288b85df-976b-44d8-ba30-ab92162c59e8 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 21 08:16:53.926275 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 08:16:53.926275 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 08:16:53.926275 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 08:16:53.926275 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 08:16:53.926275 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 08:16:53.926275 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 08:16:53.926275 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 08:16:53.926275 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 08:16:53.926275 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 08:16:53.926275 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 08:16:53.926275 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 08:16:53.926275 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 21 08:16:53.926275 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 21 08:16:53.926275 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 21 08:16:53.926275 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 21 08:16:53.927720 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 21 08:16:53.927720 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 21 08:16:53.927720 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 21 08:16:53.927720 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 21 08:16:53.927720 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 08:16:53.927720 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 08:16:53.927720 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 08:16:53.927720 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 08:16:53.927720 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 08:16:53.927720 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 08:16:53.927720 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 08:16:53.927720 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application with Lock( Jul 21 08:16:53.927720 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 08:16:53.927720 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application return self._enter() Jul 21 08:16:53.927720 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 08:16:53.927720 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 08:16:53.928531 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 08:16:53.928531 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application return self.creator() Jul 21 08:16:53.928531 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 08:16:53.928531 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application created_value = creator( Jul 21 08:16:53.928531 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 21 08:16:53.928531 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 21 08:16:53.928531 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 21 08:16:53.928531 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 21 08:16:53.928531 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 21 08:16:53.928531 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 21 08:16:53.928531 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 21 08:16:53.928531 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application Jul 21 08:16:53.929133 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: [pid: 80245|app: 0|req: 211/422] 10.222.0.47 () {56 vars in 1114 bytes} [Thu Jul 21 08:16:53 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 268 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 21 08:16:53.935240 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-288b85df-976b-44d8-ba30-ab92162c59e8 None admin] Authenticating user token {{(pid=80245) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:16:54.040976 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 212/423] 10.222.0.47 () {58 vars in 1282 bytes} [Thu Jul 21 08:16:53 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 416 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 21 08:16:54.047918 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eff7ecbc-e35a-4a3c-9766-6ecf80d75b47 None admin] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:16:54.080516 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG dbcounter [-] [80245] Writing DB stats keystone:SELECT=100 {{(pid=80245) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:16:54.149603 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-288b85df-976b-44d8-ba30-ab92162c59e8 None admin] Validating token access rules against request {{(pid=80245) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:16:54.170432 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eff7ecbc-e35a-4a3c-9766-6ecf80d75b47 None admin] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:16:54.192132 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG dbcounter [-] [80244] Writing DB stats keystone:SELECT=100 {{(pid=80244) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:16:54.224715 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd31275c-d9bc-4774-ab61-18219f04e6af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:16:54.225640 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd31275c-d9bc-4774-ab61-18219f04e6af None admin] REQUEST_METHOD: `GET` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:16:54.225928 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd31275c-d9bc-4774-ab61-18219f04e6af None admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:16:54.226701 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd31275c-d9bc-4774-ab61-18219f04e6af None admin] PATH_INFO: `/v3/projects/eab7e5ec54c94574ad68c89747ad5bcc` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:16:54.228115 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd31275c-d9bc-4774-ab61-18219f04e6af None admin] RBAC: Authorizing `identity:get_project(project_id=eab7e5ec54c94574ad68c89747ad5bcc)` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:16:54.229612 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd31275c-d9bc-4774-ab61-18219f04e6af None admin] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:16:54.231606 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 213/424] 10.222.0.47 () {56 vars in 1186 bytes} [Thu Jul 21 08:16:54 2022] GET /identity/v3/projects/eab7e5ec54c94574ad68c89747ad5bcc => generated 298 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 08:16:54.237632 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd31275c-d9bc-4774-ab61-18219f04e6af None admin] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:16:54.245879 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b8b140b-adf8-4a4b-903d-23d2140a1430 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80245) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:16:54.246600 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b8b140b-adf8-4a4b-903d-23d2140a1430 None admin] REQUEST_METHOD: `GET` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:16:54.246884 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b8b140b-adf8-4a4b-903d-23d2140a1430 None admin] SCRIPT_NAME: `/identity` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:16:54.247194 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b8b140b-adf8-4a4b-903d-23d2140a1430 None admin] PATH_INFO: `/v3/roles` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:16:54.248056 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b8b140b-adf8-4a4b-903d-23d2140a1430 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80245) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 08:16:54.248366 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b8b140b-adf8-4a4b-903d-23d2140a1430 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:16:54.255507 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b8b140b-adf8-4a4b-903d-23d2140a1430 None admin] RBAC: Authorization granted {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:16:54.265836 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: [pid: 80245|app: 0|req: 212/425] 10.222.0.47 () {56 vars in 1111 bytes} [Thu Jul 21 08:16:53 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 331 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 21 08:16:54.272662 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b8b140b-adf8-4a4b-903d-23d2140a1430 None admin] Authenticating user token {{(pid=80245) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:16:54.389952 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd31275c-d9bc-4774-ab61-18219f04e6af None admin] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:16:54.434700 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56182d83-b1ed-4f9b-a61a-40327d510769 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:16:54.436056 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56182d83-b1ed-4f9b-a61a-40327d510769 None admin] REQUEST_METHOD: `GET` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:16:54.436349 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56182d83-b1ed-4f9b-a61a-40327d510769 None admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:16:54.436630 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56182d83-b1ed-4f9b-a61a-40327d510769 None admin] PATH_INFO: `/v3/groups/f75bd62425144927b6fd534595218714` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:16:54.440055 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56182d83-b1ed-4f9b-a61a-40327d510769 None admin] RBAC: Authorizing `identity:get_group(group_id=f75bd62425144927b6fd534595218714)` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:16:54.441519 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56182d83-b1ed-4f9b-a61a-40327d510769 None admin] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:16:54.444276 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 214/426] 10.222.0.47 () {56 vars in 1180 bytes} [Thu Jul 21 08:16:54 2022] GET /identity/v3/groups/f75bd62425144927b6fd534595218714 => generated 226 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 08:16:54.455352 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56182d83-b1ed-4f9b-a61a-40327d510769 None admin] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:16:54.464192 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b8b140b-adf8-4a4b-903d-23d2140a1430 None admin] Validating token access rules against request {{(pid=80245) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:16:54.520688 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG dbcounter [-] [80244] Writing DB stats keystone:SELECT=100 {{(pid=80244) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:16:54.549406 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0538d3c-c35e-4ebd-a140-2d13705509d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80245) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:16:54.550082 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0538d3c-c35e-4ebd-a140-2d13705509d4 None admin] REQUEST_METHOD: `GET` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:16:54.550369 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0538d3c-c35e-4ebd-a140-2d13705509d4 None admin] SCRIPT_NAME: `/identity` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:16:54.550649 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0538d3c-c35e-4ebd-a140-2d13705509d4 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:16:54.556907 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG dbcounter [-] [80245] Writing DB stats keystone:SELECT=100 {{(pid=80245) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:16:54.599678 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56182d83-b1ed-4f9b-a61a-40327d510769 None admin] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:16:54.648072 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bc8f23d-b141-4bb5-893d-1928c877b2b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:16:54.648749 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bc8f23d-b141-4bb5-893d-1928c877b2b2 None admin] REQUEST_METHOD: `GET` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:16:54.649040 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bc8f23d-b141-4bb5-893d-1928c877b2b2 None admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:16:54.649323 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bc8f23d-b141-4bb5-893d-1928c877b2b2 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:16:54.650186 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bc8f23d-b141-4bb5-893d-1928c877b2b2 None admin] RBAC: Adding query filter params (group.id=f75bd62425144927b6fd534595218714, role.id=3a074738221e4601b5205715ba87d240, scope.project.id=eab7e5ec54c94574ad68c89747ad5bcc) {{(pid=80244) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 08:16:54.650502 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bc8f23d-b141-4bb5-893d-1928c877b2b2 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:16:54.651998 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bc8f23d-b141-4bb5-893d-1928c877b2b2 None admin] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:16:54.654800 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0538d3c-c35e-4ebd-a140-2d13705509d4 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:16:54.656277 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0538d3c-c35e-4ebd-a140-2d13705509d4 None admin] RBAC: Authorization granted {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:16:54.662841 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 215/427] 10.222.0.47 () {56 vars in 1376 bytes} [Thu Jul 21 08:16:54 2022] GET /identity/v3/role_assignments?group.id=f75bd62425144927b6fd534595218714&scope.project.id=eab7e5ec54c94574ad68c89747ad5bcc&role.id=3a074738221e4601b5205715ba87d240 => generated 611 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 08:16:54.765116 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: [pid: 80245|app: 0|req: 213/428] 10.222.0.47 () {58 vars in 1282 bytes} [Thu Jul 21 08:16:54 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 493 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 21 08:16:54.770771 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bc8f23d-b141-4bb5-893d-1928c877b2b2 None admin] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:16:54.926860 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bc8f23d-b141-4bb5-893d-1928c877b2b2 None admin] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:16:54.953002 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG dbcounter [-] [80244] Writing DB stats keystone:SELECT=100 {{(pid=80244) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:16:54.979399 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e93cb84a-d84b-44ef-bb8c-c9af26777d73 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:16:54.980288 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e93cb84a-d84b-44ef-bb8c-c9af26777d73 None admin] REQUEST_METHOD: `GET` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:16:54.980575 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e93cb84a-d84b-44ef-bb8c-c9af26777d73 None admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:16:54.980855 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e93cb84a-d84b-44ef-bb8c-c9af26777d73 None admin] PATH_INFO: `/v3/projects/eab7e5ec54c94574ad68c89747ad5bcc` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:16:54.982077 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e93cb84a-d84b-44ef-bb8c-c9af26777d73 None admin] RBAC: Authorizing `identity:get_project(project_id=eab7e5ec54c94574ad68c89747ad5bcc)` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:16:54.983561 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e93cb84a-d84b-44ef-bb8c-c9af26777d73 None admin] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:16:54.985585 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 216/429] 10.222.0.47 () {56 vars in 1186 bytes} [Thu Jul 21 08:16:54 2022] GET /identity/v3/projects/eab7e5ec54c94574ad68c89747ad5bcc => generated 298 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 08:16:54.991728 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0538d3c-c35e-4ebd-a140-2d13705509d4 None admin] Authenticating user token {{(pid=80245) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:16:55.033017 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ca56eef-f62c-458c-8fc0-5a4216744638 None None] REQUEST_METHOD: `GET` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:16:55.033610 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ca56eef-f62c-458c-8fc0-5a4216744638 None None] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:16:55.034086 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ca56eef-f62c-458c-8fc0-5a4216744638 None None] PATH_INFO: `/` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:16:55.037598 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 217/430] 10.222.0.47 () {54 vars in 926 bytes} [Thu Jul 21 08:16:55 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 21 08:16:55.043173 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20de8d93-7bdf-489e-961a-56bf9aaebd70 None None] REQUEST_METHOD: `POST` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:16:55.043835 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20de8d93-7bdf-489e-961a-56bf9aaebd70 None None] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:16:55.044316 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20de8d93-7bdf-489e-961a-56bf9aaebd70 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:16:55.047079 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG dbcounter [-] [80245] Writing DB stats keystone:SELECT=100 {{(pid=80245) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:16:55.062948 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.auth.core [None req-20de8d93-7bdf-489e-961a-56bf9aaebd70 None None] MFA Rules not processed for user `2dce40b8d74c41ec94cc36e6ff05d412`. Rule list: `[]` (Enabled: `True`). {{(pid=80244) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 08:16:55.064478 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.fernet_utils [None req-20de8d93-7bdf-489e-961a-56bf9aaebd70 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80244) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 08:16:55.141559 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0538d3c-c35e-4ebd-a140-2d13705509d4 None admin] Validating token access rules against request {{(pid=80245) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:16:55.154454 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 218/431] 10.222.0.47 () {58 vars in 1023 bytes} [Thu Jul 21 08:16:55 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 21 08:16:55.180529 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4adbb75-7014-425b-84b9-046e7f311c05 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80245) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:16:55.180902 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20de8d93-7bdf-489e-961a-56bf9aaebd70 None None] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:16:55.182315 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4adbb75-7014-425b-84b9-046e7f311c05 None admin] REQUEST_METHOD: `GET` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:16:55.182806 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4adbb75-7014-425b-84b9-046e7f311c05 None admin] SCRIPT_NAME: `/identity` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:16:55.183276 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4adbb75-7014-425b-84b9-046e7f311c05 None admin] PATH_INFO: `/v3/groups/f75bd62425144927b6fd534595218714` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:16:55.185605 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4adbb75-7014-425b-84b9-046e7f311c05 None admin] RBAC: Authorizing `identity:get_group(group_id=f75bd62425144927b6fd534595218714)` {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:16:55.187281 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4adbb75-7014-425b-84b9-046e7f311c05 None admin] RBAC: Authorization granted {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:16:55.189250 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: [pid: 80245|app: 0|req: 214/432] 10.222.0.47 () {56 vars in 1180 bytes} [Thu Jul 21 08:16:54 2022] GET /identity/v3/groups/f75bd62425144927b6fd534595218714 => generated 226 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 08:16:55.196681 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4adbb75-7014-425b-84b9-046e7f311c05 None admin] Authenticating user token {{(pid=80245) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:16:55.305986 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20de8d93-7bdf-489e-961a-56bf9aaebd70 None None] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:16:55.321653 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4adbb75-7014-425b-84b9-046e7f311c05 None admin] Validating token access rules against request {{(pid=80245) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:16:55.326350 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG dbcounter [-] [80245] Writing DB stats keystone:SELECT=100 {{(pid=80245) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:16:55.329831 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG dbcounter [-] [80244] Writing DB stats keystone:SELECT=100 {{(pid=80244) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:16:55.361415 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8794797-9994-4bf7-8f87-9fe2b88c35e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:16:55.362251 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8794797-9994-4bf7-8f87-9fe2b88c35e6 None admin] REQUEST_METHOD: `GET` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:16:55.362597 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8794797-9994-4bf7-8f87-9fe2b88c35e6 None admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:16:55.362884 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8794797-9994-4bf7-8f87-9fe2b88c35e6 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:16:55.373171 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-497c7559-b00e-4fdc-82fc-c3b9b96ddd08 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80245) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:16:55.373842 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-497c7559-b00e-4fdc-82fc-c3b9b96ddd08 None admin] REQUEST_METHOD: `GET` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:16:55.374128 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-497c7559-b00e-4fdc-82fc-c3b9b96ddd08 None admin] SCRIPT_NAME: `/identity` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:16:55.374405 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-497c7559-b00e-4fdc-82fc-c3b9b96ddd08 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:16:55.375241 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-497c7559-b00e-4fdc-82fc-c3b9b96ddd08 None admin] RBAC: Adding query filter params (group.id=f75bd62425144927b6fd534595218714, role.id=99a638c0aac44b79a1f2858574724495, scope.project.id=eab7e5ec54c94574ad68c89747ad5bcc) {{(pid=80245) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 08:16:55.375599 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-497c7559-b00e-4fdc-82fc-c3b9b96ddd08 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:16:55.376329 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8794797-9994-4bf7-8f87-9fe2b88c35e6 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 21 08:16:55.377031 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-497c7559-b00e-4fdc-82fc-c3b9b96ddd08 None admin] RBAC: Authorization granted {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:16:55.385057 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: [pid: 80245|app: 0|req: 215/433] 10.222.0.47 () {56 vars in 1376 bytes} [Thu Jul 21 08:16:55 2022] GET /identity/v3/role_assignments?group.id=f75bd62425144927b6fd534595218714&scope.project.id=eab7e5ec54c94574ad68c89747ad5bcc&role.id=99a638c0aac44b79a1f2858574724495 => generated 611 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 08:16:55.386602 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-911fe60f-13fb-4797-a74a-f0cd1a7894c4 None None] REQUEST_METHOD: `GET` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:16:55.387091 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-911fe60f-13fb-4797-a74a-f0cd1a7894c4 None None] SCRIPT_NAME: `/identity` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:16:55.387673 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-911fe60f-13fb-4797-a74a-f0cd1a7894c4 None None] PATH_INFO: `/` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:16:55.389230 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: [pid: 80245|app: 0|req: 216/434] 10.222.0.47 () {54 vars in 926 bytes} [Thu Jul 21 08:16:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 08:16:55.389654 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application [None req-d8794797-9994-4bf7-8f87-9fe2b88c35e6 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 21 08:16:55.389654 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 08:16:55.389654 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 08:16:55.389654 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 08:16:55.389654 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 08:16:55.389654 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 08:16:55.389654 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 08:16:55.389654 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 08:16:55.389654 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 08:16:55.389654 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 08:16:55.389654 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 08:16:55.389654 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 08:16:55.389654 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 21 08:16:55.389654 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 21 08:16:55.389654 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 21 08:16:55.389654 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 21 08:16:55.390520 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 21 08:16:55.390520 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 21 08:16:55.390520 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 21 08:16:55.390520 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 21 08:16:55.390520 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 08:16:55.390520 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 08:16:55.390520 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 08:16:55.390520 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 08:16:55.390520 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 08:16:55.390520 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 08:16:55.390520 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 08:16:55.390520 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application with Lock( Jul 21 08:16:55.390520 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 08:16:55.390520 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self._enter() Jul 21 08:16:55.390520 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 08:16:55.390520 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 08:16:55.391298 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 08:16:55.391298 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self.creator() Jul 21 08:16:55.391298 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 08:16:55.391298 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application created_value = creator( Jul 21 08:16:55.391298 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 21 08:16:55.391298 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 21 08:16:55.391298 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 21 08:16:55.391298 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 21 08:16:55.391298 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 21 08:16:55.391298 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 21 08:16:55.391298 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 21 08:16:55.391298 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application Jul 21 08:16:55.391298 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 219/435] 10.222.0.47 () {56 vars in 1099 bytes} [Thu Jul 21 08:16:55 2022] GET /identity/v3/roles/member => generated 84 bytes in 211 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 08:16:55.395199 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f6c8cdd-bddf-4ef5-8665-9058039b5fd2 None None] REQUEST_METHOD: `POST` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:16:55.395870 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f6c8cdd-bddf-4ef5-8665-9058039b5fd2 None None] SCRIPT_NAME: `/identity` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:16:55.396355 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f6c8cdd-bddf-4ef5-8665-9058039b5fd2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:16:55.399405 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8794797-9994-4bf7-8f87-9fe2b88c35e6 None admin] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:16:55.415386 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.auth.core [None req-1f6c8cdd-bddf-4ef5-8665-9058039b5fd2 None None] MFA Rules not processed for user `2dce40b8d74c41ec94cc36e6ff05d412`. Rule list: `[]` (Enabled: `True`). {{(pid=80245) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 08:16:55.416966 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.fernet_utils [None req-1f6c8cdd-bddf-4ef5-8665-9058039b5fd2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80245) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 08:16:55.496664 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: [pid: 80245|app: 0|req: 217/436] 10.222.0.47 () {58 vars in 1023 bytes} [Thu Jul 21 08:16:55 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 21 08:16:55.527121 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f6c8cdd-bddf-4ef5-8665-9058039b5fd2 None None] Authenticating user token {{(pid=80245) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:16:55.550447 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8794797-9994-4bf7-8f87-9fe2b88c35e6 None admin] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:16:55.589627 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fbb33d7-008d-4646-b53f-42dd129d6f25 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:16:55.590239 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fbb33d7-008d-4646-b53f-42dd129d6f25 None admin] REQUEST_METHOD: `GET` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:16:55.590526 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fbb33d7-008d-4646-b53f-42dd129d6f25 None admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:16:55.590806 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fbb33d7-008d-4646-b53f-42dd129d6f25 None admin] PATH_INFO: `/v3/roles` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:16:55.592473 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fbb33d7-008d-4646-b53f-42dd129d6f25 None admin] RBAC: Adding query filter params (name=member) {{(pid=80244) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 08:16:55.592473 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fbb33d7-008d-4646-b53f-42dd129d6f25 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:16:55.593318 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fbb33d7-008d-4646-b53f-42dd129d6f25 None admin] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:16:55.604738 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 220/437] 10.222.0.47 () {56 vars in 1101 bytes} [Thu Jul 21 08:16:55 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 08:16:55.610028 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fbb33d7-008d-4646-b53f-42dd129d6f25 None admin] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:16:55.652293 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG dbcounter [-] [80245] Writing DB stats keystone:SELECT=100 {{(pid=80245) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:16:55.658155 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f6c8cdd-bddf-4ef5-8665-9058039b5fd2 None None] Validating token access rules against request {{(pid=80245) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:16:55.667852 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG dbcounter [-] [80244] Writing DB stats keystone:SELECT=100 {{(pid=80244) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:16:55.707100 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4af5c4ab-b1e2-4403-b1e8-7377eca409c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80245) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:16:55.708356 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4af5c4ab-b1e2-4403-b1e8-7377eca409c3 None admin] REQUEST_METHOD: `GET` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:16:55.708864 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4af5c4ab-b1e2-4403-b1e8-7377eca409c3 None admin] SCRIPT_NAME: `/identity` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:16:55.709285 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4af5c4ab-b1e2-4403-b1e8-7377eca409c3 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:16:55.720407 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4af5c4ab-b1e2-4403-b1e8-7377eca409c3 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 21 08:16:55.729191 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application [None req-4af5c4ab-b1e2-4403-b1e8-7377eca409c3 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 21 08:16:55.729191 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 08:16:55.729191 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 08:16:55.729191 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 08:16:55.729191 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 08:16:55.729191 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 08:16:55.729191 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 08:16:55.729191 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 08:16:55.729191 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 08:16:55.729191 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 08:16:55.729191 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 08:16:55.729191 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 08:16:55.729191 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 21 08:16:55.729191 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 21 08:16:55.729191 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 21 08:16:55.729191 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 21 08:16:55.730169 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 21 08:16:55.730169 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 21 08:16:55.730169 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 21 08:16:55.730169 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 21 08:16:55.730169 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 08:16:55.730169 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 08:16:55.730169 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 08:16:55.730169 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 08:16:55.730169 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 08:16:55.730169 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 08:16:55.730169 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 08:16:55.730169 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application with Lock( Jul 21 08:16:55.730169 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 08:16:55.730169 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application return self._enter() Jul 21 08:16:55.730169 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 08:16:55.730169 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 08:16:55.730950 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 08:16:55.730950 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application return self.creator() Jul 21 08:16:55.730950 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 08:16:55.730950 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application created_value = creator( Jul 21 08:16:55.730950 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 21 08:16:55.730950 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 21 08:16:55.730950 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 21 08:16:55.730950 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 21 08:16:55.730950 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 21 08:16:55.730950 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 21 08:16:55.730950 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 21 08:16:55.730950 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application Jul 21 08:16:55.730950 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: [pid: 80245|app: 0|req: 218/438] 10.222.0.47 () {56 vars in 1114 bytes} [Thu Jul 21 08:16:55 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 08:16:55.736061 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4af5c4ab-b1e2-4403-b1e8-7377eca409c3 None admin] Authenticating user token {{(pid=80245) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:16:55.750994 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fbb33d7-008d-4646-b53f-42dd129d6f25 None admin] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:16:55.798661 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-499d948f-83ed-4cdf-be7d-99370bafd9ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:16:55.799621 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-499d948f-83ed-4cdf-be7d-99370bafd9ad None admin] REQUEST_METHOD: `GET` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:16:55.800055 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-499d948f-83ed-4cdf-be7d-99370bafd9ad None admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:16:55.800477 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-499d948f-83ed-4cdf-be7d-99370bafd9ad None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:16:55.863132 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4af5c4ab-b1e2-4403-b1e8-7377eca409c3 None admin] Validating token access rules against request {{(pid=80245) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:16:55.900835 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-499d948f-83ed-4cdf-be7d-99370bafd9ad None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:16:55.902302 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-499d948f-83ed-4cdf-be7d-99370bafd9ad None admin] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:16:55.909936 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3387167-2959-483f-a55e-91befcc0a7a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80245) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:16:55.910621 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3387167-2959-483f-a55e-91befcc0a7a8 None admin] REQUEST_METHOD: `GET` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:16:55.910897 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3387167-2959-483f-a55e-91befcc0a7a8 None admin] SCRIPT_NAME: `/identity` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:16:55.911218 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3387167-2959-483f-a55e-91befcc0a7a8 None admin] PATH_INFO: `/v3/roles` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:16:55.912499 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3387167-2959-483f-a55e-91befcc0a7a8 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80245) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 08:16:55.913305 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3387167-2959-483f-a55e-91befcc0a7a8 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:16:55.914699 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3387167-2959-483f-a55e-91befcc0a7a8 None admin] RBAC: Authorization granted {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:16:55.924483 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: [pid: 80245|app: 0|req: 219/439] 10.222.0.47 () {56 vars in 1111 bytes} [Thu Jul 21 08:16:55 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 21 08:16:55.929989 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3387167-2959-483f-a55e-91befcc0a7a8 None admin] Authenticating user token {{(pid=80245) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:16:55.953845 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG dbcounter [-] [80245] Writing DB stats keystone:SELECT=100 {{(pid=80245) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:16:55.962420 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG dbcounter [-] [80244] Writing DB stats keystone:SELECT=100 {{(pid=80244) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:16:56.045869 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 221/440] 10.222.0.47 () {58 vars in 1282 bytes} [Thu Jul 21 08:16:55 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 434 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 21 08:16:56.049021 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-499d948f-83ed-4cdf-be7d-99370bafd9ad None admin] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:16:56.073472 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3387167-2959-483f-a55e-91befcc0a7a8 None admin] Validating token access rules against request {{(pid=80245) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:16:56.110011 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5edf8368-4f2e-4b82-ab65-a85de4dde618 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80245) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:16:56.110670 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5edf8368-4f2e-4b82-ab65-a85de4dde618 None admin] REQUEST_METHOD: `GET` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:16:56.110954 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5edf8368-4f2e-4b82-ab65-a85de4dde618 None admin] SCRIPT_NAME: `/identity` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:16:56.111242 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5edf8368-4f2e-4b82-ab65-a85de4dde618 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:16:56.152945 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-499d948f-83ed-4cdf-be7d-99370bafd9ad None admin] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:16:56.181022 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5edf8368-4f2e-4b82-ab65-a85de4dde618 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:16:56.182482 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5edf8368-4f2e-4b82-ab65-a85de4dde618 None admin] RBAC: Authorization granted {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:16:56.186568 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dd5891a-2d47-471d-82b2-260bbb063236 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:16:56.187462 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dd5891a-2d47-471d-82b2-260bbb063236 None admin] REQUEST_METHOD: `GET` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:16:56.187747 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dd5891a-2d47-471d-82b2-260bbb063236 None admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:16:56.188018 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dd5891a-2d47-471d-82b2-260bbb063236 None admin] PATH_INFO: `/v3/projects/d4fa4cba54f348b68df92b37fb5fd5d1` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:16:56.189207 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dd5891a-2d47-471d-82b2-260bbb063236 None admin] RBAC: Authorizing `identity:get_project(project_id=d4fa4cba54f348b68df92b37fb5fd5d1)` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:16:56.190635 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dd5891a-2d47-471d-82b2-260bbb063236 None admin] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:16:56.192355 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 222/441] 10.222.0.47 () {56 vars in 1186 bytes} [Thu Jul 21 08:16:56 2022] GET /identity/v3/projects/d4fa4cba54f348b68df92b37fb5fd5d1 => generated 302 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 21 08:16:56.197208 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a67da35d-8f83-4963-b1a3-bc47115b8db6 None None] REQUEST_METHOD: `GET` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:16:56.197208 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a67da35d-8f83-4963-b1a3-bc47115b8db6 None None] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:16:56.197208 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a67da35d-8f83-4963-b1a3-bc47115b8db6 None None] PATH_INFO: `/` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:16:56.197208 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 223/442] 10.222.0.47 () {54 vars in 926 bytes} [Thu Jul 21 08:16:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 08:16:56.197768 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a67da35d-8f83-4963-b1a3-bc47115b8db6 None None] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:16:56.204899 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG dbcounter [-] [80245] Writing DB stats keystone:SELECT=100 {{(pid=80245) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:16:56.206702 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG dbcounter [-] [80244] Writing DB stats keystone:SELECT=100 {{(pid=80244) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:16:56.285965 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: [pid: 80245|app: 0|req: 220/443] 10.222.0.47 () {58 vars in 1282 bytes} [Thu Jul 21 08:16:55 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 356 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 21 08:16:56.287261 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56b9bbc5-0c3c-4b47-a7f6-94942f620cba None None] REQUEST_METHOD: `POST` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:16:56.287836 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56b9bbc5-0c3c-4b47-a7f6-94942f620cba None None] SCRIPT_NAME: `/identity` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:16:56.287836 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56b9bbc5-0c3c-4b47-a7f6-94942f620cba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:16:56.301153 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a67da35d-8f83-4963-b1a3-bc47115b8db6 None None] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:16:56.304925 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.auth.core [None req-56b9bbc5-0c3c-4b47-a7f6-94942f620cba None None] MFA Rules not processed for user `2dce40b8d74c41ec94cc36e6ff05d412`. Rule list: `[]` (Enabled: `True`). {{(pid=80245) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 08:16:56.306158 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.fernet_utils [None req-56b9bbc5-0c3c-4b47-a7f6-94942f620cba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80245) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 08:16:56.336033 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2b046b3-708e-46a1-b348-5fec3d762126 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:16:56.336900 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2b046b3-708e-46a1-b348-5fec3d762126 None admin] REQUEST_METHOD: `GET` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:16:56.337189 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2b046b3-708e-46a1-b348-5fec3d762126 None admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:16:56.337467 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2b046b3-708e-46a1-b348-5fec3d762126 None admin] PATH_INFO: `/v3/groups/f75bd62425144927b6fd534595218714` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:16:56.338647 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2b046b3-708e-46a1-b348-5fec3d762126 None admin] RBAC: Authorizing `identity:get_group(group_id=f75bd62425144927b6fd534595218714)` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:16:56.340138 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2b046b3-708e-46a1-b348-5fec3d762126 None admin] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:16:56.341768 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 224/444] 10.222.0.47 () {56 vars in 1180 bytes} [Thu Jul 21 08:16:56 2022] GET /identity/v3/groups/f75bd62425144927b6fd534595218714 => generated 226 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 21 08:16:56.342535 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2b046b3-708e-46a1-b348-5fec3d762126 None admin] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:16:56.375540 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: [pid: 80245|app: 0|req: 221/445] 10.222.0.47 () {58 vars in 1023 bytes} [Thu Jul 21 08:16:56 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 21 08:16:56.376370 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56b9bbc5-0c3c-4b47-a7f6-94942f620cba None None] Authenticating user token {{(pid=80245) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:16:56.420562 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG dbcounter [-] [80244] Writing DB stats keystone:SELECT=100 {{(pid=80244) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:16:56.428493 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG dbcounter [-] [80245] Writing DB stats keystone:SELECT=100 {{(pid=80245) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:16:56.447047 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2b046b3-708e-46a1-b348-5fec3d762126 None admin] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:16:56.480678 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0e74e3e-6d3f-43a9-b706-1d6e4df58e06 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:16:56.481526 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0e74e3e-6d3f-43a9-b706-1d6e4df58e06 None admin] REQUEST_METHOD: `GET` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:16:56.481805 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0e74e3e-6d3f-43a9-b706-1d6e4df58e06 None admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:16:56.482076 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0e74e3e-6d3f-43a9-b706-1d6e4df58e06 None admin] PATH_INFO: `/v3/projects/d4fa4cba54f348b68df92b37fb5fd5d1` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:16:56.482988 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56b9bbc5-0c3c-4b47-a7f6-94942f620cba None None] Validating token access rules against request {{(pid=80245) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:16:56.483258 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0e74e3e-6d3f-43a9-b706-1d6e4df58e06 None admin] RBAC: Authorizing `identity:get_project(project_id=d4fa4cba54f348b68df92b37fb5fd5d1)` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:16:56.484749 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0e74e3e-6d3f-43a9-b706-1d6e4df58e06 None admin] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:16:56.486443 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 225/446] 10.222.0.47 () {56 vars in 1186 bytes} [Thu Jul 21 08:16:56 2022] GET /identity/v3/projects/d4fa4cba54f348b68df92b37fb5fd5d1 => generated 302 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 08:16:56.487203 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0e74e3e-6d3f-43a9-b706-1d6e4df58e06 None admin] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:16:56.518870 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb3fbcea-5414-4e0c-8955-9d5077f636dc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80245) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:16:56.519472 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb3fbcea-5414-4e0c-8955-9d5077f636dc None admin] REQUEST_METHOD: `GET` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:16:56.519760 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb3fbcea-5414-4e0c-8955-9d5077f636dc None admin] SCRIPT_NAME: `/identity` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:16:56.520034 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb3fbcea-5414-4e0c-8955-9d5077f636dc None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:16:56.521498 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb3fbcea-5414-4e0c-8955-9d5077f636dc None admin] RBAC: Adding query filter params (group.id=f75bd62425144927b6fd534595218714, role.id=3a074738221e4601b5205715ba87d240, scope.project.id=d4fa4cba54f348b68df92b37fb5fd5d1) {{(pid=80245) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 08:16:56.521809 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb3fbcea-5414-4e0c-8955-9d5077f636dc None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:16:56.523246 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb3fbcea-5414-4e0c-8955-9d5077f636dc None admin] RBAC: Authorization granted {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:16:56.529097 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: [pid: 80245|app: 0|req: 222/447] 10.222.0.47 () {56 vars in 1376 bytes} [Thu Jul 21 08:16:56 2022] GET /identity/v3/role_assignments?group.id=f75bd62425144927b6fd534595218714&scope.project.id=d4fa4cba54f348b68df92b37fb5fd5d1&role.id=3a074738221e4601b5205715ba87d240 => generated 611 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 08:16:56.529876 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb3fbcea-5414-4e0c-8955-9d5077f636dc None admin] Authenticating user token {{(pid=80245) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:16:56.603248 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0e74e3e-6d3f-43a9-b706-1d6e4df58e06 None admin] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:16:56.642561 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42afb66a-03e9-450d-a453-f0eac05fc2a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:16:56.643440 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42afb66a-03e9-450d-a453-f0eac05fc2a8 None admin] REQUEST_METHOD: `GET` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:16:56.643728 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42afb66a-03e9-450d-a453-f0eac05fc2a8 None admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:16:56.644006 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42afb66a-03e9-450d-a453-f0eac05fc2a8 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:16:56.649114 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb3fbcea-5414-4e0c-8955-9d5077f636dc None admin] Validating token access rules against request {{(pid=80245) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:16:56.655834 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42afb66a-03e9-450d-a453-f0eac05fc2a8 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 21 08:16:56.663780 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application [None req-42afb66a-03e9-450d-a453-f0eac05fc2a8 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 21 08:16:56.663780 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 08:16:56.663780 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 08:16:56.663780 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 08:16:56.663780 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 08:16:56.663780 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 08:16:56.663780 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 08:16:56.663780 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 08:16:56.663780 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 08:16:56.663780 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 08:16:56.663780 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 08:16:56.663780 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 08:16:56.663780 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 21 08:16:56.663780 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 21 08:16:56.663780 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 21 08:16:56.663780 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 21 08:16:56.664719 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 21 08:16:56.664719 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 21 08:16:56.664719 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 21 08:16:56.664719 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 21 08:16:56.664719 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 08:16:56.664719 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 08:16:56.664719 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 08:16:56.664719 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 08:16:56.664719 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 08:16:56.664719 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 08:16:56.664719 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 08:16:56.664719 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application with Lock( Jul 21 08:16:56.664719 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 08:16:56.664719 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self._enter() Jul 21 08:16:56.664719 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 08:16:56.664719 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 08:16:56.665511 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 08:16:56.665511 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self.creator() Jul 21 08:16:56.665511 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 08:16:56.665511 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application created_value = creator( Jul 21 08:16:56.665511 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 21 08:16:56.665511 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 21 08:16:56.665511 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 21 08:16:56.665511 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 21 08:16:56.665511 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 21 08:16:56.665511 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 21 08:16:56.665511 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 21 08:16:56.665511 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application Jul 21 08:16:56.665511 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 226/448] 10.222.0.47 () {56 vars in 1096 bytes} [Thu Jul 21 08:16:56 2022] GET /identity/v3/roles/admin => generated 83 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 08:16:56.670289 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG dbcounter [-] [80245] Writing DB stats keystone:SELECT=100 {{(pid=80245) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:16:56.671371 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42afb66a-03e9-450d-a453-f0eac05fc2a8 None admin] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:16:56.675089 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG dbcounter [-] [80244] Writing DB stats keystone:SELECT=100 {{(pid=80244) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:16:56.699354 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c11899b-1aeb-49e5-9da5-1d3794739a49 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80245) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:16:56.700547 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c11899b-1aeb-49e5-9da5-1d3794739a49 None admin] REQUEST_METHOD: `GET` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:16:56.700547 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c11899b-1aeb-49e5-9da5-1d3794739a49 None admin] SCRIPT_NAME: `/identity` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:16:56.700812 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c11899b-1aeb-49e5-9da5-1d3794739a49 None admin] PATH_INFO: `/v3/groups/f75bd62425144927b6fd534595218714` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:16:56.701955 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c11899b-1aeb-49e5-9da5-1d3794739a49 None admin] RBAC: Authorizing `identity:get_group(group_id=f75bd62425144927b6fd534595218714)` {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:16:56.704231 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c11899b-1aeb-49e5-9da5-1d3794739a49 None admin] RBAC: Authorization granted {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:16:56.705349 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: [pid: 80245|app: 0|req: 223/449] 10.222.0.47 () {56 vars in 1180 bytes} [Thu Jul 21 08:16:56 2022] GET /identity/v3/groups/f75bd62425144927b6fd534595218714 => generated 226 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 08:16:56.711097 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c11899b-1aeb-49e5-9da5-1d3794739a49 None admin] Authenticating user token {{(pid=80245) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:16:56.793368 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42afb66a-03e9-450d-a453-f0eac05fc2a8 None admin] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:16:56.831920 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c11899b-1aeb-49e5-9da5-1d3794739a49 None admin] Validating token access rules against request {{(pid=80245) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:16:56.835150 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2608186-1b57-445f-9c4c-ad6ec7b0a938 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:16:56.835944 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2608186-1b57-445f-9c4c-ad6ec7b0a938 None admin] REQUEST_METHOD: `GET` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:16:56.836227 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2608186-1b57-445f-9c4c-ad6ec7b0a938 None admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:16:56.836499 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2608186-1b57-445f-9c4c-ad6ec7b0a938 None admin] PATH_INFO: `/v3/roles` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:16:56.837250 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2608186-1b57-445f-9c4c-ad6ec7b0a938 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80244) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 08:16:56.837553 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2608186-1b57-445f-9c4c-ad6ec7b0a938 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:16:56.838931 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2608186-1b57-445f-9c4c-ad6ec7b0a938 None admin] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:16:56.851218 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 227/450] 10.222.0.47 () {56 vars in 1099 bytes} [Thu Jul 21 08:16:56 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 21 08:16:56.856794 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2608186-1b57-445f-9c4c-ad6ec7b0a938 None admin] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:16:56.875122 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b4e5f83-a412-40cc-a93f-40fd88839403 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80245) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:16:56.875877 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b4e5f83-a412-40cc-a93f-40fd88839403 None admin] REQUEST_METHOD: `GET` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:16:56.876126 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b4e5f83-a412-40cc-a93f-40fd88839403 None admin] SCRIPT_NAME: `/identity` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:16:56.876410 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b4e5f83-a412-40cc-a93f-40fd88839403 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:16:56.877243 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b4e5f83-a412-40cc-a93f-40fd88839403 None admin] RBAC: Adding query filter params (group.id=f75bd62425144927b6fd534595218714, role.id=99a638c0aac44b79a1f2858574724495, scope.project.id=d4fa4cba54f348b68df92b37fb5fd5d1) {{(pid=80245) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 08:16:56.877583 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b4e5f83-a412-40cc-a93f-40fd88839403 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:16:56.879007 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b4e5f83-a412-40cc-a93f-40fd88839403 None admin] RBAC: Authorization granted {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:16:56.888447 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: [pid: 80245|app: 0|req: 224/451] 10.222.0.47 () {56 vars in 1376 bytes} [Thu Jul 21 08:16:56 2022] GET /identity/v3/role_assignments?group.id=f75bd62425144927b6fd534595218714&scope.project.id=d4fa4cba54f348b68df92b37fb5fd5d1&role.id=99a638c0aac44b79a1f2858574724495 => generated 611 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 21 08:16:56.930538 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG dbcounter [-] [80244] Writing DB stats keystone:SELECT=100 {{(pid=80244) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:16:56.972735 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2608186-1b57-445f-9c4c-ad6ec7b0a938 None admin] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:16:57.009661 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5991dd42-b630-49bf-9391-da6cef5a79c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:16:57.010327 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5991dd42-b630-49bf-9391-da6cef5a79c3 None admin] REQUEST_METHOD: `GET` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:16:57.010614 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5991dd42-b630-49bf-9391-da6cef5a79c3 None admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:16:57.010890 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5991dd42-b630-49bf-9391-da6cef5a79c3 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:16:57.080240 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5991dd42-b630-49bf-9391-da6cef5a79c3 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:16:57.081689 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5991dd42-b630-49bf-9391-da6cef5a79c3 None admin] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:16:57.144792 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG dbcounter [-] [80244] Writing DB stats keystone:SELECT=100 {{(pid=80244) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:16:57.186943 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 228/452] 10.222.0.47 () {58 vars in 1282 bytes} [Thu Jul 21 08:16:56 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 330 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Jul 21 08:16:57.192746 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b4e5f83-a412-40cc-a93f-40fd88839403 None admin] Authenticating user token {{(pid=80245) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:16:57.241782 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG dbcounter [-] [80245] Writing DB stats keystone:SELECT=100 {{(pid=80245) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:16:57.293735 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b4e5f83-a412-40cc-a93f-40fd88839403 None admin] Validating token access rules against request {{(pid=80245) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:16:57.327232 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-291e5d4e-d08e-4db6-b148-f98ee0bcd3cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80245) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:16:57.328124 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-291e5d4e-d08e-4db6-b148-f98ee0bcd3cf None admin] REQUEST_METHOD: `GET` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:16:57.328403 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-291e5d4e-d08e-4db6-b148-f98ee0bcd3cf None admin] SCRIPT_NAME: `/identity` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:16:57.328679 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-291e5d4e-d08e-4db6-b148-f98ee0bcd3cf None admin] PATH_INFO: `/v3/projects/42d15fc0515945acacbbffcf2356c7a5` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:16:57.329861 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-291e5d4e-d08e-4db6-b148-f98ee0bcd3cf None admin] RBAC: Authorizing `identity:get_project(project_id=42d15fc0515945acacbbffcf2356c7a5)` {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:16:57.331302 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-291e5d4e-d08e-4db6-b148-f98ee0bcd3cf None admin] RBAC: Authorization granted {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:16:57.333028 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: [pid: 80245|app: 0|req: 225/453] 10.222.0.47 () {56 vars in 1186 bytes} [Thu Jul 21 08:16:57 2022] GET /identity/v3/projects/42d15fc0515945acacbbffcf2356c7a5 => generated 344 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 08:16:57.338588 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5991dd42-b630-49bf-9391-da6cef5a79c3 None admin] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:16:57.436522 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5991dd42-b630-49bf-9391-da6cef5a79c3 None admin] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:16:57.470327 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cf2a48b-d7c3-4e2d-81de-b06d6a27ff96 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:16:57.471103 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cf2a48b-d7c3-4e2d-81de-b06d6a27ff96 None admin] REQUEST_METHOD: `GET` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:16:57.471423 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cf2a48b-d7c3-4e2d-81de-b06d6a27ff96 None admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:16:57.471961 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cf2a48b-d7c3-4e2d-81de-b06d6a27ff96 None admin] PATH_INFO: `/v3/groups/569a366732204da3a5ab12f694275b62` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:16:57.473398 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cf2a48b-d7c3-4e2d-81de-b06d6a27ff96 None admin] RBAC: Authorizing `identity:get_group(group_id=569a366732204da3a5ab12f694275b62)` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:16:57.475429 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cf2a48b-d7c3-4e2d-81de-b06d6a27ff96 None admin] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:16:57.477046 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 229/454] 10.222.0.47 () {56 vars in 1180 bytes} [Thu Jul 21 08:16:57 2022] GET /identity/v3/groups/569a366732204da3a5ab12f694275b62 => generated 229 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 08:16:57.482941 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-291e5d4e-d08e-4db6-b148-f98ee0bcd3cf None admin] Authenticating user token {{(pid=80245) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:16:57.585267 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-291e5d4e-d08e-4db6-b148-f98ee0bcd3cf None admin] Validating token access rules against request {{(pid=80245) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:16:57.598743 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG dbcounter [-] [80245] Writing DB stats keystone:SELECT=100 {{(pid=80245) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:16:57.619609 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b95dfa5-6bfa-47dc-b6ba-36cbebaf984d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80245) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:16:57.620273 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b95dfa5-6bfa-47dc-b6ba-36cbebaf984d None admin] REQUEST_METHOD: `GET` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:16:57.620558 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b95dfa5-6bfa-47dc-b6ba-36cbebaf984d None admin] SCRIPT_NAME: `/identity` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:16:57.620837 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b95dfa5-6bfa-47dc-b6ba-36cbebaf984d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:16:57.621693 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b95dfa5-6bfa-47dc-b6ba-36cbebaf984d None admin] RBAC: Adding query filter params (group.id=569a366732204da3a5ab12f694275b62, role.id=e29870f6e9e2411a9db0d5bf45f32ca5, scope.project.id=42d15fc0515945acacbbffcf2356c7a5) {{(pid=80245) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 08:16:57.622005 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b95dfa5-6bfa-47dc-b6ba-36cbebaf984d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:16:57.623460 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b95dfa5-6bfa-47dc-b6ba-36cbebaf984d None admin] RBAC: Authorization granted {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:16:57.629133 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: [pid: 80245|app: 0|req: 226/455] 10.222.0.47 () {56 vars in 1376 bytes} [Thu Jul 21 08:16:57 2022] GET /identity/v3/role_assignments?group.id=569a366732204da3a5ab12f694275b62&scope.project.id=42d15fc0515945acacbbffcf2356c7a5&role.id=e29870f6e9e2411a9db0d5bf45f32ca5 => generated 611 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 08:17:00.042977 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8e6d6e1-ab6a-4a1b-9749-55810ad3a4ba None None] REQUEST_METHOD: `GET` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:17:00.044026 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8e6d6e1-ab6a-4a1b-9749-55810ad3a4ba None None] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:17:00.044515 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8e6d6e1-ab6a-4a1b-9749-55810ad3a4ba None None] PATH_INFO: `/` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:17:00.046659 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 230/456] 10.222.0.47 () {54 vars in 926 bytes} [Thu Jul 21 08:17:00 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 08:17:00.051661 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f7e619c-2cbc-4a43-bf20-0109c0360d7f None None] REQUEST_METHOD: `POST` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:17:00.052509 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f7e619c-2cbc-4a43-bf20-0109c0360d7f None None] SCRIPT_NAME: `/identity` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:17:00.053021 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f7e619c-2cbc-4a43-bf20-0109c0360d7f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:17:00.070742 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.auth.core [None req-6f7e619c-2cbc-4a43-bf20-0109c0360d7f None None] MFA Rules not processed for user `2dce40b8d74c41ec94cc36e6ff05d412`. Rule list: `[]` (Enabled: `True`). {{(pid=80245) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 08:17:00.076389 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.fernet_utils [None req-6f7e619c-2cbc-4a43-bf20-0109c0360d7f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80245) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 08:17:00.165173 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: [pid: 80245|app: 0|req: 227/457] 10.222.0.47 () {58 vars in 1023 bytes} [Thu Jul 21 08:17:00 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 21 08:17:00.221633 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8e6d6e1-ab6a-4a1b-9749-55810ad3a4ba None None] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:17:00.265503 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG dbcounter [-] [80244] Writing DB stats keystone:SELECT=100 {{(pid=80244) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:17:00.324872 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-777e6228-8198-451f-be3d-66c98cd44648 None None] REQUEST_METHOD: `GET` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:17:00.325620 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-777e6228-8198-451f-be3d-66c98cd44648 None None] SCRIPT_NAME: `/identity` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:17:00.326140 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-777e6228-8198-451f-be3d-66c98cd44648 None None] PATH_INFO: `/` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:17:00.328927 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: [pid: 80245|app: 0|req: 228/458] 10.222.0.47 () {54 vars in 926 bytes} [Thu Jul 21 08:17:00 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 21 08:17:00.334432 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91eaa593-47a2-464d-95fd-f7915d1216d2 None None] REQUEST_METHOD: `POST` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:17:00.335004 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91eaa593-47a2-464d-95fd-f7915d1216d2 None None] SCRIPT_NAME: `/identity` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:17:00.335522 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91eaa593-47a2-464d-95fd-f7915d1216d2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:17:00.352518 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.auth.core [None req-91eaa593-47a2-464d-95fd-f7915d1216d2 None None] MFA Rules not processed for user `2dce40b8d74c41ec94cc36e6ff05d412`. Rule list: `[]` (Enabled: `True`). {{(pid=80245) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 08:17:00.354089 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.fernet_utils [None req-91eaa593-47a2-464d-95fd-f7915d1216d2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80245) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 08:17:00.391750 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8e6d6e1-ab6a-4a1b-9749-55810ad3a4ba None None] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:17:00.452234 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: [pid: 80245|app: 0|req: 229/459] 10.222.0.47 () {58 vars in 1023 bytes} [Thu Jul 21 08:17:00 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 21 08:17:00.455357 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62c91ea3-3faf-4c5f-8657-61009becdc46 None None] REQUEST_METHOD: `GET` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:17:00.455357 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62c91ea3-3faf-4c5f-8657-61009becdc46 None None] SCRIPT_NAME: `/identity` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:17:00.455357 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62c91ea3-3faf-4c5f-8657-61009becdc46 None None] PATH_INFO: `/` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:17:00.458262 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83596c86-6626-419b-9995-c7d21ad39654 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:17:00.458262 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83596c86-6626-419b-9995-c7d21ad39654 None admin] REQUEST_METHOD: `GET` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:17:00.458262 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83596c86-6626-419b-9995-c7d21ad39654 None admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:17:00.458262 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83596c86-6626-419b-9995-c7d21ad39654 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:17:00.464179 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: [pid: 80245|app: 0|req: 230/460] 10.222.0.47 () {54 vars in 926 bytes} [Thu Jul 21 08:17:00 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 08:17:00.465271 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-261457a9-3d8c-4f3b-a29f-2f0e87e5561f None None] REQUEST_METHOD: `POST` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:17:00.465558 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-261457a9-3d8c-4f3b-a29f-2f0e87e5561f None None] SCRIPT_NAME: `/identity` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:17:00.465837 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-261457a9-3d8c-4f3b-a29f-2f0e87e5561f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:17:00.478322 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83596c86-6626-419b-9995-c7d21ad39654 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:17:00.479829 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83596c86-6626-419b-9995-c7d21ad39654 None admin] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:17:00.495811 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.auth.core [None req-261457a9-3d8c-4f3b-a29f-2f0e87e5561f None None] MFA Rules not processed for user `2dce40b8d74c41ec94cc36e6ff05d412`. Rule list: `[]` (Enabled: `True`). {{(pid=80245) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 08:17:00.497068 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.fernet_utils [None req-261457a9-3d8c-4f3b-a29f-2f0e87e5561f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80245) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 08:17:00.512346 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application [None req-83596c86-6626-419b-9995-c7d21ad39654 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 08:17:00.512346 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 08:17:00.512346 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 08:17:00.512346 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 08:17:00.512346 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 08:17:00.512346 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 08:17:00.512346 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 08:17:00.512346 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 08:17:00.512346 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 08:17:00.512346 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 08:17:00.512346 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 08:17:00.512346 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 08:17:00.512346 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 21 08:17:00.512346 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 21 08:17:00.512346 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 21 08:17:00.513218 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 21 08:17:00.513218 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 08:17:00.513218 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 08:17:00.513218 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 08:17:00.513218 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 08:17:00.513218 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 08:17:00.513218 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 08:17:00.513218 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 08:17:00.513218 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application with Lock( Jul 21 08:17:00.513218 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 08:17:00.513218 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self._enter() Jul 21 08:17:00.513218 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 08:17:00.513218 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 08:17:00.513218 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 08:17:00.513218 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self.creator() Jul 21 08:17:00.513218 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 08:17:00.513218 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application created_value = creator( Jul 21 08:17:00.514063 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 21 08:17:00.514063 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 21 08:17:00.514063 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 08:17:00.514063 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application Jul 21 08:17:00.516662 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 231/461] 10.222.0.47 () {56 vars in 1108 bytes} [Thu Jul 21 08:17:00 2022] GET /identity/v3/domains/Default => generated 87 bytes in 295 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 08:17:00.517866 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83596c86-6626-419b-9995-c7d21ad39654 None admin] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:17:00.533015 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG dbcounter [-] [80245] Writing DB stats keystone:SELECT=100 {{(pid=80245) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:17:00.574406 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: [pid: 80245|app: 0|req: 231/462] 10.222.0.47 () {58 vars in 1023 bytes} [Thu Jul 21 08:17:00 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 21 08:17:00.580126 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-261457a9-3d8c-4f3b-a29f-2f0e87e5561f None None] Authenticating user token {{(pid=80245) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:17:00.631005 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG dbcounter [-] [80244] Writing DB stats keystone:SELECT=100 {{(pid=80244) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:17:00.654832 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83596c86-6626-419b-9995-c7d21ad39654 None admin] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:17:00.706264 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01ef115f-9246-4cc1-9471-5573aa12d602 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:17:00.707376 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01ef115f-9246-4cc1-9471-5573aa12d602 None admin] REQUEST_METHOD: `GET` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:17:00.707823 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01ef115f-9246-4cc1-9471-5573aa12d602 None admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:17:00.709349 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01ef115f-9246-4cc1-9471-5573aa12d602 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:17:00.722389 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-261457a9-3d8c-4f3b-a29f-2f0e87e5561f None None] Validating token access rules against request {{(pid=80245) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:17:00.726319 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01ef115f-9246-4cc1-9471-5573aa12d602 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:17:00.728052 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01ef115f-9246-4cc1-9471-5573aa12d602 None admin] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:17:00.747225 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application [None req-01ef115f-9246-4cc1-9471-5573aa12d602 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 08:17:00.747225 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 08:17:00.747225 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 08:17:00.747225 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 08:17:00.747225 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 08:17:00.747225 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 08:17:00.747225 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 08:17:00.747225 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 08:17:00.747225 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 08:17:00.747225 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 08:17:00.747225 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 08:17:00.747225 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 08:17:00.747225 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 21 08:17:00.747225 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 21 08:17:00.747225 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 21 08:17:00.748185 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 21 08:17:00.748185 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 08:17:00.748185 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 08:17:00.748185 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 08:17:00.748185 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 08:17:00.748185 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 08:17:00.748185 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 08:17:00.748185 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 08:17:00.748185 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application with Lock( Jul 21 08:17:00.748185 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 08:17:00.748185 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self._enter() Jul 21 08:17:00.748185 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 08:17:00.748185 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 08:17:00.748185 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 08:17:00.748185 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self.creator() Jul 21 08:17:00.748185 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 08:17:00.748185 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application created_value = creator( Jul 21 08:17:00.749008 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 21 08:17:00.749008 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 21 08:17:00.749008 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 08:17:00.749008 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application Jul 21 08:17:00.751108 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 232/463] 10.222.0.47 () {56 vars in 1108 bytes} [Thu Jul 21 08:17:00 2022] GET /identity/v3/domains/Default => generated 87 bytes in 234 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 08:17:00.757719 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01ef115f-9246-4cc1-9471-5573aa12d602 None admin] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:17:00.765548 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-794ecc79-f81b-4262-92f4-d864f3d7871e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80245) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:17:00.766234 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-794ecc79-f81b-4262-92f4-d864f3d7871e None admin] REQUEST_METHOD: `GET` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:17:00.766700 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-794ecc79-f81b-4262-92f4-d864f3d7871e None admin] SCRIPT_NAME: `/identity` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:17:00.766987 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-794ecc79-f81b-4262-92f4-d864f3d7871e None admin] PATH_INFO: `/v3/domains` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:17:00.767758 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-794ecc79-f81b-4262-92f4-d864f3d7871e None admin] RBAC: Adding query filter params (name=Default) {{(pid=80245) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 08:17:00.768069 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-794ecc79-f81b-4262-92f4-d864f3d7871e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:17:00.769460 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-794ecc79-f81b-4262-92f4-d864f3d7871e None admin] RBAC: Authorization granted {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:17:00.783179 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: [pid: 80245|app: 0|req: 232/464] 10.222.0.47 () {56 vars in 1109 bytes} [Thu Jul 21 08:17:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 08:17:00.784172 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-794ecc79-f81b-4262-92f4-d864f3d7871e None admin] Authenticating user token {{(pid=80245) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:17:00.814672 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG dbcounter [-] [80245] Writing DB stats keystone:SELECT=100 {{(pid=80245) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:17:00.871747 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01ef115f-9246-4cc1-9471-5573aa12d602 None admin] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:17:00.894293 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-794ecc79-f81b-4262-92f4-d864f3d7871e None admin] Validating token access rules against request {{(pid=80245) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:17:00.910151 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4ff658b-7f41-4eb4-8662-265de3a9856a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:17:00.911823 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4ff658b-7f41-4eb4-8662-265de3a9856a None admin] REQUEST_METHOD: `GET` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:17:00.912247 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4ff658b-7f41-4eb4-8662-265de3a9856a None admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:17:00.912688 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4ff658b-7f41-4eb4-8662-265de3a9856a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:17:00.916415 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG dbcounter [-] [80244] Writing DB stats keystone:SELECT=100 {{(pid=80244) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:17:00.926251 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4ff658b-7f41-4eb4-8662-265de3a9856a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:17:00.928796 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4ff658b-7f41-4eb4-8662-265de3a9856a None admin] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:17:00.932006 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9c4186f-acf4-4a06-bbc5-b4d074c42f9e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80245) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:17:00.932679 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9c4186f-acf4-4a06-bbc5-b4d074c42f9e None admin] REQUEST_METHOD: `GET` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:17:00.932969 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9c4186f-acf4-4a06-bbc5-b4d074c42f9e None admin] SCRIPT_NAME: `/identity` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:17:00.933248 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9c4186f-acf4-4a06-bbc5-b4d074c42f9e None admin] PATH_INFO: `/v3/domains` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:17:00.933970 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9c4186f-acf4-4a06-bbc5-b4d074c42f9e None admin] RBAC: Adding query filter params (name=Default) {{(pid=80245) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 08:17:00.934278 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9c4186f-acf4-4a06-bbc5-b4d074c42f9e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:17:00.935683 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9c4186f-acf4-4a06-bbc5-b4d074c42f9e None admin] RBAC: Authorization granted {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:17:00.947643 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: [pid: 80245|app: 0|req: 233/465] 10.222.0.47 () {56 vars in 1109 bytes} [Thu Jul 21 08:17:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 08:17:00.948441 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9c4186f-acf4-4a06-bbc5-b4d074c42f9e None admin] Authenticating user token {{(pid=80245) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:17:00.949344 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application [None req-a4ff658b-7f41-4eb4-8662-265de3a9856a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 08:17:00.949344 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 08:17:00.949344 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 08:17:00.949344 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 08:17:00.949344 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 08:17:00.949344 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 08:17:00.949344 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 08:17:00.949344 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 08:17:00.949344 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 08:17:00.949344 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 08:17:00.949344 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 08:17:00.949344 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 08:17:00.949344 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 21 08:17:00.949344 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 21 08:17:00.949344 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 21 08:17:00.950252 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 21 08:17:00.950252 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 08:17:00.950252 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 08:17:00.950252 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 08:17:00.950252 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 08:17:00.950252 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 08:17:00.950252 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 08:17:00.950252 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 08:17:00.950252 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application with Lock( Jul 21 08:17:00.950252 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 08:17:00.950252 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self._enter() Jul 21 08:17:00.950252 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 08:17:00.950252 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 08:17:00.950252 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 08:17:00.950252 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self.creator() Jul 21 08:17:00.950252 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 08:17:00.950252 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application created_value = creator( Jul 21 08:17:00.951057 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 21 08:17:00.951057 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 21 08:17:00.951057 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 08:17:00.951057 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application Jul 21 08:17:00.955256 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 233/466] 10.222.0.47 () {56 vars in 1108 bytes} [Thu Jul 21 08:17:00 2022] GET /identity/v3/domains/Default => generated 87 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 08:17:00.956634 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22544f5f-7565-47bf-8af6-47be5c25e338 None None] REQUEST_METHOD: `GET` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:17:00.956925 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22544f5f-7565-47bf-8af6-47be5c25e338 None None] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:17:00.957265 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22544f5f-7565-47bf-8af6-47be5c25e338 None None] PATH_INFO: `/` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:17:00.962659 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 234/467] 10.222.0.47 () {54 vars in 926 bytes} [Thu Jul 21 08:17:00 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 08:17:00.963239 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22544f5f-7565-47bf-8af6-47be5c25e338 None None] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:17:01.050619 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG dbcounter [-] [80245] Writing DB stats keystone:SELECT=100 {{(pid=80245) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:17:01.064930 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9c4186f-acf4-4a06-bbc5-b4d074c42f9e None admin] Validating token access rules against request {{(pid=80245) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:17:01.067308 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22544f5f-7565-47bf-8af6-47be5c25e338 None None] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:17:01.100330 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de019618-56a1-42fc-b975-17e70c210916 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80245) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:17:01.101017 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de019618-56a1-42fc-b975-17e70c210916 None admin] REQUEST_METHOD: `POST` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:17:01.101295 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de019618-56a1-42fc-b975-17e70c210916 None admin] SCRIPT_NAME: `/identity` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:17:01.101569 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de019618-56a1-42fc-b975-17e70c210916 None admin] PATH_INFO: `/v3/users` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:17:01.102276 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de019618-56a1-42fc-b975-17e70c210916 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:17:01.103724 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f48f296c-109c-4533-a24c-ca8cf8ca76e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:17:01.103874 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de019618-56a1-42fc-b975-17e70c210916 None admin] RBAC: Authorization granted {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:17:01.104669 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f48f296c-109c-4533-a24c-ca8cf8ca76e0 None admin] REQUEST_METHOD: `POST` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:17:01.105204 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f48f296c-109c-4533-a24c-ca8cf8ca76e0 None admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:17:01.105624 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f48f296c-109c-4533-a24c-ca8cf8ca76e0 None admin] PATH_INFO: `/v3/users` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:17:01.106524 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f48f296c-109c-4533-a24c-ca8cf8ca76e0 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:17:01.108153 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f48f296c-109c-4533-a24c-ca8cf8ca76e0 None admin] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:17:01.126363 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: [pid: 80245|app: 0|req: 234/468] 10.222.0.47 () {60 vars in 1132 bytes} [Thu Jul 21 08:17:00 2022] POST /identity/v3/users => generated 246 bytes in 178 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 08:17:01.127128 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de019618-56a1-42fc-b975-17e70c210916 None admin] Authenticating user token {{(pid=80245) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:17:01.136663 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 235/469] 10.222.0.47 () {60 vars in 1132 bytes} [Thu Jul 21 08:17:00 2022] POST /identity/v3/users => generated 248 bytes in 173 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 08:17:01.138270 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebb6b00e-bef3-4ef2-9863-fe2767423413 None None] REQUEST_METHOD: `POST` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:17:01.138821 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebb6b00e-bef3-4ef2-9863-fe2767423413 None None] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:17:01.139253 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebb6b00e-bef3-4ef2-9863-fe2767423413 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:17:01.162038 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.auth.core [None req-ebb6b00e-bef3-4ef2-9863-fe2767423413 None None] MFA Rules not processed for user `2dce40b8d74c41ec94cc36e6ff05d412`. Rule list: `[]` (Enabled: `True`). {{(pid=80244) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 08:17:01.163937 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.fernet_utils [None req-ebb6b00e-bef3-4ef2-9863-fe2767423413 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80244) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 08:17:01.247829 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG dbcounter [-] [80244] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80244) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:17:01.323004 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 236/470] 10.222.0.47 () {58 vars in 1023 bytes} [Thu Jul 21 08:17:01 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 185 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 21 08:17:01.363111 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de019618-56a1-42fc-b975-17e70c210916 None admin] Validating token access rules against request {{(pid=80245) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:17:01.374600 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebb6b00e-bef3-4ef2-9863-fe2767423413 None None] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:17:01.422550 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a24b15bf-27ca-48e3-a15f-306137cc753c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80245) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:17:01.424720 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a24b15bf-27ca-48e3-a15f-306137cc753c None admin] REQUEST_METHOD: `GET` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:17:01.424720 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a24b15bf-27ca-48e3-a15f-306137cc753c None admin] SCRIPT_NAME: `/identity` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:17:01.424720 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a24b15bf-27ca-48e3-a15f-306137cc753c None admin] PATH_INFO: `/v3/domains` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:17:01.425068 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a24b15bf-27ca-48e3-a15f-306137cc753c None admin] RBAC: Adding query filter params (name=Default) {{(pid=80245) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 08:17:01.425373 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a24b15bf-27ca-48e3-a15f-306137cc753c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:17:01.428042 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a24b15bf-27ca-48e3-a15f-306137cc753c None admin] RBAC: Authorization granted {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:17:01.446676 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: [pid: 80245|app: 0|req: 235/471] 10.222.0.47 () {56 vars in 1109 bytes} [Thu Jul 21 08:17:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 320 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 08:17:01.454417 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a24b15bf-27ca-48e3-a15f-306137cc753c None admin] Authenticating user token {{(pid=80245) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:17:01.460940 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG dbcounter [-] [80245] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80245) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:17:01.529689 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebb6b00e-bef3-4ef2-9863-fe2767423413 None None] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:17:01.588796 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5beb6989-679d-46c2-9f72-06074f3823cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:17:01.589607 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5beb6989-679d-46c2-9f72-06074f3823cf None admin] REQUEST_METHOD: `GET` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:17:01.589890 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5beb6989-679d-46c2-9f72-06074f3823cf None admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:17:01.590165 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5beb6989-679d-46c2-9f72-06074f3823cf None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:17:01.607995 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5beb6989-679d-46c2-9f72-06074f3823cf None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:17:01.609466 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5beb6989-679d-46c2-9f72-06074f3823cf None admin] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:17:01.637531 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application [None req-5beb6989-679d-46c2-9f72-06074f3823cf None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 08:17:01.637531 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 08:17:01.637531 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 08:17:01.637531 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 08:17:01.637531 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 08:17:01.637531 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 08:17:01.637531 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 08:17:01.637531 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 08:17:01.637531 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 08:17:01.637531 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 08:17:01.637531 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 08:17:01.637531 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 08:17:01.637531 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 21 08:17:01.637531 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 21 08:17:01.637531 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 21 08:17:01.638371 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 21 08:17:01.638371 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 08:17:01.638371 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 08:17:01.638371 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 08:17:01.638371 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 08:17:01.638371 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 08:17:01.638371 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 08:17:01.638371 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 08:17:01.638371 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application with Lock( Jul 21 08:17:01.638371 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 08:17:01.638371 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self._enter() Jul 21 08:17:01.638371 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 08:17:01.638371 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 08:17:01.638371 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 08:17:01.638371 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self.creator() Jul 21 08:17:01.638371 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 08:17:01.638371 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application created_value = creator( Jul 21 08:17:01.639222 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 21 08:17:01.639222 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 21 08:17:01.639222 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 08:17:01.639222 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application Jul 21 08:17:01.639222 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 237/472] 10.222.0.47 () {56 vars in 1108 bytes} [Thu Jul 21 08:17:01 2022] GET /identity/v3/domains/Default => generated 87 bytes in 265 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 08:17:01.646786 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a24b15bf-27ca-48e3-a15f-306137cc753c None admin] Validating token access rules against request {{(pid=80245) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:17:01.654579 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5beb6989-679d-46c2-9f72-06074f3823cf None admin] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:17:01.701202 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG dbcounter [-] [80244] Writing DB stats keystone:SELECT=100 {{(pid=80244) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:17:01.707198 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7a0f289-4995-425b-8ac9-6d1366053e44 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80245) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:17:01.707943 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7a0f289-4995-425b-8ac9-6d1366053e44 None admin] REQUEST_METHOD: `POST` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:17:01.708227 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7a0f289-4995-425b-8ac9-6d1366053e44 None admin] SCRIPT_NAME: `/identity` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:17:01.708497 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7a0f289-4995-425b-8ac9-6d1366053e44 None admin] PATH_INFO: `/v3/users` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:17:01.709229 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7a0f289-4995-425b-8ac9-6d1366053e44 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:17:01.710668 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7a0f289-4995-425b-8ac9-6d1366053e44 None admin] RBAC: Authorization granted {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:17:01.753041 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: [pid: 80245|app: 0|req: 236/473] 10.222.0.47 () {60 vars in 1132 bytes} [Thu Jul 21 08:17:01 2022] POST /identity/v3/users => generated 248 bytes in 299 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 21 08:17:01.763625 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7694b029-d1e3-4036-ae1f-0ce3729cc0d5 None None] REQUEST_METHOD: `GET` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:17:01.763902 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7694b029-d1e3-4036-ae1f-0ce3729cc0d5 None None] SCRIPT_NAME: `/identity` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:17:01.764300 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7694b029-d1e3-4036-ae1f-0ce3729cc0d5 None None] PATH_INFO: `/` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:17:01.766448 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: [pid: 80245|app: 0|req: 237/474] 10.222.0.47 () {54 vars in 926 bytes} [Thu Jul 21 08:17:01 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 21 08:17:01.779701 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d14f047-76cd-4cad-8110-1f1311b2fe6c None None] REQUEST_METHOD: `POST` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:17:01.780095 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d14f047-76cd-4cad-8110-1f1311b2fe6c None None] SCRIPT_NAME: `/identity` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:17:01.780324 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d14f047-76cd-4cad-8110-1f1311b2fe6c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:17:01.819596 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.auth.core [None req-5d14f047-76cd-4cad-8110-1f1311b2fe6c None None] MFA Rules not processed for user `2dce40b8d74c41ec94cc36e6ff05d412`. Rule list: `[]` (Enabled: `True`). {{(pid=80245) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 08:17:01.820881 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.fernet_utils [None req-5d14f047-76cd-4cad-8110-1f1311b2fe6c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80245) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 08:17:01.845693 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5beb6989-679d-46c2-9f72-06074f3823cf None admin] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:17:01.907768 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG dbcounter [-] [80245] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80245) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:17:01.917912 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: [pid: 80245|app: 0|req: 238/475] 10.222.0.47 () {58 vars in 1023 bytes} [Thu Jul 21 08:17:01 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 139 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 21 08:17:01.925579 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6df7bfba-66a1-4a28-a120-b1ac4326a625 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:17:01.926262 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6df7bfba-66a1-4a28-a120-b1ac4326a625 None admin] REQUEST_METHOD: `GET` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:17:01.926549 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6df7bfba-66a1-4a28-a120-b1ac4326a625 None admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:17:01.926866 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6df7bfba-66a1-4a28-a120-b1ac4326a625 None admin] PATH_INFO: `/v3/domains` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:17:01.927668 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6df7bfba-66a1-4a28-a120-b1ac4326a625 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80244) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 08:17:01.928038 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6df7bfba-66a1-4a28-a120-b1ac4326a625 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:17:01.929469 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6df7bfba-66a1-4a28-a120-b1ac4326a625 None admin] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:17:01.944067 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 238/476] 10.222.0.47 () {56 vars in 1109 bytes} [Thu Jul 21 08:17:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 295 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 08:17:01.952213 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6df7bfba-66a1-4a28-a120-b1ac4326a625 None admin] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:17:01.952356 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d14f047-76cd-4cad-8110-1f1311b2fe6c None None] Authenticating user token {{(pid=80245) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:17:02.048423 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG dbcounter [-] [80244] Writing DB stats keystone:SELECT=100 {{(pid=80244) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:17:02.065960 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6df7bfba-66a1-4a28-a120-b1ac4326a625 None admin] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:17:02.108912 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05ca7df9-14ef-4e3a-bde1-65895e33dfce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:17:02.109640 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05ca7df9-14ef-4e3a-bde1-65895e33dfce None admin] REQUEST_METHOD: `POST` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:17:02.109920 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05ca7df9-14ef-4e3a-bde1-65895e33dfce None admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:17:02.110200 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05ca7df9-14ef-4e3a-bde1-65895e33dfce None admin] PATH_INFO: `/v3/roles` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:17:02.110979 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05ca7df9-14ef-4e3a-bde1-65895e33dfce None admin] RBAC: Authorizing `identity:create_role()` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:17:02.112421 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05ca7df9-14ef-4e3a-bde1-65895e33dfce None admin] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:17:02.122388 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.sql.core [None req-05ca7df9-14ef-4e3a-bde1-65895e33dfce None admin] Conflict role: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Jul 21 08:17:02.122388 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: [SQL: INSERT INTO `role` (id, name, description, extra) VALUES (%(id)s, %(name)s, %(description)s, %(extra)s)] Jul 21 08:17:02.122388 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: [parameters: {'id': 'b86c981f7501481f86c93534337fc390', 'name': 'anotherrole', 'description': None, 'extra': '{}'}] Jul 21 08:17:02.122388 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=80244) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Jul 21 08:17:02.142928 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application [None req-05ca7df9-14ef-4e3a-bde1-65895e33dfce None admin] Conflict occurred attempting to store role - Duplicate entry found with name anotherrole.: keystone.exception.Conflict: Conflict occurred attempting to store role - Duplicate entry found with name anotherrole. Jul 21 08:17:02.142928 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 08:17:02.142928 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Jul 21 08:17:02.142928 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application self.dialect.do_execute( Jul 21 08:17:02.142928 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Jul 21 08:17:02.142928 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Jul 21 08:17:02.142928 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Jul 21 08:17:02.142928 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application result = self._query(query) Jul 21 08:17:02.142928 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Jul 21 08:17:02.142928 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application conn.query(q) Jul 21 08:17:02.142928 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Jul 21 08:17:02.142928 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Jul 21 08:17:02.142928 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Jul 21 08:17:02.142928 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application result.read() Jul 21 08:17:02.142928 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Jul 21 08:17:02.143874 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Jul 21 08:17:02.143874 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Jul 21 08:17:02.143874 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application packet.raise_for_error() Jul 21 08:17:02.143874 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Jul 21 08:17:02.143874 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Jul 21 08:17:02.143874 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Jul 21 08:17:02.143874 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Jul 21 08:17:02.143874 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Jul 21 08:17:02.143874 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application Jul 21 08:17:02.143874 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Jul 21 08:17:02.143874 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application Jul 21 08:17:02.143874 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 08:17:02.143874 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Jul 21 08:17:02.143874 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return method(*args, **kwargs) Jul 21 08:17:02.143874 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 33, in create_role Jul 21 08:17:02.143874 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return ref.to_dict() Jul 21 08:17:02.143874 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Jul 21 08:17:02.143874 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application next(self.gen) Jul 21 08:17:02.144789 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Jul 21 08:17:02.144789 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application yield resource Jul 21 08:17:02.144789 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Jul 21 08:17:02.144789 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application next(self.gen) Jul 21 08:17:02.144789 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Jul 21 08:17:02.144789 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application self.session.rollback() Jul 21 08:17:02.144789 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Jul 21 08:17:02.144789 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application self.force_reraise() Jul 21 08:17:02.144789 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Jul 21 08:17:02.144789 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application raise self.value Jul 21 08:17:02.144789 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Jul 21 08:17:02.144789 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Jul 21 08:17:02.144789 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Jul 21 08:17:02.144789 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application session.commit() Jul 21 08:17:02.144789 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Jul 21 08:17:02.144789 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Jul 21 08:17:02.144789 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Jul 21 08:17:02.146229 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application self._prepare_impl() Jul 21 08:17:02.146229 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Jul 21 08:17:02.146229 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application self.session.flush() Jul 21 08:17:02.146229 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Jul 21 08:17:02.146229 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application self._flush(objects) Jul 21 08:17:02.146229 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Jul 21 08:17:02.146229 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Jul 21 08:17:02.146229 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Jul 21 08:17:02.146229 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application compat.raise_( Jul 21 08:17:02.146229 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Jul 21 08:17:02.146229 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application raise exception Jul 21 08:17:02.146229 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Jul 21 08:17:02.146229 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application flush_context.execute() Jul 21 08:17:02.146229 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Jul 21 08:17:02.146229 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application rec.execute(self) Jul 21 08:17:02.146229 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Jul 21 08:17:02.146229 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Jul 21 08:17:02.147070 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Jul 21 08:17:02.147070 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application _emit_insert_statements( Jul 21 08:17:02.147070 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1097, in _emit_insert_statements Jul 21 08:17:02.147070 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application c = connection._execute_20( Jul 21 08:17:02.147070 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1631, in _execute_20 Jul 21 08:17:02.147070 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Jul 21 08:17:02.147070 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 332, in _execute_on_connection Jul 21 08:17:02.147070 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Jul 21 08:17:02.147070 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1498, in _execute_clauseelement Jul 21 08:17:02.147070 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application ret = self._execute_context( Jul 21 08:17:02.147070 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1862, in _execute_context Jul 21 08:17:02.147070 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Jul 21 08:17:02.147070 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2041, in _handle_dbapi_exception Jul 21 08:17:02.147070 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Jul 21 08:17:02.147070 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Jul 21 08:17:02.147070 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application raise exception Jul 21 08:17:02.149475 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Jul 21 08:17:02.149475 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application self.dialect.do_execute( Jul 21 08:17:02.149475 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Jul 21 08:17:02.149475 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Jul 21 08:17:02.149475 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Jul 21 08:17:02.149475 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application result = self._query(query) Jul 21 08:17:02.149475 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Jul 21 08:17:02.149475 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application conn.query(q) Jul 21 08:17:02.149475 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Jul 21 08:17:02.149475 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Jul 21 08:17:02.149475 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Jul 21 08:17:02.149475 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application result.read() Jul 21 08:17:02.149475 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Jul 21 08:17:02.149475 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Jul 21 08:17:02.149475 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Jul 21 08:17:02.149475 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application packet.raise_for_error() Jul 21 08:17:02.151438 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Jul 21 08:17:02.151438 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Jul 21 08:17:02.151438 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Jul 21 08:17:02.151438 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Jul 21 08:17:02.151438 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Jul 21 08:17:02.151438 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application [SQL: INSERT INTO `role` (id, name, description, extra) VALUES (%(id)s, %(name)s, %(description)s, %(extra)s)] Jul 21 08:17:02.151438 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application [parameters: {'id': 'b86c981f7501481f86c93534337fc390', 'name': 'anotherrole', 'description': None, 'extra': '{}'}] Jul 21 08:17:02.151438 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Jul 21 08:17:02.151438 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application Jul 21 08:17:02.151438 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 21 08:17:02.151438 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application Jul 21 08:17:02.151438 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 08:17:02.151438 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 08:17:02.151438 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 08:17:02.151438 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 08:17:02.151438 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 08:17:02.153580 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 08:17:02.153580 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 08:17:02.153580 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 08:17:02.153580 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 08:17:02.153580 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 08:17:02.153580 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 08:17:02.153580 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 115, in post Jul 21 08:17:02.153580 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application ref = PROVIDERS.role_api.create_role( Jul 21 08:17:02.153580 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 08:17:02.153580 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 08:17:02.153580 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1298, in create_role Jul 21 08:17:02.153580 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application ret = self.driver.create_role(role_id, role) Jul 21 08:17:02.153580 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Jul 21 08:17:02.153580 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Jul 21 08:17:02.153580 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store role - Duplicate entry found with name anotherrole. Jul 21 08:17:02.153580 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application Jul 21 08:17:02.160461 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 239/477] 10.222.0.47 () {60 vars in 1131 bytes} [Thu Jul 21 08:17:01 2022] POST /identity/v3/roles => generated 144 bytes in 193 msecs (HTTP/1.1 409) 5 headers in 183 bytes (1 switches on core 0) Jul 21 08:17:02.160461 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05ca7df9-14ef-4e3a-bde1-65895e33dfce None admin] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:17:02.160685 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d14f047-76cd-4cad-8110-1f1311b2fe6c None None] Validating token access rules against request {{(pid=80245) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:17:02.224727 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94128be3-444a-40af-b5e8-6eeb9634eb13 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80245) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:17:02.225345 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94128be3-444a-40af-b5e8-6eeb9634eb13 None admin] REQUEST_METHOD: `POST` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:17:02.225630 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94128be3-444a-40af-b5e8-6eeb9634eb13 None admin] SCRIPT_NAME: `/identity` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:17:02.225910 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94128be3-444a-40af-b5e8-6eeb9634eb13 None admin] PATH_INFO: `/v3/users` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:17:02.227096 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94128be3-444a-40af-b5e8-6eeb9634eb13 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:17:02.228881 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94128be3-444a-40af-b5e8-6eeb9634eb13 None admin] RBAC: Authorization granted {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:17:02.261887 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: [pid: 80245|app: 0|req: 239/478] 10.222.0.47 () {60 vars in 1132 bytes} [Thu Jul 21 08:17:01 2022] POST /identity/v3/users => generated 249 bytes in 310 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 08:17:02.305656 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05ca7df9-14ef-4e3a-bde1-65895e33dfce None admin] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:17:02.345016 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40841abb-655c-4779-9034-b062e17709fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:17:02.346131 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40841abb-655c-4779-9034-b062e17709fe None admin] REQUEST_METHOD: `GET` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:17:02.346562 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40841abb-655c-4779-9034-b062e17709fe None admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:17:02.346983 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40841abb-655c-4779-9034-b062e17709fe None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:17:02.359194 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40841abb-655c-4779-9034-b062e17709fe None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 21 08:17:02.367986 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application [None req-40841abb-655c-4779-9034-b062e17709fe None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 21 08:17:02.367986 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 08:17:02.367986 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 08:17:02.367986 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 08:17:02.367986 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 08:17:02.367986 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 08:17:02.367986 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 08:17:02.367986 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 08:17:02.367986 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 08:17:02.367986 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 08:17:02.367986 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 08:17:02.367986 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 08:17:02.367986 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 21 08:17:02.367986 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 21 08:17:02.367986 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 21 08:17:02.367986 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 21 08:17:02.369101 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 21 08:17:02.369101 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 21 08:17:02.369101 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 21 08:17:02.369101 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 21 08:17:02.369101 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 08:17:02.369101 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 08:17:02.369101 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 08:17:02.369101 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 08:17:02.369101 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 08:17:02.369101 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 08:17:02.369101 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 08:17:02.369101 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application with Lock( Jul 21 08:17:02.369101 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 08:17:02.369101 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self._enter() Jul 21 08:17:02.369101 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 08:17:02.369101 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 08:17:02.370063 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 08:17:02.370063 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self.creator() Jul 21 08:17:02.370063 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 08:17:02.370063 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application created_value = creator( Jul 21 08:17:02.370063 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 21 08:17:02.370063 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 21 08:17:02.370063 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 21 08:17:02.370063 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 21 08:17:02.370063 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 21 08:17:02.370063 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 21 08:17:02.370063 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 21 08:17:02.370063 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application Jul 21 08:17:02.375439 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 240/479] 10.222.0.47 () {56 vars in 1114 bytes} [Thu Jul 21 08:17:02 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 219 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 21 08:17:02.381155 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94128be3-444a-40af-b5e8-6eeb9634eb13 None admin] Authenticating user token {{(pid=80245) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:17:02.485376 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG dbcounter [-] [80245] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80245) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:17:02.556473 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94128be3-444a-40af-b5e8-6eeb9634eb13 None admin] Validating token access rules against request {{(pid=80245) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:17:02.594472 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76949ce4-0f0d-480c-a9ae-ab0de2eff604 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80245) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:17:02.595157 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76949ce4-0f0d-480c-a9ae-ab0de2eff604 None admin] REQUEST_METHOD: `GET` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:17:02.595462 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76949ce4-0f0d-480c-a9ae-ab0de2eff604 None admin] SCRIPT_NAME: `/identity` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:17:02.595740 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76949ce4-0f0d-480c-a9ae-ab0de2eff604 None admin] PATH_INFO: `/v3/roles` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:17:02.596496 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76949ce4-0f0d-480c-a9ae-ab0de2eff604 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80245) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 08:17:02.596800 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76949ce4-0f0d-480c-a9ae-ab0de2eff604 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:17:02.598183 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76949ce4-0f0d-480c-a9ae-ab0de2eff604 None admin] RBAC: Authorization granted {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:17:02.607781 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: [pid: 80245|app: 0|req: 240/480] 10.222.0.47 () {56 vars in 1111 bytes} [Thu Jul 21 08:17:02 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 228 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 08:17:03.802150 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f97031d7-f01d-410a-a7c0-23935b32d0b2 None None] REQUEST_METHOD: `GET` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:17:03.803154 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f97031d7-f01d-410a-a7c0-23935b32d0b2 None None] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:17:03.803676 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f97031d7-f01d-410a-a7c0-23935b32d0b2 None None] PATH_INFO: `/` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:17:03.805571 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 241/481] 10.222.0.47 () {54 vars in 926 bytes} [Thu Jul 21 08:17:03 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 08:17:03.810549 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24a6961e-4bde-40a0-8747-be7ca2bc14d1 None None] REQUEST_METHOD: `POST` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:17:03.811004 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24a6961e-4bde-40a0-8747-be7ca2bc14d1 None None] SCRIPT_NAME: `/identity` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:17:03.811112 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24a6961e-4bde-40a0-8747-be7ca2bc14d1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:17:03.837123 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.auth.core [None req-24a6961e-4bde-40a0-8747-be7ca2bc14d1 None None] MFA Rules not processed for user `2dce40b8d74c41ec94cc36e6ff05d412`. Rule list: `[]` (Enabled: `True`). {{(pid=80245) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 08:17:03.838316 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.fernet_utils [None req-24a6961e-4bde-40a0-8747-be7ca2bc14d1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80245) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 08:17:03.872780 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f8b87a2-dff9-4f43-9a73-aab88b3b4fef None None] REQUEST_METHOD: `GET` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:17:03.873389 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f8b87a2-dff9-4f43-9a73-aab88b3b4fef None None] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:17:03.873896 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f8b87a2-dff9-4f43-9a73-aab88b3b4fef None None] PATH_INFO: `/` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:17:03.880500 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 242/482] 10.222.0.47 () {54 vars in 926 bytes} [Thu Jul 21 08:17:03 2022] GET /identity => generated 270 bytes in 9 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 08:17:03.886054 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e20d00c0-f2c2-4c43-bbc1-a7983586e907 None None] REQUEST_METHOD: `POST` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:17:03.886346 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e20d00c0-f2c2-4c43-bbc1-a7983586e907 None None] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:17:03.886622 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e20d00c0-f2c2-4c43-bbc1-a7983586e907 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:17:03.904978 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.auth.core [None req-e20d00c0-f2c2-4c43-bbc1-a7983586e907 None None] MFA Rules not processed for user `2dce40b8d74c41ec94cc36e6ff05d412`. Rule list: `[]` (Enabled: `True`). {{(pid=80244) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 08:17:03.906289 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.fernet_utils [None req-e20d00c0-f2c2-4c43-bbc1-a7983586e907 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80244) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 08:17:03.909366 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG dbcounter [-] [80244] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80244) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:17:03.930814 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: [pid: 80245|app: 0|req: 241/483] 10.222.0.47 () {58 vars in 1023 bytes} [Thu Jul 21 08:17:03 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 21 08:17:03.998752 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24a6961e-4bde-40a0-8747-be7ca2bc14d1 None None] Authenticating user token {{(pid=80245) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:17:04.018634 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 243/484] 10.222.0.47 () {58 vars in 1023 bytes} [Thu Jul 21 08:17:03 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 21 08:17:04.040738 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG dbcounter [-] [80245] Writing DB stats keystone:SELECT=100 {{(pid=80245) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:17:04.054836 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e20d00c0-f2c2-4c43-bbc1-a7983586e907 None None] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:17:04.107956 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24a6961e-4bde-40a0-8747-be7ca2bc14d1 None None] Validating token access rules against request {{(pid=80245) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:17:04.163942 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ae1b5d3-2514-4bdf-af86-37bc198cafd8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80245) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:17:04.164923 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ae1b5d3-2514-4bdf-af86-37bc198cafd8 None admin] REQUEST_METHOD: `GET` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:17:04.165332 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ae1b5d3-2514-4bdf-af86-37bc198cafd8 None admin] SCRIPT_NAME: `/identity` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:17:04.165714 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ae1b5d3-2514-4bdf-af86-37bc198cafd8 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:17:04.178126 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ae1b5d3-2514-4bdf-af86-37bc198cafd8 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 21 08:17:04.186618 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application [None req-1ae1b5d3-2514-4bdf-af86-37bc198cafd8 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 21 08:17:04.186618 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 08:17:04.186618 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 08:17:04.186618 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 08:17:04.186618 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 08:17:04.186618 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 08:17:04.186618 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 08:17:04.186618 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 08:17:04.186618 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 08:17:04.186618 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 08:17:04.186618 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 08:17:04.186618 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 08:17:04.186618 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 21 08:17:04.186618 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 21 08:17:04.186618 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 21 08:17:04.186618 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 21 08:17:04.187572 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 21 08:17:04.187572 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 21 08:17:04.187572 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 21 08:17:04.187572 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 21 08:17:04.187572 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 08:17:04.187572 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 08:17:04.187572 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 08:17:04.187572 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 08:17:04.187572 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 08:17:04.187572 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 08:17:04.187572 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 08:17:04.187572 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application with Lock( Jul 21 08:17:04.187572 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 08:17:04.187572 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application return self._enter() Jul 21 08:17:04.187572 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 08:17:04.187572 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 08:17:04.188805 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 08:17:04.188805 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application return self.creator() Jul 21 08:17:04.188805 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 08:17:04.188805 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application created_value = creator( Jul 21 08:17:04.188805 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 21 08:17:04.188805 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 21 08:17:04.188805 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 21 08:17:04.188805 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 21 08:17:04.188805 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 21 08:17:04.188805 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 21 08:17:04.188805 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 21 08:17:04.188805 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application Jul 21 08:17:04.188805 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: [pid: 80245|app: 0|req: 242/485] 10.222.0.47 () {56 vars in 1102 bytes} [Thu Jul 21 08:17:03 2022] GET /identity/v3/roles/service => generated 85 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 21 08:17:04.194967 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ae1b5d3-2514-4bdf-af86-37bc198cafd8 None admin] Authenticating user token {{(pid=80245) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:17:04.294115 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e20d00c0-f2c2-4c43-bbc1-a7983586e907 None None] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:17:04.351259 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d23335ca-cead-4d90-ba1d-52483359e747 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:17:04.352022 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d23335ca-cead-4d90-ba1d-52483359e747 None admin] REQUEST_METHOD: `GET` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:17:04.352304 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d23335ca-cead-4d90-ba1d-52483359e747 None admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:17:04.352576 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d23335ca-cead-4d90-ba1d-52483359e747 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:17:04.367778 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG dbcounter [-] [80244] Writing DB stats keystone:SELECT=100 {{(pid=80244) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:17:04.372273 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d23335ca-cead-4d90-ba1d-52483359e747 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 21 08:17:04.383640 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application [None req-d23335ca-cead-4d90-ba1d-52483359e747 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 21 08:17:04.383640 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 08:17:04.383640 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 08:17:04.383640 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 08:17:04.383640 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 08:17:04.383640 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 08:17:04.383640 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 08:17:04.383640 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 08:17:04.383640 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 08:17:04.383640 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 08:17:04.383640 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 08:17:04.383640 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 08:17:04.383640 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 21 08:17:04.383640 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 21 08:17:04.383640 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 21 08:17:04.383640 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 21 08:17:04.384598 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 21 08:17:04.384598 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 21 08:17:04.384598 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 21 08:17:04.384598 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 21 08:17:04.384598 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 08:17:04.384598 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 08:17:04.384598 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 08:17:04.384598 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 08:17:04.384598 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 08:17:04.384598 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 08:17:04.384598 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 08:17:04.384598 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application with Lock( Jul 21 08:17:04.384598 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 08:17:04.384598 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self._enter() Jul 21 08:17:04.384598 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 08:17:04.384598 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 08:17:04.385585 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 08:17:04.385585 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self.creator() Jul 21 08:17:04.385585 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 08:17:04.385585 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application created_value = creator( Jul 21 08:17:04.385585 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 21 08:17:04.385585 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 21 08:17:04.385585 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 21 08:17:04.385585 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 21 08:17:04.385585 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 21 08:17:04.385585 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 21 08:17:04.385585 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 21 08:17:04.385585 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application Jul 21 08:17:04.385585 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 244/486] 10.222.0.47 () {56 vars in 1102 bytes} [Thu Jul 21 08:17:04 2022] GET /identity/v3/roles/service => generated 85 bytes in 331 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 08:17:04.390622 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d23335ca-cead-4d90-ba1d-52483359e747 None admin] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:17:04.398130 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG dbcounter [-] [80245] Writing DB stats keystone:SELECT=100 {{(pid=80245) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:17:04.406114 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ae1b5d3-2514-4bdf-af86-37bc198cafd8 None admin] Validating token access rules against request {{(pid=80245) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:17:04.462846 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec9e6b9b-e51c-43e5-aa2c-ac0a1e3ce26a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80245) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:17:04.463992 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec9e6b9b-e51c-43e5-aa2c-ac0a1e3ce26a None admin] REQUEST_METHOD: `GET` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:17:04.464475 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec9e6b9b-e51c-43e5-aa2c-ac0a1e3ce26a None admin] SCRIPT_NAME: `/identity` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:17:04.465041 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec9e6b9b-e51c-43e5-aa2c-ac0a1e3ce26a None admin] PATH_INFO: `/v3/roles` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:17:04.465970 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec9e6b9b-e51c-43e5-aa2c-ac0a1e3ce26a None admin] RBAC: Adding query filter params (name=service) {{(pid=80245) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 08:17:04.466470 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec9e6b9b-e51c-43e5-aa2c-ac0a1e3ce26a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:17:04.468346 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec9e6b9b-e51c-43e5-aa2c-ac0a1e3ce26a None admin] RBAC: Authorization granted {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:17:04.487041 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: [pid: 80245|app: 0|req: 243/487] 10.222.0.47 () {56 vars in 1103 bytes} [Thu Jul 21 08:17:04 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 292 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 08:17:04.493508 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec9e6b9b-e51c-43e5-aa2c-ac0a1e3ce26a None admin] Authenticating user token {{(pid=80245) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:17:04.541104 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d23335ca-cead-4d90-ba1d-52483359e747 None admin] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:17:04.601156 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b21e7ffa-a60c-4385-91fe-e06dc0787634 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:17:04.602197 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b21e7ffa-a60c-4385-91fe-e06dc0787634 None admin] REQUEST_METHOD: `GET` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:17:04.604139 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b21e7ffa-a60c-4385-91fe-e06dc0787634 None admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:17:04.604597 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b21e7ffa-a60c-4385-91fe-e06dc0787634 None admin] PATH_INFO: `/v3/roles` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:17:04.605533 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b21e7ffa-a60c-4385-91fe-e06dc0787634 None admin] RBAC: Adding query filter params (name=service) {{(pid=80244) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 08:17:04.607899 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b21e7ffa-a60c-4385-91fe-e06dc0787634 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:17:04.609523 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b21e7ffa-a60c-4385-91fe-e06dc0787634 None admin] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:17:04.627358 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 245/488] 10.222.0.47 () {56 vars in 1103 bytes} [Thu Jul 21 08:17:04 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 237 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 21 08:17:04.632726 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b21e7ffa-a60c-4385-91fe-e06dc0787634 None admin] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:17:04.676457 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec9e6b9b-e51c-43e5-aa2c-ac0a1e3ce26a None admin] Validating token access rules against request {{(pid=80245) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:17:04.717936 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd8570fe-5df6-4f0a-8ef5-1c2fff49e117 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80245) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:17:04.718743 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd8570fe-5df6-4f0a-8ef5-1c2fff49e117 None admin] REQUEST_METHOD: `GET` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:17:04.719027 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd8570fe-5df6-4f0a-8ef5-1c2fff49e117 None admin] SCRIPT_NAME: `/identity` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:17:04.719307 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd8570fe-5df6-4f0a-8ef5-1c2fff49e117 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:17:04.724201 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG dbcounter [-] [80244] Writing DB stats keystone:SELECT=100 {{(pid=80244) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:17:04.731798 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd8570fe-5df6-4f0a-8ef5-1c2fff49e117 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:17:04.733254 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd8570fe-5df6-4f0a-8ef5-1c2fff49e117 None admin] RBAC: Authorization granted {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:17:04.748768 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b21e7ffa-a60c-4385-91fe-e06dc0787634 None admin] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:17:04.751239 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application [None req-dd8570fe-5df6-4f0a-8ef5-1c2fff49e117 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 08:17:04.751239 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 08:17:04.751239 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 08:17:04.751239 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 08:17:04.751239 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 08:17:04.751239 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 08:17:04.751239 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 08:17:04.751239 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 08:17:04.751239 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 08:17:04.751239 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 08:17:04.751239 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 08:17:04.751239 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 08:17:04.751239 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 21 08:17:04.751239 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 21 08:17:04.751239 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 21 08:17:04.752074 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 21 08:17:04.752074 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 08:17:04.752074 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 08:17:04.752074 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 08:17:04.752074 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 08:17:04.752074 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 08:17:04.752074 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 08:17:04.752074 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 08:17:04.752074 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application with Lock( Jul 21 08:17:04.752074 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 08:17:04.752074 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application return self._enter() Jul 21 08:17:04.752074 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 08:17:04.752074 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 08:17:04.752074 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 08:17:04.752074 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application return self.creator() Jul 21 08:17:04.752074 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 08:17:04.752074 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application created_value = creator( Jul 21 08:17:04.752896 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 21 08:17:04.752896 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 21 08:17:04.752896 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 08:17:04.752896 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application Jul 21 08:17:04.753109 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: [pid: 80245|app: 0|req: 244/489] 10.222.0.47 () {56 vars in 1108 bytes} [Thu Jul 21 08:17:04 2022] GET /identity/v3/domains/Default => generated 87 bytes in 260 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 21 08:17:04.754274 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e94172e-0c7a-480c-82f9-9d4c263d9455 None None] REQUEST_METHOD: `GET` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:17:04.754559 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e94172e-0c7a-480c-82f9-9d4c263d9455 None None] SCRIPT_NAME: `/identity` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:17:04.759348 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e94172e-0c7a-480c-82f9-9d4c263d9455 None None] PATH_INFO: `/` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:17:04.761184 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: [pid: 80245|app: 0|req: 245/490] 10.222.0.47 () {54 vars in 926 bytes} [Thu Jul 21 08:17:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 08:17:04.762130 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e94172e-0c7a-480c-82f9-9d4c263d9455 None None] Authenticating user token {{(pid=80245) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:17:04.782800 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG dbcounter [-] [80245] Writing DB stats keystone:SELECT=100 {{(pid=80245) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:17:04.789396 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16eb2deb-e444-4df5-807f-ba57b4e55d3f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:17:04.790187 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16eb2deb-e444-4df5-807f-ba57b4e55d3f None admin] REQUEST_METHOD: `GET` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:17:04.790464 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16eb2deb-e444-4df5-807f-ba57b4e55d3f None admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:17:04.790733 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16eb2deb-e444-4df5-807f-ba57b4e55d3f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:17:04.802327 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16eb2deb-e444-4df5-807f-ba57b4e55d3f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:17:04.803893 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16eb2deb-e444-4df5-807f-ba57b4e55d3f None admin] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:17:04.822192 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application [None req-16eb2deb-e444-4df5-807f-ba57b4e55d3f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 08:17:04.822192 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 08:17:04.822192 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 08:17:04.822192 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 08:17:04.822192 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 08:17:04.822192 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 08:17:04.822192 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 08:17:04.822192 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 08:17:04.822192 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 08:17:04.822192 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 08:17:04.822192 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 08:17:04.822192 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 08:17:04.822192 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 21 08:17:04.822192 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 21 08:17:04.822192 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 21 08:17:04.824213 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 21 08:17:04.824213 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 08:17:04.824213 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 08:17:04.824213 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 08:17:04.824213 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 08:17:04.824213 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 08:17:04.824213 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 08:17:04.824213 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 08:17:04.824213 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application with Lock( Jul 21 08:17:04.824213 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 08:17:04.824213 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self._enter() Jul 21 08:17:04.824213 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 08:17:04.824213 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 08:17:04.824213 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 08:17:04.824213 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self.creator() Jul 21 08:17:04.824213 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 08:17:04.824213 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application created_value = creator( Jul 21 08:17:04.826255 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 21 08:17:04.826255 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 21 08:17:04.826255 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 08:17:04.826255 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application Jul 21 08:17:04.831359 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 246/491] 10.222.0.47 () {56 vars in 1108 bytes} [Thu Jul 21 08:17:04 2022] GET /identity/v3/domains/Default => generated 87 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 08:17:04.831359 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2857235d-f966-4b8a-805f-5b4029922ec3 None None] REQUEST_METHOD: `POST` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:17:04.831359 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2857235d-f966-4b8a-805f-5b4029922ec3 None None] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:17:04.831359 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2857235d-f966-4b8a-805f-5b4029922ec3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:17:04.851647 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.auth.core [None req-2857235d-f966-4b8a-805f-5b4029922ec3 None None] MFA Rules not processed for user `2dce40b8d74c41ec94cc36e6ff05d412`. Rule list: `[]` (Enabled: `True`). {{(pid=80244) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 08:17:04.853168 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.fernet_utils [None req-2857235d-f966-4b8a-805f-5b4029922ec3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80244) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 08:17:04.882722 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e94172e-0c7a-480c-82f9-9d4c263d9455 None None] Validating token access rules against request {{(pid=80245) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:17:04.921480 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb29a0e2-fa3d-4d4d-a094-52a9c9ee9539 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80245) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:17:04.922178 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb29a0e2-fa3d-4d4d-a094-52a9c9ee9539 None admin] REQUEST_METHOD: `GET` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:17:04.922538 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb29a0e2-fa3d-4d4d-a094-52a9c9ee9539 None admin] SCRIPT_NAME: `/identity` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:17:04.922824 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb29a0e2-fa3d-4d4d-a094-52a9c9ee9539 None admin] PATH_INFO: `/v3/domains` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:17:04.924755 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb29a0e2-fa3d-4d4d-a094-52a9c9ee9539 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80245) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 08:17:04.924755 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb29a0e2-fa3d-4d4d-a094-52a9c9ee9539 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:17:04.925436 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb29a0e2-fa3d-4d4d-a094-52a9c9ee9539 None admin] RBAC: Authorization granted {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:17:04.930657 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 247/492] 10.222.0.47 () {58 vars in 1023 bytes} [Thu Jul 21 08:17:04 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 21 08:17:04.931536 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2857235d-f966-4b8a-805f-5b4029922ec3 None None] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:17:04.949501 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: [pid: 80245|app: 0|req: 246/493] 10.222.0.47 () {56 vars in 1109 bytes} [Thu Jul 21 08:17:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 08:17:04.955912 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb29a0e2-fa3d-4d4d-a094-52a9c9ee9539 None admin] Authenticating user token {{(pid=80245) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:17:04.985209 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG dbcounter [-] [80244] Writing DB stats keystone:SELECT=100 {{(pid=80244) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:17:05.042485 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2857235d-f966-4b8a-805f-5b4029922ec3 None None] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:17:05.046598 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG dbcounter [-] [80245] Writing DB stats keystone:SELECT=100 {{(pid=80245) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:17:05.079917 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb29a0e2-fa3d-4d4d-a094-52a9c9ee9539 None admin] Validating token access rules against request {{(pid=80245) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:17:05.080501 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-295e2ebc-ad83-49d0-8216-30a713eb45a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:17:05.081207 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-295e2ebc-ad83-49d0-8216-30a713eb45a2 None admin] REQUEST_METHOD: `GET` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:17:05.081645 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-295e2ebc-ad83-49d0-8216-30a713eb45a2 None admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:17:05.081925 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-295e2ebc-ad83-49d0-8216-30a713eb45a2 None admin] PATH_INFO: `/v3/domains` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:17:05.083499 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-295e2ebc-ad83-49d0-8216-30a713eb45a2 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80244) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 08:17:05.083807 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-295e2ebc-ad83-49d0-8216-30a713eb45a2 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:17:05.085201 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-295e2ebc-ad83-49d0-8216-30a713eb45a2 None admin] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:17:05.099618 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 248/494] 10.222.0.47 () {56 vars in 1109 bytes} [Thu Jul 21 08:17:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 08:17:05.100975 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-295e2ebc-ad83-49d0-8216-30a713eb45a2 None admin] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:17:05.120884 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b56985a-46bc-4f8b-a7cf-6501fe34498f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80245) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:17:05.121609 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b56985a-46bc-4f8b-a7cf-6501fe34498f None admin] REQUEST_METHOD: `GET` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:17:05.121897 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b56985a-46bc-4f8b-a7cf-6501fe34498f None admin] SCRIPT_NAME: `/identity` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:17:05.122177 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b56985a-46bc-4f8b-a7cf-6501fe34498f None admin] PATH_INFO: `/v3/users/nova` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:17:05.130343 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b56985a-46bc-4f8b-a7cf-6501fe34498f None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:17:05.131776 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b56985a-46bc-4f8b-a7cf-6501fe34498f None admin] RBAC: Authorization granted {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:17:05.149556 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application [None req-6b56985a-46bc-4f8b-a7cf-6501fe34498f None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jul 21 08:17:05.149556 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 08:17:05.149556 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 08:17:05.149556 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 08:17:05.149556 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 08:17:05.149556 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 08:17:05.149556 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 08:17:05.149556 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 08:17:05.149556 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 08:17:05.149556 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 08:17:05.149556 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 08:17:05.149556 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 08:17:05.149556 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 21 08:17:05.149556 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 21 08:17:05.149556 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 21 08:17:05.149556 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 21 08:17:05.150503 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 08:17:05.150503 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 08:17:05.150503 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 21 08:17:05.150503 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 21 08:17:05.150503 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 21 08:17:05.150503 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 21 08:17:05.150503 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 08:17:05.150503 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 08:17:05.150503 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 08:17:05.150503 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 08:17:05.150503 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 08:17:05.150503 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application with Lock( Jul 21 08:17:05.150503 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 08:17:05.150503 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application return self._enter() Jul 21 08:17:05.150503 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 08:17:05.150503 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 08:17:05.150503 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 08:17:05.150503 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application return self.creator() Jul 21 08:17:05.156516 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 08:17:05.156516 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application created_value = creator( Jul 21 08:17:05.156516 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 21 08:17:05.156516 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 21 08:17:05.156516 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 21 08:17:05.156516 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 21 08:17:05.156516 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 21 08:17:05.156516 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 21 08:17:05.156516 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jul 21 08:17:05.156516 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application Jul 21 08:17:05.156516 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: [pid: 80245|app: 0|req: 247/495] 10.222.0.47 () {56 vars in 1093 bytes} [Thu Jul 21 08:17:04 2022] GET /identity/v3/users/nova => generated 82 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 21 08:17:05.156516 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db4fd693-312d-4d3d-9ff3-c3b929eae3fe None None] REQUEST_METHOD: `GET` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:17:05.156516 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db4fd693-312d-4d3d-9ff3-c3b929eae3fe None None] SCRIPT_NAME: `/identity` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:17:05.157167 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db4fd693-312d-4d3d-9ff3-c3b929eae3fe None None] PATH_INFO: `/` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:17:05.157167 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: [pid: 80245|app: 0|req: 248/496] 10.222.0.47 () {54 vars in 926 bytes} [Thu Jul 21 08:17:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 08:17:05.157167 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db4fd693-312d-4d3d-9ff3-c3b929eae3fe None None] Authenticating user token {{(pid=80245) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:17:05.219249 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-295e2ebc-ad83-49d0-8216-30a713eb45a2 None admin] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:17:05.221606 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG dbcounter [-] [80244] Writing DB stats keystone:SELECT=100 {{(pid=80244) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:17:05.257119 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df9ab8b8-8cbf-4a8b-8a29-7e5ff8304c1e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:17:05.257911 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df9ab8b8-8cbf-4a8b-8a29-7e5ff8304c1e None admin] REQUEST_METHOD: `GET` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:17:05.258775 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df9ab8b8-8cbf-4a8b-8a29-7e5ff8304c1e None admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:17:05.259057 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df9ab8b8-8cbf-4a8b-8a29-7e5ff8304c1e None admin] PATH_INFO: `/v3/roles/service` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:17:05.261663 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db4fd693-312d-4d3d-9ff3-c3b929eae3fe None None] Validating token access rules against request {{(pid=80245) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:17:05.269485 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df9ab8b8-8cbf-4a8b-8a29-7e5ff8304c1e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 21 08:17:05.277203 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application [None req-df9ab8b8-8cbf-4a8b-8a29-7e5ff8304c1e None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 21 08:17:05.277203 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 08:17:05.277203 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 08:17:05.277203 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 08:17:05.277203 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 08:17:05.277203 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 08:17:05.277203 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 08:17:05.277203 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 08:17:05.277203 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 08:17:05.277203 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 08:17:05.277203 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 08:17:05.277203 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 08:17:05.277203 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 21 08:17:05.277203 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 21 08:17:05.277203 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 21 08:17:05.277203 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 21 08:17:05.278212 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 21 08:17:05.278212 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 21 08:17:05.278212 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 21 08:17:05.278212 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 21 08:17:05.278212 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 08:17:05.278212 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 08:17:05.278212 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 08:17:05.278212 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 08:17:05.278212 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 08:17:05.278212 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 08:17:05.278212 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 08:17:05.278212 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application with Lock( Jul 21 08:17:05.278212 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 08:17:05.278212 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self._enter() Jul 21 08:17:05.278212 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 08:17:05.278212 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 08:17:05.279046 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 08:17:05.279046 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self.creator() Jul 21 08:17:05.279046 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 08:17:05.279046 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application created_value = creator( Jul 21 08:17:05.279046 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 21 08:17:05.279046 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 21 08:17:05.279046 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 21 08:17:05.279046 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 21 08:17:05.279046 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 21 08:17:05.279046 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 21 08:17:05.279046 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 21 08:17:05.279046 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application Jul 21 08:17:05.279046 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 249/497] 10.222.0.47 () {56 vars in 1102 bytes} [Thu Jul 21 08:17:05 2022] GET /identity/v3/roles/service => generated 85 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 08:17:05.279739 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df9ab8b8-8cbf-4a8b-8a29-7e5ff8304c1e None admin] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:17:05.293275 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG dbcounter [-] [80245] Writing DB stats keystone:SELECT=100 {{(pid=80245) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:17:05.298805 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-582eca8f-7c43-45cf-bcc8-219eebcfba96 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80245) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:17:05.299656 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-582eca8f-7c43-45cf-bcc8-219eebcfba96 None admin] REQUEST_METHOD: `GET` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:17:05.299940 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-582eca8f-7c43-45cf-bcc8-219eebcfba96 None admin] SCRIPT_NAME: `/identity` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:17:05.300218 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-582eca8f-7c43-45cf-bcc8-219eebcfba96 None admin] PATH_INFO: `/v3/users/glance` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:17:05.307972 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-582eca8f-7c43-45cf-bcc8-219eebcfba96 None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:17:05.309427 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-582eca8f-7c43-45cf-bcc8-219eebcfba96 None admin] RBAC: Authorization granted {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:17:05.323754 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application [None req-582eca8f-7c43-45cf-bcc8-219eebcfba96 None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Jul 21 08:17:05.323754 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 08:17:05.323754 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 08:17:05.323754 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 08:17:05.323754 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 08:17:05.323754 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 08:17:05.323754 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 08:17:05.323754 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 08:17:05.323754 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 08:17:05.323754 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 08:17:05.323754 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 08:17:05.323754 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 08:17:05.323754 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 21 08:17:05.323754 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 21 08:17:05.323754 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 21 08:17:05.323754 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 21 08:17:05.324655 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 08:17:05.324655 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 08:17:05.324655 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 21 08:17:05.324655 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 21 08:17:05.324655 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 21 08:17:05.324655 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 21 08:17:05.324655 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 08:17:05.324655 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 08:17:05.324655 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 08:17:05.324655 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 08:17:05.324655 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 08:17:05.324655 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application with Lock( Jul 21 08:17:05.324655 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 08:17:05.324655 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application return self._enter() Jul 21 08:17:05.324655 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 08:17:05.324655 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 08:17:05.324655 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 08:17:05.324655 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application return self.creator() Jul 21 08:17:05.325515 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 08:17:05.325515 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application created_value = creator( Jul 21 08:17:05.325515 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 21 08:17:05.325515 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 21 08:17:05.325515 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 21 08:17:05.325515 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 21 08:17:05.325515 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 21 08:17:05.325515 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 21 08:17:05.325515 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Jul 21 08:17:05.325515 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application Jul 21 08:17:05.325515 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: [pid: 80245|app: 0|req: 249/498] 10.222.0.47 () {56 vars in 1099 bytes} [Thu Jul 21 08:17:05 2022] GET /identity/v3/users/glance => generated 84 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 08:17:05.326117 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e758626-edd1-4ed7-afbb-c1890e7da1ac None None] REQUEST_METHOD: `POST` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:17:05.326400 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e758626-edd1-4ed7-afbb-c1890e7da1ac None None] SCRIPT_NAME: `/identity` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:17:05.326680 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e758626-edd1-4ed7-afbb-c1890e7da1ac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:17:05.342843 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.auth.core [None req-1e758626-edd1-4ed7-afbb-c1890e7da1ac None None] MFA Rules not processed for user `2dce40b8d74c41ec94cc36e6ff05d412`. Rule list: `[]` (Enabled: `True`). {{(pid=80245) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 08:17:05.344083 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.fernet_utils [None req-1e758626-edd1-4ed7-afbb-c1890e7da1ac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80245) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 08:17:05.388115 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df9ab8b8-8cbf-4a8b-8a29-7e5ff8304c1e None admin] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:17:05.410557 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: [pid: 80245|app: 0|req: 250/499] 10.222.0.47 () {58 vars in 1023 bytes} [Thu Jul 21 08:17:05 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 21 08:17:05.412246 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f50cc3a-3971-4d89-b2e0-954646f10e27 None None] REQUEST_METHOD: `GET` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:17:05.412536 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f50cc3a-3971-4d89-b2e0-954646f10e27 None None] SCRIPT_NAME: `/identity` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:17:05.412811 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f50cc3a-3971-4d89-b2e0-954646f10e27 None None] PATH_INFO: `/` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:17:05.414080 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: [pid: 80245|app: 0|req: 251/500] 10.222.0.47 () {54 vars in 926 bytes} [Thu Jul 21 08:17:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 08:17:05.414769 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f50cc3a-3971-4d89-b2e0-954646f10e27 None None] Authenticating user token {{(pid=80245) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:17:05.435455 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b440c080-a0c2-42b0-bc36-4bd8a08dd94d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:17:05.436135 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b440c080-a0c2-42b0-bc36-4bd8a08dd94d None admin] REQUEST_METHOD: `GET` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:17:05.436420 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b440c080-a0c2-42b0-bc36-4bd8a08dd94d None admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:17:05.436696 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b440c080-a0c2-42b0-bc36-4bd8a08dd94d None admin] PATH_INFO: `/v3/users` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:17:05.437549 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b440c080-a0c2-42b0-bc36-4bd8a08dd94d None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80244) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 08:17:05.437861 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b440c080-a0c2-42b0-bc36-4bd8a08dd94d None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:17:05.439298 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b440c080-a0c2-42b0-bc36-4bd8a08dd94d None admin] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:17:05.476087 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 250/501] 10.222.0.47 () {56 vars in 1133 bytes} [Thu Jul 21 08:17:05 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 08:17:05.476839 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b440c080-a0c2-42b0-bc36-4bd8a08dd94d None admin] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:17:05.502354 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG dbcounter [-] [80244] Writing DB stats keystone:SELECT=100 {{(pid=80244) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:17:05.528201 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f50cc3a-3971-4d89-b2e0-954646f10e27 None None] Validating token access rules against request {{(pid=80245) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:17:05.554059 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG dbcounter [-] [80245] Writing DB stats keystone:SELECT=100 {{(pid=80245) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:17:05.566586 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96b5a812-2c17-4846-8100-e0914c798554 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80245) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:17:05.567281 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96b5a812-2c17-4846-8100-e0914c798554 None admin] REQUEST_METHOD: `GET` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:17:05.567589 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96b5a812-2c17-4846-8100-e0914c798554 None admin] SCRIPT_NAME: `/identity` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:17:05.567859 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96b5a812-2c17-4846-8100-e0914c798554 None admin] PATH_INFO: `/v3/roles` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:17:05.568591 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96b5a812-2c17-4846-8100-e0914c798554 None admin] RBAC: Adding query filter params (name=service) {{(pid=80245) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 08:17:05.568906 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96b5a812-2c17-4846-8100-e0914c798554 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:17:05.570257 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96b5a812-2c17-4846-8100-e0914c798554 None admin] RBAC: Authorization granted {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:17:05.578150 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: [pid: 80245|app: 0|req: 252/502] 10.222.0.47 () {56 vars in 1103 bytes} [Thu Jul 21 08:17:05 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 08:17:05.578544 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b440c080-a0c2-42b0-bc36-4bd8a08dd94d None admin] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:17:05.579368 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-524749cf-84d6-4982-a4d1-34458cca04a1 None None] REQUEST_METHOD: `POST` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:17:05.579656 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-524749cf-84d6-4982-a4d1-34458cca04a1 None None] SCRIPT_NAME: `/identity` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:17:05.579938 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-524749cf-84d6-4982-a4d1-34458cca04a1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:17:05.599536 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.auth.core [None req-524749cf-84d6-4982-a4d1-34458cca04a1 None None] MFA Rules not processed for user `2dce40b8d74c41ec94cc36e6ff05d412`. Rule list: `[]` (Enabled: `True`). {{(pid=80245) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 08:17:05.600748 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.fernet_utils [None req-524749cf-84d6-4982-a4d1-34458cca04a1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80245) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 08:17:05.620744 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84965e0c-a471-4bdb-832f-ed4de055feca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:17:05.621415 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84965e0c-a471-4bdb-832f-ed4de055feca None admin] REQUEST_METHOD: `GET` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:17:05.621694 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84965e0c-a471-4bdb-832f-ed4de055feca None admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:17:05.621966 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84965e0c-a471-4bdb-832f-ed4de055feca None admin] PATH_INFO: `/v3/users` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:17:05.622786 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84965e0c-a471-4bdb-832f-ed4de055feca None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=80244) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 08:17:05.623117 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84965e0c-a471-4bdb-832f-ed4de055feca None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:17:05.624583 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84965e0c-a471-4bdb-832f-ed4de055feca None admin] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:17:05.642016 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 251/503] 10.222.0.47 () {56 vars in 1137 bytes} [Thu Jul 21 08:17:05 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 08:17:05.642825 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84965e0c-a471-4bdb-832f-ed4de055feca None admin] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:17:05.673235 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: [pid: 80245|app: 0|req: 253/504] 10.222.0.47 () {58 vars in 1023 bytes} [Thu Jul 21 08:17:05 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 21 08:17:05.674085 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-524749cf-84d6-4982-a4d1-34458cca04a1 None None] Authenticating user token {{(pid=80245) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:17:05.729925 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG dbcounter [-] [80244] Writing DB stats keystone:SELECT=100 {{(pid=80244) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:17:05.748720 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84965e0c-a471-4bdb-832f-ed4de055feca None admin] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:17:05.774382 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-524749cf-84d6-4982-a4d1-34458cca04a1 None None] Validating token access rules against request {{(pid=80245) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:17:05.782493 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f06323f-e17d-4f0a-8b60-a8e1ee05879f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:17:05.783289 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f06323f-e17d-4f0a-8b60-a8e1ee05879f None admin] REQUEST_METHOD: `GET` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:17:05.783600 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f06323f-e17d-4f0a-8b60-a8e1ee05879f None admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:17:05.783870 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f06323f-e17d-4f0a-8b60-a8e1ee05879f None admin] PATH_INFO: `/v3/roles/service` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:17:05.788903 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG dbcounter [-] [80245] Writing DB stats keystone:SELECT=100 {{(pid=80245) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:17:05.795750 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f06323f-e17d-4f0a-8b60-a8e1ee05879f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 21 08:17:05.803543 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application [None req-0f06323f-e17d-4f0a-8b60-a8e1ee05879f None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 21 08:17:05.803543 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 08:17:05.803543 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 08:17:05.803543 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 08:17:05.803543 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 08:17:05.803543 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 08:17:05.803543 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 08:17:05.803543 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 08:17:05.803543 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 08:17:05.803543 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 08:17:05.803543 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 08:17:05.803543 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 08:17:05.803543 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 21 08:17:05.803543 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 21 08:17:05.803543 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 21 08:17:05.803543 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 21 08:17:05.804973 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 21 08:17:05.804973 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 21 08:17:05.804973 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 21 08:17:05.804973 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 21 08:17:05.804973 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 08:17:05.804973 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 08:17:05.804973 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 08:17:05.804973 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 08:17:05.804973 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 08:17:05.804973 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 08:17:05.804973 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 08:17:05.804973 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application with Lock( Jul 21 08:17:05.804973 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 08:17:05.804973 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self._enter() Jul 21 08:17:05.804973 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 08:17:05.804973 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 08:17:05.805753 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 08:17:05.805753 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self.creator() Jul 21 08:17:05.805753 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 08:17:05.805753 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application created_value = creator( Jul 21 08:17:05.805753 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 21 08:17:05.805753 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 21 08:17:05.805753 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 21 08:17:05.805753 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 21 08:17:05.805753 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 21 08:17:05.805753 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 21 08:17:05.805753 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 21 08:17:05.805753 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application Jul 21 08:17:05.805753 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 252/505] 10.222.0.47 () {56 vars in 1102 bytes} [Thu Jul 21 08:17:05 2022] GET /identity/v3/roles/service => generated 85 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 08:17:05.805753 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f06323f-e17d-4f0a-8b60-a8e1ee05879f None admin] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:17:05.813058 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-141a306c-82d6-4dd7-8e99-999308cf20c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80245) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:17:05.813724 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-141a306c-82d6-4dd7-8e99-999308cf20c1 None admin] REQUEST_METHOD: `GET` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:17:05.814007 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-141a306c-82d6-4dd7-8e99-999308cf20c1 None admin] SCRIPT_NAME: `/identity` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:17:05.814286 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-141a306c-82d6-4dd7-8e99-999308cf20c1 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:17:05.885822 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-141a306c-82d6-4dd7-8e99-999308cf20c1 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:17:05.887348 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-141a306c-82d6-4dd7-8e99-999308cf20c1 None admin] RBAC: Authorization granted {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:17:05.912659 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f06323f-e17d-4f0a-8b60-a8e1ee05879f None admin] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:17:05.949006 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87cc0535-1729-4f9e-ad1c-2d889b69187c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:17:05.949792 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87cc0535-1729-4f9e-ad1c-2d889b69187c None admin] REQUEST_METHOD: `GET` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:17:05.950074 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87cc0535-1729-4f9e-ad1c-2d889b69187c None admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:17:05.950353 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87cc0535-1729-4f9e-ad1c-2d889b69187c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:17:05.959483 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG dbcounter [-] [80244] Writing DB stats keystone:SELECT=100 {{(pid=80244) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:17:05.961911 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87cc0535-1729-4f9e-ad1c-2d889b69187c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:17:05.963478 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87cc0535-1729-4f9e-ad1c-2d889b69187c None admin] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:17:05.981724 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application [None req-87cc0535-1729-4f9e-ad1c-2d889b69187c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 08:17:05.981724 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 08:17:05.981724 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 08:17:05.981724 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 08:17:05.981724 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 08:17:05.981724 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 08:17:05.981724 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 08:17:05.981724 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 08:17:05.981724 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 08:17:05.981724 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 08:17:05.981724 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 08:17:05.981724 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 08:17:05.981724 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 21 08:17:05.981724 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 21 08:17:05.981724 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 21 08:17:05.982708 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 21 08:17:05.982708 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 08:17:05.982708 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 08:17:05.982708 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 08:17:05.982708 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 08:17:05.982708 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 08:17:05.982708 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 08:17:05.982708 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 08:17:05.982708 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application with Lock( Jul 21 08:17:05.982708 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 08:17:05.982708 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self._enter() Jul 21 08:17:05.982708 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 08:17:05.982708 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 08:17:05.982708 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 08:17:05.982708 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self.creator() Jul 21 08:17:05.982708 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 08:17:05.982708 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application created_value = creator( Jul 21 08:17:05.983841 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 21 08:17:05.983841 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 21 08:17:05.983841 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 08:17:05.983841 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application Jul 21 08:17:05.983841 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 253/506] 10.222.0.47 () {56 vars in 1108 bytes} [Thu Jul 21 08:17:05 2022] GET /identity/v3/domains/Default => generated 87 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 08:17:05.983841 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87cc0535-1729-4f9e-ad1c-2d889b69187c None admin] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:17:05.994529 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: [pid: 80245|app: 0|req: 254/507] 10.222.0.47 () {58 vars in 1282 bytes} [Thu Jul 21 08:17:05 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 321 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 21 08:17:05.995399 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-141a306c-82d6-4dd7-8e99-999308cf20c1 None admin] Authenticating user token {{(pid=80245) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:17:06.010918 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG dbcounter [-] [80245] Writing DB stats keystone:SELECT=100 {{(pid=80245) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:17:06.084062 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87cc0535-1729-4f9e-ad1c-2d889b69187c None admin] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:17:06.093966 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-141a306c-82d6-4dd7-8e99-999308cf20c1 None admin] Validating token access rules against request {{(pid=80245) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:17:06.117059 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bee6559-e95e-44d0-84ce-66a3e7fe477b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:17:06.117661 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bee6559-e95e-44d0-84ce-66a3e7fe477b None admin] REQUEST_METHOD: `GET` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:17:06.117942 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bee6559-e95e-44d0-84ce-66a3e7fe477b None admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:17:06.118220 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bee6559-e95e-44d0-84ce-66a3e7fe477b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:17:06.127659 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f30592f4-879a-4c1b-9629-ca2c87b4f2fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80245) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:17:06.128462 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f30592f4-879a-4c1b-9629-ca2c87b4f2fb None admin] REQUEST_METHOD: `GET` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:17:06.128744 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f30592f4-879a-4c1b-9629-ca2c87b4f2fb None admin] SCRIPT_NAME: `/identity` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:17:06.129026 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f30592f4-879a-4c1b-9629-ca2c87b4f2fb None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:17:06.140968 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f30592f4-879a-4c1b-9629-ca2c87b4f2fb None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:17:06.142505 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f30592f4-879a-4c1b-9629-ca2c87b4f2fb None admin] RBAC: Authorization granted {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:17:06.160645 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application [None req-f30592f4-879a-4c1b-9629-ca2c87b4f2fb None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 08:17:06.160645 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 08:17:06.160645 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 08:17:06.160645 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 08:17:06.160645 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 08:17:06.160645 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 08:17:06.160645 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 08:17:06.160645 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 08:17:06.160645 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 08:17:06.160645 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 08:17:06.160645 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 08:17:06.160645 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 08:17:06.160645 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 21 08:17:06.160645 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 21 08:17:06.160645 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 21 08:17:06.161516 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 21 08:17:06.161516 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 08:17:06.161516 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 08:17:06.161516 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 08:17:06.161516 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 08:17:06.161516 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 08:17:06.161516 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 08:17:06.161516 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 08:17:06.161516 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application with Lock( Jul 21 08:17:06.161516 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 08:17:06.161516 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application return self._enter() Jul 21 08:17:06.161516 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 08:17:06.161516 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 08:17:06.161516 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 08:17:06.161516 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application return self.creator() Jul 21 08:17:06.161516 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 08:17:06.161516 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application created_value = creator( Jul 21 08:17:06.162331 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 21 08:17:06.162331 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 21 08:17:06.162331 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 08:17:06.162331 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application Jul 21 08:17:06.162331 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: [pid: 80245|app: 0|req: 255/508] 10.222.0.47 () {56 vars in 1108 bytes} [Thu Jul 21 08:17:05 2022] GET /identity/v3/domains/Default => generated 87 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 08:17:06.162762 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f30592f4-879a-4c1b-9629-ca2c87b4f2fb None admin] Authenticating user token {{(pid=80245) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:17:06.180865 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG dbcounter [-] [80244] Writing DB stats keystone:SELECT=100 {{(pid=80244) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:17:06.191433 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bee6559-e95e-44d0-84ce-66a3e7fe477b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:17:06.193417 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bee6559-e95e-44d0-84ce-66a3e7fe477b None admin] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:17:06.229733 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG dbcounter [-] [80245] Writing DB stats keystone:SELECT=100 {{(pid=80245) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:17:06.264450 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f30592f4-879a-4c1b-9629-ca2c87b4f2fb None admin] Validating token access rules against request {{(pid=80245) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:17:06.291750 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 254/509] 10.222.0.47 () {58 vars in 1282 bytes} [Thu Jul 21 08:17:05 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 308 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 21 08:17:06.292602 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bee6559-e95e-44d0-84ce-66a3e7fe477b None admin] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:17:06.305200 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ee25963-df3c-4a96-a217-5d325314a5d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80245) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:17:06.305200 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ee25963-df3c-4a96-a217-5d325314a5d4 None admin] REQUEST_METHOD: `GET` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:17:06.305200 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ee25963-df3c-4a96-a217-5d325314a5d4 None admin] SCRIPT_NAME: `/identity` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:17:06.305572 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ee25963-df3c-4a96-a217-5d325314a5d4 None admin] PATH_INFO: `/v3/roles` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:17:06.306155 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ee25963-df3c-4a96-a217-5d325314a5d4 None admin] RBAC: Adding query filter params (name=service) {{(pid=80245) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 08:17:06.306462 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ee25963-df3c-4a96-a217-5d325314a5d4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:17:06.307890 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ee25963-df3c-4a96-a217-5d325314a5d4 None admin] RBAC: Authorization granted {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:17:06.316032 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: [pid: 80245|app: 0|req: 256/510] 10.222.0.47 () {56 vars in 1103 bytes} [Thu Jul 21 08:17:06 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 08:17:06.316837 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ee25963-df3c-4a96-a217-5d325314a5d4 None admin] Authenticating user token {{(pid=80245) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:17:06.385870 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG dbcounter [-] [80244] Writing DB stats keystone:SELECT=100 {{(pid=80244) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:17:06.396176 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bee6559-e95e-44d0-84ce-66a3e7fe477b None admin] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:17:06.420224 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ee25963-df3c-4a96-a217-5d325314a5d4 None admin] Validating token access rules against request {{(pid=80245) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:17:06.431393 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7904faf-a933-462c-9fdf-7358cac120a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:17:06.432061 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7904faf-a933-462c-9fdf-7358cac120a5 None admin] REQUEST_METHOD: `GET` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:17:06.432345 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7904faf-a933-462c-9fdf-7358cac120a5 None admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:17:06.432628 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7904faf-a933-462c-9fdf-7358cac120a5 None admin] PATH_INFO: `/v3/domains` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:17:06.433353 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7904faf-a933-462c-9fdf-7358cac120a5 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80244) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 08:17:06.433660 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7904faf-a933-462c-9fdf-7358cac120a5 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:17:06.435054 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7904faf-a933-462c-9fdf-7358cac120a5 None admin] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:17:06.446646 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 255/511] 10.222.0.47 () {56 vars in 1109 bytes} [Thu Jul 21 08:17:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 08:17:06.447249 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG dbcounter [-] [80245] Writing DB stats keystone:SELECT=100 {{(pid=80245) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:17:06.447509 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7904faf-a933-462c-9fdf-7358cac120a5 None admin] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:17:06.459587 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f6b6927-8615-488d-928d-caa492876588 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80245) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:17:06.460388 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f6b6927-8615-488d-928d-caa492876588 None admin] REQUEST_METHOD: `GET` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:17:06.460672 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f6b6927-8615-488d-928d-caa492876588 None admin] SCRIPT_NAME: `/identity` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:17:06.460949 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f6b6927-8615-488d-928d-caa492876588 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:17:06.471933 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f6b6927-8615-488d-928d-caa492876588 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:17:06.473937 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f6b6927-8615-488d-928d-caa492876588 None admin] RBAC: Authorization granted {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:17:06.491854 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application [None req-5f6b6927-8615-488d-928d-caa492876588 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 08:17:06.491854 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 08:17:06.491854 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 08:17:06.491854 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 08:17:06.491854 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 08:17:06.491854 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 08:17:06.491854 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 08:17:06.491854 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 08:17:06.491854 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 08:17:06.491854 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 08:17:06.491854 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 08:17:06.491854 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 08:17:06.491854 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 21 08:17:06.491854 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 21 08:17:06.491854 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 21 08:17:06.492877 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 21 08:17:06.492877 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 08:17:06.492877 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 08:17:06.492877 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 08:17:06.492877 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 08:17:06.492877 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 08:17:06.492877 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 08:17:06.492877 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 08:17:06.492877 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application with Lock( Jul 21 08:17:06.492877 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 08:17:06.492877 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application return self._enter() Jul 21 08:17:06.492877 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 08:17:06.492877 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 08:17:06.492877 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 08:17:06.492877 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application return self.creator() Jul 21 08:17:06.492877 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 08:17:06.492877 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application created_value = creator( Jul 21 08:17:06.493851 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 21 08:17:06.493851 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 21 08:17:06.493851 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 08:17:06.493851 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application Jul 21 08:17:06.494479 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: [pid: 80245|app: 0|req: 257/512] 10.222.0.47 () {56 vars in 1108 bytes} [Thu Jul 21 08:17:06 2022] GET /identity/v3/domains/Default => generated 87 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 08:17:06.495547 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f6b6927-8615-488d-928d-caa492876588 None admin] Authenticating user token {{(pid=80245) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:17:06.563132 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7904faf-a933-462c-9fdf-7358cac120a5 None admin] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:17:06.597201 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4165ac3c-3fb2-4612-8c27-fb954a494b94 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:17:06.597468 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f6b6927-8615-488d-928d-caa492876588 None admin] Validating token access rules against request {{(pid=80245) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:17:06.597871 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4165ac3c-3fb2-4612-8c27-fb954a494b94 None admin] REQUEST_METHOD: `GET` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:17:06.598155 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4165ac3c-3fb2-4612-8c27-fb954a494b94 None admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:17:06.599007 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4165ac3c-3fb2-4612-8c27-fb954a494b94 None admin] PATH_INFO: `/v3/domains` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:17:06.599759 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4165ac3c-3fb2-4612-8c27-fb954a494b94 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80244) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 08:17:06.600063 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4165ac3c-3fb2-4612-8c27-fb954a494b94 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:17:06.601446 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4165ac3c-3fb2-4612-8c27-fb954a494b94 None admin] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:17:06.613078 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 256/513] 10.222.0.47 () {56 vars in 1109 bytes} [Thu Jul 21 08:17:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 08:17:06.614231 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4165ac3c-3fb2-4612-8c27-fb954a494b94 None admin] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:17:06.628360 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG dbcounter [-] [80244] Writing DB stats keystone:SELECT=100 {{(pid=80244) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:17:06.636584 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7ed2035-15ec-4525-8dba-4f1c54049ef5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80245) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:17:06.637483 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7ed2035-15ec-4525-8dba-4f1c54049ef5 None admin] REQUEST_METHOD: `GET` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:17:06.637778 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7ed2035-15ec-4525-8dba-4f1c54049ef5 None admin] SCRIPT_NAME: `/identity` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:17:06.638057 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7ed2035-15ec-4525-8dba-4f1c54049ef5 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:17:06.649495 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7ed2035-15ec-4525-8dba-4f1c54049ef5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:17:06.650942 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7ed2035-15ec-4525-8dba-4f1c54049ef5 None admin] RBAC: Authorization granted {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:17:06.668830 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application [None req-f7ed2035-15ec-4525-8dba-4f1c54049ef5 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 08:17:06.668830 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 08:17:06.668830 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 08:17:06.668830 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 08:17:06.668830 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 08:17:06.668830 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 08:17:06.668830 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 08:17:06.668830 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 08:17:06.668830 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 08:17:06.668830 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 08:17:06.668830 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 08:17:06.668830 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 08:17:06.668830 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 21 08:17:06.668830 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 21 08:17:06.668830 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 21 08:17:06.669713 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 21 08:17:06.669713 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 08:17:06.669713 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 08:17:06.669713 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 08:17:06.669713 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 08:17:06.669713 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 08:17:06.669713 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 08:17:06.669713 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 08:17:06.669713 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application with Lock( Jul 21 08:17:06.669713 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 08:17:06.669713 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application return self._enter() Jul 21 08:17:06.669713 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 08:17:06.669713 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 08:17:06.669713 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 08:17:06.669713 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application return self.creator() Jul 21 08:17:06.669713 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 08:17:06.669713 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application created_value = creator( Jul 21 08:17:06.670589 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 21 08:17:06.670589 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 21 08:17:06.670589 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 08:17:06.670589 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application Jul 21 08:17:06.670589 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: [pid: 80245|app: 0|req: 258/514] 10.222.0.47 () {56 vars in 1108 bytes} [Thu Jul 21 08:17:06 2022] GET /identity/v3/domains/Default => generated 87 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 08:17:06.670866 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7ed2035-15ec-4525-8dba-4f1c54049ef5 None admin] Authenticating user token {{(pid=80245) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:17:06.701233 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG dbcounter [-] [80245] Writing DB stats keystone:SELECT=100 {{(pid=80245) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:17:06.717763 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4165ac3c-3fb2-4612-8c27-fb954a494b94 None admin] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:17:06.751956 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e181b369-9aee-4f9d-809a-ce21ca15f0e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:17:06.752741 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e181b369-9aee-4f9d-809a-ce21ca15f0e0 None admin] REQUEST_METHOD: `GET` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:17:06.753026 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e181b369-9aee-4f9d-809a-ce21ca15f0e0 None admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:17:06.753304 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e181b369-9aee-4f9d-809a-ce21ca15f0e0 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:17:06.764011 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e181b369-9aee-4f9d-809a-ce21ca15f0e0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:17:06.765402 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e181b369-9aee-4f9d-809a-ce21ca15f0e0 None admin] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:17:06.769059 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7ed2035-15ec-4525-8dba-4f1c54049ef5 None admin] Validating token access rules against request {{(pid=80245) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:17:06.783385 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application [None req-e181b369-9aee-4f9d-809a-ce21ca15f0e0 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 08:17:06.783385 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 08:17:06.783385 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 08:17:06.783385 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 08:17:06.783385 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 08:17:06.783385 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 08:17:06.783385 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 08:17:06.783385 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 08:17:06.783385 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 08:17:06.783385 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 08:17:06.783385 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 08:17:06.783385 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 08:17:06.783385 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 21 08:17:06.783385 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 21 08:17:06.783385 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 21 08:17:06.784346 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 21 08:17:06.784346 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 08:17:06.784346 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 08:17:06.784346 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 08:17:06.784346 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 08:17:06.784346 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 08:17:06.784346 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 08:17:06.784346 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 08:17:06.784346 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application with Lock( Jul 21 08:17:06.784346 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 08:17:06.784346 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self._enter() Jul 21 08:17:06.784346 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 08:17:06.784346 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 08:17:06.784346 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 08:17:06.784346 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self.creator() Jul 21 08:17:06.784346 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 08:17:06.784346 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application created_value = creator( Jul 21 08:17:06.785233 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 21 08:17:06.785233 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 21 08:17:06.785233 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 08:17:06.785233 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application Jul 21 08:17:06.785233 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 257/515] 10.222.0.47 () {56 vars in 1108 bytes} [Thu Jul 21 08:17:06 2022] GET /identity/v3/domains/Default => generated 87 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 08:17:06.785828 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e181b369-9aee-4f9d-809a-ce21ca15f0e0 None admin] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:17:06.809227 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9557ab90-0a38-44f7-9bc5-8b7e98a6c176 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80245) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:17:06.810030 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9557ab90-0a38-44f7-9bc5-8b7e98a6c176 None admin] REQUEST_METHOD: `GET` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:17:06.810311 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9557ab90-0a38-44f7-9bc5-8b7e98a6c176 None admin] SCRIPT_NAME: `/identity` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:17:06.810584 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9557ab90-0a38-44f7-9bc5-8b7e98a6c176 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:17:06.818754 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9557ab90-0a38-44f7-9bc5-8b7e98a6c176 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:17:06.820246 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9557ab90-0a38-44f7-9bc5-8b7e98a6c176 None admin] RBAC: Authorization granted {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:17:06.836094 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application [None req-9557ab90-0a38-44f7-9bc5-8b7e98a6c176 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Jul 21 08:17:06.836094 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 08:17:06.836094 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 08:17:06.836094 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 08:17:06.836094 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 08:17:06.836094 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 08:17:06.836094 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 08:17:06.836094 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 08:17:06.836094 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 08:17:06.836094 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 08:17:06.836094 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 08:17:06.836094 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 08:17:06.836094 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 21 08:17:06.836094 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 21 08:17:06.836094 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 21 08:17:06.836094 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 21 08:17:06.837228 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 08:17:06.837228 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 08:17:06.837228 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 21 08:17:06.837228 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 21 08:17:06.837228 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 21 08:17:06.837228 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 21 08:17:06.837228 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 08:17:06.837228 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 08:17:06.837228 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 08:17:06.837228 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 08:17:06.837228 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 08:17:06.837228 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application with Lock( Jul 21 08:17:06.837228 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 08:17:06.837228 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application return self._enter() Jul 21 08:17:06.837228 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 08:17:06.837228 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 08:17:06.837228 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 08:17:06.837228 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application return self.creator() Jul 21 08:17:06.838790 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 08:17:06.838790 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application created_value = creator( Jul 21 08:17:06.838790 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 21 08:17:06.838790 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 21 08:17:06.838790 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 21 08:17:06.838790 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 21 08:17:06.838790 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 21 08:17:06.838790 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 21 08:17:06.838790 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Jul 21 08:17:06.838790 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application Jul 21 08:17:06.838790 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: [pid: 80245|app: 0|req: 259/516] 10.222.0.47 () {56 vars in 1099 bytes} [Thu Jul 21 08:17:06 2022] GET /identity/v3/users/cinder => generated 84 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 08:17:06.838790 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9557ab90-0a38-44f7-9bc5-8b7e98a6c176 None admin] Authenticating user token {{(pid=80245) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:17:06.858542 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG dbcounter [-] [80244] Writing DB stats keystone:SELECT=100 {{(pid=80244) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:17:06.898839 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e181b369-9aee-4f9d-809a-ce21ca15f0e0 None admin] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:17:06.930518 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG dbcounter [-] [80245] Writing DB stats keystone:SELECT=100 {{(pid=80245) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:17:06.932529 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-256075a8-e839-49f3-826f-51778265d1f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:17:06.933196 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-256075a8-e839-49f3-826f-51778265d1f3 None admin] REQUEST_METHOD: `GET` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:17:06.933467 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-256075a8-e839-49f3-826f-51778265d1f3 None admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:17:06.933740 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-256075a8-e839-49f3-826f-51778265d1f3 None admin] PATH_INFO: `/v3/domains` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:17:06.934457 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-256075a8-e839-49f3-826f-51778265d1f3 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80244) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 08:17:06.934758 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-256075a8-e839-49f3-826f-51778265d1f3 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:17:06.936156 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-256075a8-e839-49f3-826f-51778265d1f3 None admin] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:17:06.940081 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9557ab90-0a38-44f7-9bc5-8b7e98a6c176 None admin] Validating token access rules against request {{(pid=80245) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:17:06.948334 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 258/517] 10.222.0.47 () {56 vars in 1109 bytes} [Thu Jul 21 08:17:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 08:17:06.949147 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-256075a8-e839-49f3-826f-51778265d1f3 None admin] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:17:06.977753 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e2a8a1c-33ee-4bd3-948c-fb6b28554f17 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80245) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:17:06.978432 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e2a8a1c-33ee-4bd3-948c-fb6b28554f17 None admin] REQUEST_METHOD: `POST` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:17:06.978715 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e2a8a1c-33ee-4bd3-948c-fb6b28554f17 None admin] SCRIPT_NAME: `/identity` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:17:06.978990 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e2a8a1c-33ee-4bd3-948c-fb6b28554f17 None admin] PATH_INFO: `/v3/users` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:17:06.979716 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e2a8a1c-33ee-4bd3-948c-fb6b28554f17 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:17:06.981182 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e2a8a1c-33ee-4bd3-948c-fb6b28554f17 None admin] RBAC: Authorization granted {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:17:07.000119 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: [pid: 80245|app: 0|req: 260/518] 10.222.0.47 () {60 vars in 1132 bytes} [Thu Jul 21 08:17:06 2022] POST /identity/v3/users => generated 247 bytes in 163 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 08:17:07.000929 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e2a8a1c-33ee-4bd3-948c-fb6b28554f17 None admin] Authenticating user token {{(pid=80245) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:17:07.059130 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-256075a8-e839-49f3-826f-51778265d1f3 None admin] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:17:07.089416 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG dbcounter [-] [80244] Writing DB stats keystone:SELECT=100 {{(pid=80244) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:17:07.103051 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb600461-055c-4c56-a01b-5dbd44a8bddc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:17:07.103773 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb600461-055c-4c56-a01b-5dbd44a8bddc None admin] REQUEST_METHOD: `GET` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:17:07.104058 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb600461-055c-4c56-a01b-5dbd44a8bddc None admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:17:07.104338 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb600461-055c-4c56-a01b-5dbd44a8bddc None admin] PATH_INFO: `/v3/domains` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:17:07.105079 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb600461-055c-4c56-a01b-5dbd44a8bddc None admin] RBAC: Adding query filter params (name=Default) {{(pid=80244) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 08:17:07.105394 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb600461-055c-4c56-a01b-5dbd44a8bddc None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:17:07.106852 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb600461-055c-4c56-a01b-5dbd44a8bddc None admin] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:17:07.120694 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 259/519] 10.222.0.47 () {56 vars in 1109 bytes} [Thu Jul 21 08:17:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 08:17:07.121296 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e2a8a1c-33ee-4bd3-948c-fb6b28554f17 None admin] Validating token access rules against request {{(pid=80245) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:17:07.121958 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb600461-055c-4c56-a01b-5dbd44a8bddc None admin] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:17:07.164937 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c82ab9ca-3422-4bfd-b61a-c52107faba20 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80245) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:17:07.166221 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c82ab9ca-3422-4bfd-b61a-c52107faba20 None admin] REQUEST_METHOD: `GET` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:17:07.166221 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c82ab9ca-3422-4bfd-b61a-c52107faba20 None admin] SCRIPT_NAME: `/identity` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:17:07.166221 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c82ab9ca-3422-4bfd-b61a-c52107faba20 None admin] PATH_INFO: `/v3/domains` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:17:07.167448 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c82ab9ca-3422-4bfd-b61a-c52107faba20 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80245) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 08:17:07.167448 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c82ab9ca-3422-4bfd-b61a-c52107faba20 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:17:07.168554 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c82ab9ca-3422-4bfd-b61a-c52107faba20 None admin] RBAC: Authorization granted {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:17:07.181606 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: [pid: 80245|app: 0|req: 261/520] 10.222.0.47 () {56 vars in 1109 bytes} [Thu Jul 21 08:17:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 08:17:07.182373 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c82ab9ca-3422-4bfd-b61a-c52107faba20 None admin] Authenticating user token {{(pid=80245) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:17:07.195372 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG dbcounter [-] [80245] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80245) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:17:07.245261 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb600461-055c-4c56-a01b-5dbd44a8bddc None admin] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:17:07.282703 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2d33546-60bf-4ab4-899a-02c7a3dc572f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:17:07.283506 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2d33546-60bf-4ab4-899a-02c7a3dc572f None admin] REQUEST_METHOD: `GET` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:17:07.283891 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2d33546-60bf-4ab4-899a-02c7a3dc572f None admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:17:07.284180 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2d33546-60bf-4ab4-899a-02c7a3dc572f None admin] PATH_INFO: `/v3/users` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:17:07.285018 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2d33546-60bf-4ab4-899a-02c7a3dc572f None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=80244) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 08:17:07.285327 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2d33546-60bf-4ab4-899a-02c7a3dc572f None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:17:07.286848 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2d33546-60bf-4ab4-899a-02c7a3dc572f None admin] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:17:07.292379 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c82ab9ca-3422-4bfd-b61a-c52107faba20 None admin] Validating token access rules against request {{(pid=80245) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:17:07.309855 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 260/521] 10.222.0.47 () {56 vars in 1137 bytes} [Thu Jul 21 08:17:07 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 08:17:07.309855 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2d33546-60bf-4ab4-899a-02c7a3dc572f None admin] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:17:07.334241 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-613b5600-bd67-428a-ab5a-b74f7c156af5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80245) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:17:07.335045 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-613b5600-bd67-428a-ab5a-b74f7c156af5 None admin] REQUEST_METHOD: `GET` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:17:07.335346 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-613b5600-bd67-428a-ab5a-b74f7c156af5 None admin] SCRIPT_NAME: `/identity` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:17:07.335639 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-613b5600-bd67-428a-ab5a-b74f7c156af5 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:17:07.342732 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-613b5600-bd67-428a-ab5a-b74f7c156af5 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:17:07.344233 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-613b5600-bd67-428a-ab5a-b74f7c156af5 None admin] RBAC: Authorization granted {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:17:07.356940 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG dbcounter [-] [80244] Writing DB stats keystone:SELECT=100 {{(pid=80244) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:17:07.359060 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application [None req-613b5600-bd67-428a-ab5a-b74f7c156af5 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 21 08:17:07.359060 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 08:17:07.359060 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 08:17:07.359060 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 08:17:07.359060 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 08:17:07.359060 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 08:17:07.359060 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 08:17:07.359060 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 08:17:07.359060 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 08:17:07.359060 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 08:17:07.359060 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 08:17:07.359060 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 08:17:07.359060 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 21 08:17:07.359060 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 21 08:17:07.359060 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 21 08:17:07.359947 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 21 08:17:07.359947 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 08:17:07.359947 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 08:17:07.359947 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 08:17:07.359947 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 08:17:07.359947 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 08:17:07.359947 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 08:17:07.359947 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 08:17:07.359947 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application with Lock( Jul 21 08:17:07.359947 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 08:17:07.359947 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application return self._enter() Jul 21 08:17:07.359947 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 08:17:07.359947 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 08:17:07.359947 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 08:17:07.359947 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application return self.creator() Jul 21 08:17:07.359947 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 08:17:07.359947 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application created_value = creator( Jul 21 08:17:07.360759 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 21 08:17:07.360759 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 21 08:17:07.360759 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 21 08:17:07.360759 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 21 08:17:07.360759 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 21 08:17:07.360759 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 21 08:17:07.360759 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 21 08:17:07.360759 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application Jul 21 08:17:07.360759 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: [pid: 80245|app: 0|req: 262/522] 10.222.0.47 () {56 vars in 1111 bytes} [Thu Jul 21 08:17:07 2022] GET /identity/v3/projects/service => generated 88 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 08:17:07.361214 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-613b5600-bd67-428a-ab5a-b74f7c156af5 None admin] Authenticating user token {{(pid=80245) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:17:07.424532 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2d33546-60bf-4ab4-899a-02c7a3dc572f None admin] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:17:07.434183 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG dbcounter [-] [80245] Writing DB stats keystone:SELECT=100 {{(pid=80245) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:17:07.461894 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87c7bc73-a123-4a29-b4c6-1e69a3063c56 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:17:07.462482 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87c7bc73-a123-4a29-b4c6-1e69a3063c56 None admin] REQUEST_METHOD: `GET` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:17:07.462767 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87c7bc73-a123-4a29-b4c6-1e69a3063c56 None admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:17:07.463042 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87c7bc73-a123-4a29-b4c6-1e69a3063c56 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:17:07.470411 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-613b5600-bd67-428a-ab5a-b74f7c156af5 None admin] Validating token access rules against request {{(pid=80245) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:17:07.470540 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87c7bc73-a123-4a29-b4c6-1e69a3063c56 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:17:07.475206 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87c7bc73-a123-4a29-b4c6-1e69a3063c56 None admin] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:17:07.493334 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application [None req-87c7bc73-a123-4a29-b4c6-1e69a3063c56 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 21 08:17:07.493334 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 08:17:07.493334 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 08:17:07.493334 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 08:17:07.493334 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 08:17:07.493334 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 08:17:07.493334 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 08:17:07.493334 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 08:17:07.493334 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 08:17:07.493334 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 08:17:07.493334 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 08:17:07.493334 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 08:17:07.493334 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 21 08:17:07.493334 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 21 08:17:07.493334 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 21 08:17:07.494248 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 21 08:17:07.494248 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 08:17:07.494248 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 08:17:07.494248 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 08:17:07.494248 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 08:17:07.494248 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 08:17:07.494248 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 08:17:07.494248 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 08:17:07.494248 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application with Lock( Jul 21 08:17:07.494248 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 08:17:07.494248 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self._enter() Jul 21 08:17:07.494248 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 08:17:07.494248 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 08:17:07.494248 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 08:17:07.494248 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self.creator() Jul 21 08:17:07.494248 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 08:17:07.494248 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application created_value = creator( Jul 21 08:17:07.495244 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 21 08:17:07.495244 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 21 08:17:07.495244 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 21 08:17:07.495244 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 21 08:17:07.495244 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 21 08:17:07.495244 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 21 08:17:07.495244 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 21 08:17:07.495244 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application Jul 21 08:17:07.499207 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 261/523] 10.222.0.47 () {56 vars in 1111 bytes} [Thu Jul 21 08:17:07 2022] GET /identity/v3/projects/service => generated 88 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 08:17:07.499981 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87c7bc73-a123-4a29-b4c6-1e69a3063c56 None admin] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:17:07.514981 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7301067a-9bc3-4692-a4e3-21bac1c41975 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80245) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:17:07.515976 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7301067a-9bc3-4692-a4e3-21bac1c41975 None admin] REQUEST_METHOD: `GET` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:17:07.516353 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7301067a-9bc3-4692-a4e3-21bac1c41975 None admin] SCRIPT_NAME: `/identity` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:17:07.516727 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7301067a-9bc3-4692-a4e3-21bac1c41975 None admin] PATH_INFO: `/v3/users/neutron` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:17:07.525081 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7301067a-9bc3-4692-a4e3-21bac1c41975 None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:17:07.526628 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7301067a-9bc3-4692-a4e3-21bac1c41975 None admin] RBAC: Authorization granted {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:17:07.545623 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application [None req-7301067a-9bc3-4692-a4e3-21bac1c41975 None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Jul 21 08:17:07.545623 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 08:17:07.545623 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 08:17:07.545623 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 08:17:07.545623 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 08:17:07.545623 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 08:17:07.545623 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 08:17:07.545623 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 08:17:07.545623 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 08:17:07.545623 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 08:17:07.545623 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 08:17:07.545623 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 08:17:07.545623 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 21 08:17:07.545623 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 21 08:17:07.545623 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 21 08:17:07.545623 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 21 08:17:07.546756 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 08:17:07.546756 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 08:17:07.546756 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 21 08:17:07.546756 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 21 08:17:07.546756 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 21 08:17:07.546756 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 21 08:17:07.546756 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 08:17:07.546756 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 08:17:07.546756 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 08:17:07.546756 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 08:17:07.546756 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 08:17:07.546756 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application with Lock( Jul 21 08:17:07.546756 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 08:17:07.546756 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application return self._enter() Jul 21 08:17:07.546756 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 08:17:07.546756 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 08:17:07.546756 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 08:17:07.546756 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application return self.creator() Jul 21 08:17:07.547870 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 08:17:07.547870 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application created_value = creator( Jul 21 08:17:07.547870 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 21 08:17:07.547870 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 21 08:17:07.547870 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 21 08:17:07.547870 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 21 08:17:07.547870 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 21 08:17:07.547870 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 21 08:17:07.547870 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Jul 21 08:17:07.547870 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application Jul 21 08:17:07.548437 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: [pid: 80245|app: 0|req: 263/524] 10.222.0.47 () {56 vars in 1102 bytes} [Thu Jul 21 08:17:07 2022] GET /identity/v3/users/neutron => generated 85 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 08:17:07.548786 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7301067a-9bc3-4692-a4e3-21bac1c41975 None admin] Authenticating user token {{(pid=80245) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:17:07.617584 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87c7bc73-a123-4a29-b4c6-1e69a3063c56 None admin] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:17:07.619966 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG dbcounter [-] [80244] Writing DB stats keystone:SELECT=100 {{(pid=80244) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:17:07.663416 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf3dd7ef-0518-4f92-bf7e-3829d6fb60c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:17:07.664315 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf3dd7ef-0518-4f92-bf7e-3829d6fb60c0 None admin] REQUEST_METHOD: `GET` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:17:07.664597 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf3dd7ef-0518-4f92-bf7e-3829d6fb60c0 None admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:17:07.665290 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf3dd7ef-0518-4f92-bf7e-3829d6fb60c0 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:17:07.676179 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7301067a-9bc3-4692-a4e3-21bac1c41975 None admin] Validating token access rules against request {{(pid=80245) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:17:07.707302 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG dbcounter [-] [80245] Writing DB stats keystone:SELECT=100 {{(pid=80245) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:17:07.718334 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24e8e1f2-a215-41d2-89e5-c406ba09cedb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80245) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:17:07.719232 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24e8e1f2-a215-41d2-89e5-c406ba09cedb None admin] REQUEST_METHOD: `GET` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:17:07.719554 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24e8e1f2-a215-41d2-89e5-c406ba09cedb None admin] SCRIPT_NAME: `/identity` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:17:07.719841 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24e8e1f2-a215-41d2-89e5-c406ba09cedb None admin] PATH_INFO: `/v3/projects` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:17:07.720860 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24e8e1f2-a215-41d2-89e5-c406ba09cedb None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80245) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 08:17:07.720993 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24e8e1f2-a215-41d2-89e5-c406ba09cedb None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:17:07.722341 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24e8e1f2-a215-41d2-89e5-c406ba09cedb None admin] RBAC: Authorization granted {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:17:07.746742 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: [pid: 80245|app: 0|req: 264/525] 10.222.0.47 () {56 vars in 1148 bytes} [Thu Jul 21 08:17:07 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 08:17:07.746742 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24e8e1f2-a215-41d2-89e5-c406ba09cedb None admin] Authenticating user token {{(pid=80245) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:17:07.748900 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf3dd7ef-0518-4f92-bf7e-3829d6fb60c0 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:17:07.750025 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf3dd7ef-0518-4f92-bf7e-3829d6fb60c0 None admin] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:17:07.849798 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24e8e1f2-a215-41d2-89e5-c406ba09cedb None admin] Validating token access rules against request {{(pid=80245) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:17:07.857750 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 262/526] 10.222.0.47 () {58 vars in 1282 bytes} [Thu Jul 21 08:17:07 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 358 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 21 08:17:07.860248 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf3dd7ef-0518-4f92-bf7e-3829d6fb60c0 None admin] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:17:07.865753 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG dbcounter [-] [80244] Writing DB stats keystone:SELECT=100 {{(pid=80244) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:17:07.890879 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04c82f5f-e5aa-4e7a-9e16-e92190bb3363 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80245) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:17:07.891976 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04c82f5f-e5aa-4e7a-9e16-e92190bb3363 None admin] REQUEST_METHOD: `GET` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:17:07.892309 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04c82f5f-e5aa-4e7a-9e16-e92190bb3363 None admin] SCRIPT_NAME: `/identity` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:17:07.892593 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04c82f5f-e5aa-4e7a-9e16-e92190bb3363 None admin] PATH_INFO: `/v3/projects` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:17:07.893666 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04c82f5f-e5aa-4e7a-9e16-e92190bb3363 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80245) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 08:17:07.893802 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04c82f5f-e5aa-4e7a-9e16-e92190bb3363 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:17:07.895438 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04c82f5f-e5aa-4e7a-9e16-e92190bb3363 None admin] RBAC: Authorization granted {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:17:07.908454 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: [pid: 80245|app: 0|req: 265/527] 10.222.0.47 () {56 vars in 1148 bytes} [Thu Jul 21 08:17:07 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 08:17:07.909287 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04c82f5f-e5aa-4e7a-9e16-e92190bb3363 None admin] Authenticating user token {{(pid=80245) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:17:07.964808 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG dbcounter [-] [80245] Writing DB stats keystone:SELECT=100 {{(pid=80245) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:17:07.972357 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf3dd7ef-0518-4f92-bf7e-3829d6fb60c0 None admin] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:17:08.006325 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d5741b9-fdb3-4813-a4d5-35ab7170df1e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:17:08.006997 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d5741b9-fdb3-4813-a4d5-35ab7170df1e None admin] REQUEST_METHOD: `GET` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:17:08.007276 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d5741b9-fdb3-4813-a4d5-35ab7170df1e None admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:17:08.007584 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d5741b9-fdb3-4813-a4d5-35ab7170df1e None admin] PATH_INFO: `/v3/users` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:17:08.008671 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d5741b9-fdb3-4813-a4d5-35ab7170df1e None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=80244) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 08:17:08.008976 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d5741b9-fdb3-4813-a4d5-35ab7170df1e None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:17:08.010407 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d5741b9-fdb3-4813-a4d5-35ab7170df1e None admin] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:17:08.021438 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04c82f5f-e5aa-4e7a-9e16-e92190bb3363 None admin] Validating token access rules against request {{(pid=80245) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:17:08.028766 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 263/528] 10.222.0.47 () {56 vars in 1139 bytes} [Thu Jul 21 08:17:07 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 08:17:08.029587 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d5741b9-fdb3-4813-a4d5-35ab7170df1e None admin] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:17:08.062445 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46e57618-a4eb-4f43-b62b-659472720fef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80245) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:17:08.063433 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46e57618-a4eb-4f43-b62b-659472720fef None admin] REQUEST_METHOD: `GET` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:17:08.063895 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46e57618-a4eb-4f43-b62b-659472720fef None admin] SCRIPT_NAME: `/identity` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:17:08.064350 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46e57618-a4eb-4f43-b62b-659472720fef None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:17:08.065684 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46e57618-a4eb-4f43-b62b-659472720fef None admin] RBAC: Adding query filter params (role.id=50d1ec7f947f42acb65c57e07f2269d6, scope.project.id=4faef13aa50e4e36b6c265fb1398fa14, user.id=ac2b5b99a36442148fe743269054e991) {{(pid=80245) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 08:17:08.066001 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46e57618-a4eb-4f43-b62b-659472720fef None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:17:08.067500 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46e57618-a4eb-4f43-b62b-659472720fef None admin] RBAC: Authorization granted {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:17:08.083505 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: [pid: 80245|app: 0|req: 266/529] 10.222.0.47 () {56 vars in 1374 bytes} [Thu Jul 21 08:17:07 2022] GET /identity/v3/role_assignments?user.id=ac2b5b99a36442148fe743269054e991&scope.project.id=4faef13aa50e4e36b6c265fb1398fa14&role.id=50d1ec7f947f42acb65c57e07f2269d6 => generated 259 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 08:17:08.083505 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46e57618-a4eb-4f43-b62b-659472720fef None admin] Authenticating user token {{(pid=80245) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:17:08.104488 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG dbcounter [-] [80244] Writing DB stats keystone:SELECT=100 {{(pid=80244) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:17:08.160419 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d5741b9-fdb3-4813-a4d5-35ab7170df1e None admin] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:17:08.209392 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a2f937e-a771-40d8-b2d4-d53a98bb02a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:17:08.210209 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a2f937e-a771-40d8-b2d4-d53a98bb02a5 None admin] REQUEST_METHOD: `GET` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:17:08.210769 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a2f937e-a771-40d8-b2d4-d53a98bb02a5 None admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:17:08.211153 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a2f937e-a771-40d8-b2d4-d53a98bb02a5 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:17:08.224460 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46e57618-a4eb-4f43-b62b-659472720fef None admin] Validating token access rules against request {{(pid=80245) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:17:08.225813 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a2f937e-a771-40d8-b2d4-d53a98bb02a5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:17:08.228561 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a2f937e-a771-40d8-b2d4-d53a98bb02a5 None admin] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:17:08.241949 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG dbcounter [-] [80245] Writing DB stats keystone:SELECT=100 {{(pid=80245) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:17:08.250756 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application [None req-4a2f937e-a771-40d8-b2d4-d53a98bb02a5 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 08:17:08.250756 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 08:17:08.250756 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 08:17:08.250756 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 08:17:08.250756 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 08:17:08.250756 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 08:17:08.250756 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 08:17:08.250756 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 08:17:08.250756 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 08:17:08.250756 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 08:17:08.250756 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 08:17:08.250756 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 08:17:08.250756 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 21 08:17:08.250756 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 21 08:17:08.250756 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 21 08:17:08.251712 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 21 08:17:08.251712 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 08:17:08.251712 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 08:17:08.251712 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 08:17:08.251712 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 08:17:08.251712 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 08:17:08.251712 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 08:17:08.251712 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 08:17:08.251712 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application with Lock( Jul 21 08:17:08.251712 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 08:17:08.251712 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self._enter() Jul 21 08:17:08.251712 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 08:17:08.251712 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 08:17:08.251712 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 08:17:08.251712 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self.creator() Jul 21 08:17:08.251712 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 08:17:08.251712 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application created_value = creator( Jul 21 08:17:08.252542 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 21 08:17:08.252542 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 21 08:17:08.252542 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 08:17:08.252542 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application Jul 21 08:17:08.257421 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 264/530] 10.222.0.47 () {56 vars in 1108 bytes} [Thu Jul 21 08:17:08 2022] GET /identity/v3/domains/Default => generated 87 bytes in 223 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 08:17:08.258895 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a2f937e-a771-40d8-b2d4-d53a98bb02a5 None admin] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:17:08.278389 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b44619b-3fc0-4a08-8caf-c6fe15e2be5f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80245) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:17:08.279700 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b44619b-3fc0-4a08-8caf-c6fe15e2be5f None admin] REQUEST_METHOD: `GET` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:17:08.280192 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b44619b-3fc0-4a08-8caf-c6fe15e2be5f None admin] SCRIPT_NAME: `/identity` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:17:08.280895 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b44619b-3fc0-4a08-8caf-c6fe15e2be5f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:17:08.282355 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b44619b-3fc0-4a08-8caf-c6fe15e2be5f None admin] RBAC: Adding query filter params (role.id=50d1ec7f947f42acb65c57e07f2269d6, scope.project.id=4faef13aa50e4e36b6c265fb1398fa14, user.id=9b0524aed8c74abd8949b860d6b93936) {{(pid=80245) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 08:17:08.282879 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b44619b-3fc0-4a08-8caf-c6fe15e2be5f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:17:08.285292 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b44619b-3fc0-4a08-8caf-c6fe15e2be5f None admin] RBAC: Authorization granted {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:17:08.299796 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: [pid: 80245|app: 0|req: 267/531] 10.222.0.47 () {56 vars in 1374 bytes} [Thu Jul 21 08:17:08 2022] GET /identity/v3/role_assignments?user.id=9b0524aed8c74abd8949b860d6b93936&scope.project.id=4faef13aa50e4e36b6c265fb1398fa14&role.id=50d1ec7f947f42acb65c57e07f2269d6 => generated 259 bytes in 220 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 08:17:08.299796 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b44619b-3fc0-4a08-8caf-c6fe15e2be5f None admin] Authenticating user token {{(pid=80245) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:17:08.452579 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a2f937e-a771-40d8-b2d4-d53a98bb02a5 None admin] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:17:08.487453 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG dbcounter [-] [80244] Writing DB stats keystone:SELECT=100 {{(pid=80244) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:17:08.494735 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b44619b-3fc0-4a08-8caf-c6fe15e2be5f None admin] Validating token access rules against request {{(pid=80245) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:17:08.529994 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b77c5e2-6662-42c1-a46b-73cd50a24691 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:17:08.530996 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b77c5e2-6662-42c1-a46b-73cd50a24691 None admin] REQUEST_METHOD: `GET` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:17:08.531519 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b77c5e2-6662-42c1-a46b-73cd50a24691 None admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:17:08.532000 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b77c5e2-6662-42c1-a46b-73cd50a24691 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:17:08.532294 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6bdb283-fe75-4f0d-b8ec-69eef06bd270 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80245) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:17:08.532971 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6bdb283-fe75-4f0d-b8ec-69eef06bd270 None admin] REQUEST_METHOD: `GET` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:17:08.533252 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6bdb283-fe75-4f0d-b8ec-69eef06bd270 None admin] SCRIPT_NAME: `/identity` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:17:08.533532 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6bdb283-fe75-4f0d-b8ec-69eef06bd270 None admin] PATH_INFO: `/v3/domains` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:17:08.534254 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6bdb283-fe75-4f0d-b8ec-69eef06bd270 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80245) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 08:17:08.534561 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6bdb283-fe75-4f0d-b8ec-69eef06bd270 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:17:08.536553 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6bdb283-fe75-4f0d-b8ec-69eef06bd270 None admin] RBAC: Authorization granted {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:17:08.551163 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: [pid: 80245|app: 0|req: 268/532] 10.222.0.47 () {56 vars in 1109 bytes} [Thu Jul 21 08:17:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 255 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 08:17:08.560516 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6bdb283-fe75-4f0d-b8ec-69eef06bd270 None admin] Authenticating user token {{(pid=80245) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:17:08.604398 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG dbcounter [-] [80245] Writing DB stats keystone:SELECT=100 {{(pid=80245) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:17:08.681871 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6bdb283-fe75-4f0d-b8ec-69eef06bd270 None admin] Validating token access rules against request {{(pid=80245) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:17:08.698917 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b77c5e2-6662-42c1-a46b-73cd50a24691 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:17:08.700516 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b77c5e2-6662-42c1-a46b-73cd50a24691 None admin] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:17:08.728994 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c79c755-8e74-4d53-9dc7-9e8bb57aa592 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80245) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:17:08.730103 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c79c755-8e74-4d53-9dc7-9e8bb57aa592 None admin] REQUEST_METHOD: `GET` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:17:08.730571 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c79c755-8e74-4d53-9dc7-9e8bb57aa592 None admin] SCRIPT_NAME: `/identity` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:17:08.731051 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c79c755-8e74-4d53-9dc7-9e8bb57aa592 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:17:08.746676 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c79c755-8e74-4d53-9dc7-9e8bb57aa592 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:17:08.748445 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c79c755-8e74-4d53-9dc7-9e8bb57aa592 None admin] RBAC: Authorization granted {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:17:08.776892 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application [None req-2c79c755-8e74-4d53-9dc7-9e8bb57aa592 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 21 08:17:08.776892 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 08:17:08.776892 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 08:17:08.776892 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 08:17:08.776892 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 08:17:08.776892 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 08:17:08.776892 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 08:17:08.776892 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 08:17:08.776892 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 08:17:08.776892 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 08:17:08.776892 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 08:17:08.776892 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 08:17:08.776892 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 21 08:17:08.776892 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 21 08:17:08.776892 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 21 08:17:08.777800 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 21 08:17:08.777800 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 08:17:08.777800 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 08:17:08.777800 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 08:17:08.777800 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 08:17:08.777800 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 08:17:08.777800 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 08:17:08.777800 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 08:17:08.777800 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application with Lock( Jul 21 08:17:08.777800 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 08:17:08.777800 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application return self._enter() Jul 21 08:17:08.777800 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 08:17:08.777800 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 08:17:08.777800 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 08:17:08.777800 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application return self.creator() Jul 21 08:17:08.777800 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 08:17:08.777800 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application created_value = creator( Jul 21 08:17:08.778614 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 21 08:17:08.778614 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 21 08:17:08.778614 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 21 08:17:08.778614 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 21 08:17:08.778614 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 21 08:17:08.778614 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 21 08:17:08.778614 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 21 08:17:08.778614 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application Jul 21 08:17:08.789785 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: [pid: 80245|app: 0|req: 269/533] 10.222.0.47 () {56 vars in 1111 bytes} [Thu Jul 21 08:17:08 2022] GET /identity/v3/projects/service => generated 88 bytes in 230 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 08:17:08.795105 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c79c755-8e74-4d53-9dc7-9e8bb57aa592 None admin] Authenticating user token {{(pid=80245) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:17:08.901022 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 265/534] 10.222.0.47 () {58 vars in 1282 bytes} [Thu Jul 21 08:17:08 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 643 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 21 08:17:08.907182 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG dbcounter [-] [80245] Writing DB stats keystone:SELECT=100 {{(pid=80245) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:17:08.908282 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b77c5e2-6662-42c1-a46b-73cd50a24691 None admin] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:17:08.913897 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c79c755-8e74-4d53-9dc7-9e8bb57aa592 None admin] Validating token access rules against request {{(pid=80245) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:17:08.939724 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG dbcounter [-] [80244] Writing DB stats keystone:SELECT=100 {{(pid=80244) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:17:08.969810 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07b82deb-e49c-49f0-adfb-5a1fdfb87076 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80245) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:17:08.970505 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07b82deb-e49c-49f0-adfb-5a1fdfb87076 None admin] REQUEST_METHOD: `GET` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:17:08.970774 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07b82deb-e49c-49f0-adfb-5a1fdfb87076 None admin] SCRIPT_NAME: `/identity` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:17:08.971055 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07b82deb-e49c-49f0-adfb-5a1fdfb87076 None admin] PATH_INFO: `/v3/projects` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:17:08.971885 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07b82deb-e49c-49f0-adfb-5a1fdfb87076 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80245) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 08:17:08.972199 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07b82deb-e49c-49f0-adfb-5a1fdfb87076 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:17:08.974197 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07b82deb-e49c-49f0-adfb-5a1fdfb87076 None admin] RBAC: Authorization granted {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:17:08.994982 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: [pid: 80245|app: 0|req: 270/535] 10.222.0.47 () {56 vars in 1148 bytes} [Thu Jul 21 08:17:08 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 08:17:09.000541 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07b82deb-e49c-49f0-adfb-5a1fdfb87076 None admin] Authenticating user token {{(pid=80245) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:17:09.092761 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b77c5e2-6662-42c1-a46b-73cd50a24691 None admin] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:17:09.147470 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a432b2fd-254a-42ac-913c-43cfe893a954 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:17:09.148273 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a432b2fd-254a-42ac-913c-43cfe893a954 None admin] REQUEST_METHOD: `GET` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:17:09.148553 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a432b2fd-254a-42ac-913c-43cfe893a954 None admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:17:09.148824 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a432b2fd-254a-42ac-913c-43cfe893a954 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:17:09.160157 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a432b2fd-254a-42ac-913c-43cfe893a954 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:17:09.162283 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a432b2fd-254a-42ac-913c-43cfe893a954 None admin] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:17:09.169801 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07b82deb-e49c-49f0-adfb-5a1fdfb87076 None admin] Validating token access rules against request {{(pid=80245) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:17:09.181622 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application [None req-a432b2fd-254a-42ac-913c-43cfe893a954 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 08:17:09.181622 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 08:17:09.181622 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 08:17:09.181622 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 08:17:09.181622 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 08:17:09.181622 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 08:17:09.181622 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 08:17:09.181622 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 08:17:09.181622 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 08:17:09.181622 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 08:17:09.181622 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 08:17:09.181622 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 08:17:09.181622 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 21 08:17:09.181622 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 21 08:17:09.181622 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 21 08:17:09.182448 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 21 08:17:09.182448 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 08:17:09.182448 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 08:17:09.182448 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 08:17:09.182448 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 08:17:09.182448 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 08:17:09.182448 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 08:17:09.182448 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 08:17:09.182448 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application with Lock( Jul 21 08:17:09.182448 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 08:17:09.182448 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self._enter() Jul 21 08:17:09.182448 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 08:17:09.182448 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 08:17:09.182448 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 08:17:09.182448 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self.creator() Jul 21 08:17:09.182448 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 08:17:09.182448 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application created_value = creator( Jul 21 08:17:09.183266 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 21 08:17:09.183266 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 21 08:17:09.183266 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 08:17:09.183266 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application Jul 21 08:17:09.183266 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 266/536] 10.222.0.47 () {56 vars in 1108 bytes} [Thu Jul 21 08:17:08 2022] GET /identity/v3/domains/Default => generated 87 bytes in 277 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 21 08:17:09.184533 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9bf8409-4c34-4230-8d3b-2792c3bb3b14 None None] REQUEST_METHOD: `GET` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:17:09.186721 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9bf8409-4c34-4230-8d3b-2792c3bb3b14 None None] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:17:09.186721 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9bf8409-4c34-4230-8d3b-2792c3bb3b14 None None] PATH_INFO: `/` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:17:09.190709 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 267/537] 10.222.0.47 () {54 vars in 926 bytes} [Thu Jul 21 08:17:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 08:17:09.192108 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82135d3e-6cee-4e36-8ef6-6a41f7175763 None None] REQUEST_METHOD: `POST` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:17:09.192399 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82135d3e-6cee-4e36-8ef6-6a41f7175763 None None] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:17:09.192678 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82135d3e-6cee-4e36-8ef6-6a41f7175763 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:17:09.206884 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0304382-cefc-48bb-b799-6fe612e9d51a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80245) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:17:09.207913 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0304382-cefc-48bb-b799-6fe612e9d51a None admin] REQUEST_METHOD: `GET` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:17:09.208199 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0304382-cefc-48bb-b799-6fe612e9d51a None admin] SCRIPT_NAME: `/identity` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:17:09.208477 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0304382-cefc-48bb-b799-6fe612e9d51a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:17:09.209312 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0304382-cefc-48bb-b799-6fe612e9d51a None admin] RBAC: Adding query filter params (role.id=50d1ec7f947f42acb65c57e07f2269d6, scope.project.id=4faef13aa50e4e36b6c265fb1398fa14, user.id=b4f40410b7674687ab5362ac982248ef) {{(pid=80245) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 08:17:09.209621 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0304382-cefc-48bb-b799-6fe612e9d51a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:17:09.211038 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.auth.core [None req-82135d3e-6cee-4e36-8ef6-6a41f7175763 None None] MFA Rules not processed for user `2dce40b8d74c41ec94cc36e6ff05d412`. Rule list: `[]` (Enabled: `True`). {{(pid=80244) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 08:17:09.211214 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0304382-cefc-48bb-b799-6fe612e9d51a None admin] RBAC: Authorization granted {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:17:09.212285 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.fernet_utils [None req-82135d3e-6cee-4e36-8ef6-6a41f7175763 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80244) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 08:17:09.218471 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: [pid: 80245|app: 0|req: 271/538] 10.222.0.47 () {56 vars in 1374 bytes} [Thu Jul 21 08:17:08 2022] GET /identity/v3/role_assignments?user.id=b4f40410b7674687ab5362ac982248ef&scope.project.id=4faef13aa50e4e36b6c265fb1398fa14&role.id=50d1ec7f947f42acb65c57e07f2269d6 => generated 259 bytes in 218 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 21 08:17:09.219310 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0304382-cefc-48bb-b799-6fe612e9d51a None admin] Authenticating user token {{(pid=80245) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:17:09.245825 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG dbcounter [-] [80245] Writing DB stats keystone:SELECT=100 {{(pid=80245) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:17:09.337764 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG dbcounter [-] [80244] Writing DB stats keystone:SELECT=100 {{(pid=80244) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:17:09.356147 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 268/539] 10.222.0.47 () {58 vars in 1023 bytes} [Thu Jul 21 08:17:09 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 21 08:17:09.379755 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82135d3e-6cee-4e36-8ef6-6a41f7175763 None None] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:17:09.395830 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0304382-cefc-48bb-b799-6fe612e9d51a None admin] Validating token access rules against request {{(pid=80245) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:17:09.437372 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9371332d-d689-4315-b6f7-ad58e0712423 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80245) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:17:09.438052 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9371332d-d689-4315-b6f7-ad58e0712423 None admin] REQUEST_METHOD: `GET` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:17:09.438336 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9371332d-d689-4315-b6f7-ad58e0712423 None admin] SCRIPT_NAME: `/identity` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:17:09.438615 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9371332d-d689-4315-b6f7-ad58e0712423 None admin] PATH_INFO: `/v3/domains` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:17:09.439352 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9371332d-d689-4315-b6f7-ad58e0712423 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80245) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 08:17:09.439697 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9371332d-d689-4315-b6f7-ad58e0712423 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:17:09.441070 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9371332d-d689-4315-b6f7-ad58e0712423 None admin] RBAC: Authorization granted {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:17:09.454545 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: [pid: 80245|app: 0|req: 272/540] 10.222.0.47 () {56 vars in 1109 bytes} [Thu Jul 21 08:17:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 235 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 08:17:09.460589 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9371332d-d689-4315-b6f7-ad58e0712423 None admin] Authenticating user token {{(pid=80245) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:17:09.518604 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82135d3e-6cee-4e36-8ef6-6a41f7175763 None None] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:17:09.558725 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d43b7cd0-f867-4221-b881-add0c1a903d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:17:09.559563 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d43b7cd0-f867-4221-b881-add0c1a903d7 None admin] REQUEST_METHOD: `GET` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:17:09.559845 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d43b7cd0-f867-4221-b881-add0c1a903d7 None admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:17:09.560112 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d43b7cd0-f867-4221-b881-add0c1a903d7 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:17:09.572362 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d43b7cd0-f867-4221-b881-add0c1a903d7 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 21 08:17:09.581156 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application [None req-d43b7cd0-f867-4221-b881-add0c1a903d7 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 21 08:17:09.581156 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 08:17:09.581156 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 08:17:09.581156 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 08:17:09.581156 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 08:17:09.581156 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 08:17:09.581156 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 08:17:09.581156 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 08:17:09.581156 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 08:17:09.581156 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 08:17:09.581156 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 08:17:09.581156 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 08:17:09.581156 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 21 08:17:09.581156 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 21 08:17:09.581156 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 21 08:17:09.581156 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 21 08:17:09.582038 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 21 08:17:09.582038 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 21 08:17:09.582038 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 21 08:17:09.582038 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 21 08:17:09.582038 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 08:17:09.582038 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 08:17:09.582038 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 08:17:09.582038 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 08:17:09.582038 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 08:17:09.582038 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 08:17:09.582038 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 08:17:09.582038 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application with Lock( Jul 21 08:17:09.582038 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 08:17:09.582038 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self._enter() Jul 21 08:17:09.582038 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 08:17:09.582038 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 08:17:09.583387 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 08:17:09.583387 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self.creator() Jul 21 08:17:09.583387 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 08:17:09.583387 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application created_value = creator( Jul 21 08:17:09.583387 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 21 08:17:09.583387 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 21 08:17:09.583387 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 21 08:17:09.583387 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 21 08:17:09.583387 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 21 08:17:09.583387 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 21 08:17:09.583387 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 21 08:17:09.583387 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application Jul 21 08:17:09.583999 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 269/541] 10.222.0.47 () {56 vars in 1102 bytes} [Thu Jul 21 08:17:09 2022] GET /identity/v3/roles/service => generated 85 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 21 08:17:09.592274 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d43b7cd0-f867-4221-b881-add0c1a903d7 None admin] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:17:09.654141 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG dbcounter [-] [80245] Writing DB stats keystone:SELECT=100 {{(pid=80245) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:17:09.680293 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG dbcounter [-] [80244] Writing DB stats keystone:SELECT=100 {{(pid=80244) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:17:09.717541 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9371332d-d689-4315-b6f7-ad58e0712423 None admin] Validating token access rules against request {{(pid=80245) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:17:09.733159 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d43b7cd0-f867-4221-b881-add0c1a903d7 None admin] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:17:09.773442 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62850600-a5b7-4383-b90d-097a5191b132 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:17:09.774391 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62850600-a5b7-4383-b90d-097a5191b132 None admin] REQUEST_METHOD: `GET` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:17:09.774802 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62850600-a5b7-4383-b90d-097a5191b132 None admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:17:09.775202 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62850600-a5b7-4383-b90d-097a5191b132 None admin] PATH_INFO: `/v3/roles` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:17:09.776108 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62850600-a5b7-4383-b90d-097a5191b132 None admin] RBAC: Adding query filter params (name=service) {{(pid=80244) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 08:17:09.776568 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62850600-a5b7-4383-b90d-097a5191b132 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:17:09.778201 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62850600-a5b7-4383-b90d-097a5191b132 None admin] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:17:09.788380 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 270/542] 10.222.0.47 () {56 vars in 1103 bytes} [Thu Jul 21 08:17:09 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 08:17:09.794114 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62850600-a5b7-4383-b90d-097a5191b132 None admin] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:17:09.812028 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7f377cf-e50e-45a9-b512-e060e0972863 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80245) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:17:09.813101 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7f377cf-e50e-45a9-b512-e060e0972863 None admin] REQUEST_METHOD: `GET` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:17:09.813521 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7f377cf-e50e-45a9-b512-e060e0972863 None admin] SCRIPT_NAME: `/identity` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:17:09.813941 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7f377cf-e50e-45a9-b512-e060e0972863 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:17:09.824094 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7f377cf-e50e-45a9-b512-e060e0972863 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:17:09.825880 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7f377cf-e50e-45a9-b512-e060e0972863 None admin] RBAC: Authorization granted {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:17:09.849442 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application [None req-c7f377cf-e50e-45a9-b512-e060e0972863 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 21 08:17:09.849442 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 08:17:09.849442 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 08:17:09.849442 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 08:17:09.849442 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 08:17:09.849442 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 08:17:09.849442 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 08:17:09.849442 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 08:17:09.849442 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 08:17:09.849442 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 08:17:09.849442 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 08:17:09.849442 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 08:17:09.849442 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 21 08:17:09.849442 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 21 08:17:09.849442 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 21 08:17:09.850442 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 21 08:17:09.850442 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 08:17:09.850442 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 08:17:09.850442 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 08:17:09.850442 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 08:17:09.850442 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 08:17:09.850442 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 08:17:09.850442 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 08:17:09.850442 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application with Lock( Jul 21 08:17:09.850442 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 08:17:09.850442 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application return self._enter() Jul 21 08:17:09.850442 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 08:17:09.850442 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 08:17:09.850442 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 08:17:09.850442 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application return self.creator() Jul 21 08:17:09.850442 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 08:17:09.850442 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application created_value = creator( Jul 21 08:17:09.851470 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 21 08:17:09.851470 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 21 08:17:09.851470 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 21 08:17:09.851470 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 21 08:17:09.851470 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 21 08:17:09.851470 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 21 08:17:09.851470 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 21 08:17:09.851470 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application Jul 21 08:17:09.853550 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: [pid: 80245|app: 0|req: 273/543] 10.222.0.47 () {56 vars in 1111 bytes} [Thu Jul 21 08:17:09 2022] GET /identity/v3/projects/service => generated 88 bytes in 393 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 08:17:09.864505 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7f377cf-e50e-45a9-b512-e060e0972863 None admin] Authenticating user token {{(pid=80245) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:17:09.938338 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62850600-a5b7-4383-b90d-097a5191b132 None admin] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:17:09.958046 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG dbcounter [-] [80244] Writing DB stats keystone:SELECT=100 {{(pid=80244) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:17:09.991953 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e6aaa2d-c348-4be6-ab73-638804440dd1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:17:09.992753 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e6aaa2d-c348-4be6-ab73-638804440dd1 None admin] REQUEST_METHOD: `GET` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:17:09.993032 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e6aaa2d-c348-4be6-ab73-638804440dd1 None admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:17:09.993305 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e6aaa2d-c348-4be6-ab73-638804440dd1 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:17:10.018170 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e6aaa2d-c348-4be6-ab73-638804440dd1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:17:10.024765 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e6aaa2d-c348-4be6-ab73-638804440dd1 None admin] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:17:10.036811 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7f377cf-e50e-45a9-b512-e060e0972863 None admin] Validating token access rules against request {{(pid=80245) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:17:10.054884 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application [None req-9e6aaa2d-c348-4be6-ab73-638804440dd1 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 08:17:10.054884 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 08:17:10.054884 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 08:17:10.054884 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 08:17:10.054884 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 08:17:10.054884 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 08:17:10.054884 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 08:17:10.054884 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 08:17:10.054884 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 08:17:10.054884 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 08:17:10.054884 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 08:17:10.054884 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 08:17:10.054884 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 21 08:17:10.054884 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 21 08:17:10.054884 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 21 08:17:10.055762 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 21 08:17:10.055762 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 08:17:10.055762 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 08:17:10.055762 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 08:17:10.055762 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 08:17:10.055762 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 08:17:10.055762 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 08:17:10.055762 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 08:17:10.055762 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application with Lock( Jul 21 08:17:10.055762 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 08:17:10.055762 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self._enter() Jul 21 08:17:10.055762 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 08:17:10.055762 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 08:17:10.055762 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 08:17:10.055762 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self.creator() Jul 21 08:17:10.055762 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 08:17:10.055762 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application created_value = creator( Jul 21 08:17:10.056568 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 21 08:17:10.056568 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 21 08:17:10.056568 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 08:17:10.056568 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application Jul 21 08:17:10.068662 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 271/544] 10.222.0.47 () {56 vars in 1108 bytes} [Thu Jul 21 08:17:09 2022] GET /identity/v3/domains/Default => generated 87 bytes in 275 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 08:17:10.075066 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e6aaa2d-c348-4be6-ab73-638804440dd1 None admin] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:17:10.076865 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74c33e80-8c39-42cb-aed8-d6dae2ed1cd5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80245) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:17:10.077550 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74c33e80-8c39-42cb-aed8-d6dae2ed1cd5 None admin] REQUEST_METHOD: `GET` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:17:10.077828 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74c33e80-8c39-42cb-aed8-d6dae2ed1cd5 None admin] SCRIPT_NAME: `/identity` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:17:10.078100 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74c33e80-8c39-42cb-aed8-d6dae2ed1cd5 None admin] PATH_INFO: `/v3/projects` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:17:10.078887 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74c33e80-8c39-42cb-aed8-d6dae2ed1cd5 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80245) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 08:17:10.079196 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74c33e80-8c39-42cb-aed8-d6dae2ed1cd5 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:17:10.080685 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74c33e80-8c39-42cb-aed8-d6dae2ed1cd5 None admin] RBAC: Authorization granted {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:17:10.090257 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG dbcounter [-] [80245] Writing DB stats keystone:SELECT=100 {{(pid=80245) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:17:10.094927 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: [pid: 80245|app: 0|req: 274/545] 10.222.0.47 () {56 vars in 1148 bytes} [Thu Jul 21 08:17:09 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 231 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 08:17:10.101052 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74c33e80-8c39-42cb-aed8-d6dae2ed1cd5 None admin] Authenticating user token {{(pid=80245) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:17:10.200034 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e6aaa2d-c348-4be6-ab73-638804440dd1 None admin] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:17:10.239167 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94f4df70-9d8f-4dcb-aaab-77fbe0089153 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:17:10.239865 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94f4df70-9d8f-4dcb-aaab-77fbe0089153 None admin] REQUEST_METHOD: `GET` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:17:10.240160 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94f4df70-9d8f-4dcb-aaab-77fbe0089153 None admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:17:10.240446 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94f4df70-9d8f-4dcb-aaab-77fbe0089153 None admin] PATH_INFO: `/v3/domains` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:17:10.241834 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94f4df70-9d8f-4dcb-aaab-77fbe0089153 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80244) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 08:17:10.242156 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94f4df70-9d8f-4dcb-aaab-77fbe0089153 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:17:10.243667 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94f4df70-9d8f-4dcb-aaab-77fbe0089153 None admin] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:17:10.258755 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 272/546] 10.222.0.47 () {56 vars in 1109 bytes} [Thu Jul 21 08:17:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 21 08:17:10.264097 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94f4df70-9d8f-4dcb-aaab-77fbe0089153 None admin] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:17:10.344493 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG dbcounter [-] [80244] Writing DB stats keystone:SELECT=100 {{(pid=80244) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:17:10.378947 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94f4df70-9d8f-4dcb-aaab-77fbe0089153 None admin] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:17:10.417723 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2c52222-0138-44a3-98d7-90ca6eebba9e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:17:10.418637 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2c52222-0138-44a3-98d7-90ca6eebba9e None admin] REQUEST_METHOD: `GET` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:17:10.418996 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2c52222-0138-44a3-98d7-90ca6eebba9e None admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:17:10.419426 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2c52222-0138-44a3-98d7-90ca6eebba9e None admin] PATH_INFO: `/v3/users/swift` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:17:10.428338 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2c52222-0138-44a3-98d7-90ca6eebba9e None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:17:10.430236 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2c52222-0138-44a3-98d7-90ca6eebba9e None admin] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:17:10.439185 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74c33e80-8c39-42cb-aed8-d6dae2ed1cd5 None admin] Validating token access rules against request {{(pid=80245) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:17:10.451181 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application [None req-f2c52222-0138-44a3-98d7-90ca6eebba9e None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Jul 21 08:17:10.451181 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 08:17:10.451181 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 08:17:10.451181 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 08:17:10.451181 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 08:17:10.451181 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 08:17:10.451181 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 08:17:10.451181 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 08:17:10.451181 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 08:17:10.451181 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 08:17:10.451181 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 08:17:10.451181 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 08:17:10.451181 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 21 08:17:10.451181 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 21 08:17:10.451181 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 21 08:17:10.451181 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 21 08:17:10.452084 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 08:17:10.452084 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 08:17:10.452084 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 21 08:17:10.452084 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 21 08:17:10.452084 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 21 08:17:10.452084 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 21 08:17:10.452084 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 08:17:10.452084 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 08:17:10.452084 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 08:17:10.452084 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 08:17:10.452084 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 08:17:10.452084 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application with Lock( Jul 21 08:17:10.452084 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 08:17:10.452084 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self._enter() Jul 21 08:17:10.452084 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 08:17:10.452084 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 08:17:10.452084 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 08:17:10.452084 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self.creator() Jul 21 08:17:10.453482 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 08:17:10.453482 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application created_value = creator( Jul 21 08:17:10.453482 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 21 08:17:10.453482 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 21 08:17:10.453482 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 21 08:17:10.453482 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 21 08:17:10.453482 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 21 08:17:10.453482 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 21 08:17:10.453482 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Jul 21 08:17:10.453482 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application Jul 21 08:17:10.453482 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 273/547] 10.222.0.47 () {56 vars in 1096 bytes} [Thu Jul 21 08:17:10 2022] GET /identity/v3/users/swift => generated 83 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 21 08:17:10.459348 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dbe27bd-1530-4c20-9f4d-0b29ea187d90 None None] REQUEST_METHOD: `GET` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:17:10.459348 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dbe27bd-1530-4c20-9f4d-0b29ea187d90 None None] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:17:10.459348 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dbe27bd-1530-4c20-9f4d-0b29ea187d90 None None] PATH_INFO: `/` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:17:10.459348 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 274/548] 10.222.0.47 () {54 vars in 926 bytes} [Thu Jul 21 08:17:10 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 08:17:10.459348 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dbe27bd-1530-4c20-9f4d-0b29ea187d90 None None] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:17:10.483381 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8593902-e172-4c46-9f44-4c299dfcebda None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80245) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:17:10.483381 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8593902-e172-4c46-9f44-4c299dfcebda None admin] REQUEST_METHOD: `GET` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:17:10.483381 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8593902-e172-4c46-9f44-4c299dfcebda None admin] SCRIPT_NAME: `/identity` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:17:10.483381 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8593902-e172-4c46-9f44-4c299dfcebda None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:17:10.483775 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8593902-e172-4c46-9f44-4c299dfcebda None admin] RBAC: Adding query filter params (role.id=50d1ec7f947f42acb65c57e07f2269d6, scope.project.id=4faef13aa50e4e36b6c265fb1398fa14, user.id=20ca3c3bd3774691b3374cfbfec6e494) {{(pid=80245) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 08:17:10.483775 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8593902-e172-4c46-9f44-4c299dfcebda None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:17:10.487105 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8593902-e172-4c46-9f44-4c299dfcebda None admin] RBAC: Authorization granted {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:17:10.494461 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: [pid: 80245|app: 0|req: 275/549] 10.222.0.47 () {56 vars in 1374 bytes} [Thu Jul 21 08:17:10 2022] GET /identity/v3/role_assignments?user.id=20ca3c3bd3774691b3374cfbfec6e494&scope.project.id=4faef13aa50e4e36b6c265fb1398fa14&role.id=50d1ec7f947f42acb65c57e07f2269d6 => generated 259 bytes in 394 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 08:17:10.501036 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f115eb4-9b5e-4c0e-93b5-9e7954767d3b None None] REQUEST_METHOD: `POST` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:17:10.501633 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f115eb4-9b5e-4c0e-93b5-9e7954767d3b None None] SCRIPT_NAME: `/identity` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:17:10.502113 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f115eb4-9b5e-4c0e-93b5-9e7954767d3b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:17:10.534678 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.auth.core [None req-2f115eb4-9b5e-4c0e-93b5-9e7954767d3b None None] MFA Rules not processed for user `2dce40b8d74c41ec94cc36e6ff05d412`. Rule list: `[]` (Enabled: `True`). {{(pid=80245) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 08:17:10.536793 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.fernet_utils [None req-2f115eb4-9b5e-4c0e-93b5-9e7954767d3b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80245) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 08:17:10.589316 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG dbcounter [-] [80244] Writing DB stats keystone:SELECT=100 {{(pid=80244) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:17:10.619271 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dbe27bd-1530-4c20-9f4d-0b29ea187d90 None None] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:17:10.675933 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG dbcounter [-] [80245] Writing DB stats keystone:SELECT=100 {{(pid=80245) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:17:10.694699 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: [pid: 80245|app: 0|req: 276/550] 10.222.0.47 () {58 vars in 1023 bytes} [Thu Jul 21 08:17:10 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 194 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 21 08:17:10.698484 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f141bd62-c208-4f0d-8e43-2f8a12426df4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:17:10.699431 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f141bd62-c208-4f0d-8e43-2f8a12426df4 None admin] REQUEST_METHOD: `GET` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:17:10.699940 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f141bd62-c208-4f0d-8e43-2f8a12426df4 None admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:17:10.700434 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f141bd62-c208-4f0d-8e43-2f8a12426df4 None admin] PATH_INFO: `/v3/users` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:17:10.701538 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f141bd62-c208-4f0d-8e43-2f8a12426df4 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80244) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 08:17:10.702074 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f141bd62-c208-4f0d-8e43-2f8a12426df4 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:17:10.703862 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f141bd62-c208-4f0d-8e43-2f8a12426df4 None admin] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:17:10.719357 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f115eb4-9b5e-4c0e-93b5-9e7954767d3b None None] Authenticating user token {{(pid=80245) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:17:10.726203 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 275/551] 10.222.0.47 () {56 vars in 1135 bytes} [Thu Jul 21 08:17:10 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 267 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 08:17:10.731969 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f141bd62-c208-4f0d-8e43-2f8a12426df4 None admin] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:17:10.839939 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f115eb4-9b5e-4c0e-93b5-9e7954767d3b None None] Validating token access rules against request {{(pid=80245) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:17:10.845585 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f141bd62-c208-4f0d-8e43-2f8a12426df4 None admin] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:17:10.880494 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39e116b1-cd2c-415c-87fb-b5a8104c9592 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80245) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:17:10.881931 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39e116b1-cd2c-415c-87fb-b5a8104c9592 None admin] REQUEST_METHOD: `GET` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:17:10.882427 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39e116b1-cd2c-415c-87fb-b5a8104c9592 None admin] SCRIPT_NAME: `/identity` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:17:10.882888 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39e116b1-cd2c-415c-87fb-b5a8104c9592 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:17:10.890313 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae2b6c70-4a3e-4ed0-95c5-8c65507877ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:17:10.890981 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae2b6c70-4a3e-4ed0-95c5-8c65507877ae None admin] REQUEST_METHOD: `GET` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:17:10.891569 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae2b6c70-4a3e-4ed0-95c5-8c65507877ae None admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:17:10.891853 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae2b6c70-4a3e-4ed0-95c5-8c65507877ae None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:17:10.895941 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39e116b1-cd2c-415c-87fb-b5a8104c9592 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 21 08:17:10.907496 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application [None req-39e116b1-cd2c-415c-87fb-b5a8104c9592 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 21 08:17:10.907496 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 08:17:10.907496 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 08:17:10.907496 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 08:17:10.907496 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 08:17:10.907496 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 08:17:10.907496 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 08:17:10.907496 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 08:17:10.907496 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 08:17:10.907496 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 08:17:10.907496 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 08:17:10.907496 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 08:17:10.907496 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 21 08:17:10.907496 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 21 08:17:10.907496 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 21 08:17:10.907496 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 21 08:17:10.908581 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 21 08:17:10.908581 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 21 08:17:10.908581 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 21 08:17:10.908581 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 21 08:17:10.908581 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 08:17:10.908581 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 08:17:10.908581 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 08:17:10.908581 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 08:17:10.908581 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 08:17:10.908581 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 08:17:10.908581 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 08:17:10.908581 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application with Lock( Jul 21 08:17:10.908581 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 08:17:10.908581 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application return self._enter() Jul 21 08:17:10.908581 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 08:17:10.908581 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 08:17:10.909949 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 08:17:10.909949 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application return self.creator() Jul 21 08:17:10.909949 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 08:17:10.909949 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application created_value = creator( Jul 21 08:17:10.909949 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 21 08:17:10.909949 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 21 08:17:10.909949 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 21 08:17:10.909949 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 21 08:17:10.909949 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 21 08:17:10.909949 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 21 08:17:10.909949 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 21 08:17:10.909949 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application Jul 21 08:17:10.909949 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: [pid: 80245|app: 0|req: 277/552] 10.222.0.47 () {56 vars in 1102 bytes} [Thu Jul 21 08:17:10 2022] GET /identity/v3/roles/service => generated 85 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 21 08:17:10.915881 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc90c569-1597-45d9-be93-7e95fbc35688 None None] REQUEST_METHOD: `GET` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:17:10.915881 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc90c569-1597-45d9-be93-7e95fbc35688 None None] SCRIPT_NAME: `/identity` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:17:10.915881 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc90c569-1597-45d9-be93-7e95fbc35688 None None] PATH_INFO: `/` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:17:10.916503 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG dbcounter [-] [80244] Writing DB stats keystone:SELECT=100 {{(pid=80244) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:17:10.916604 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: [pid: 80245|app: 0|req: 278/553] 10.222.0.47 () {54 vars in 926 bytes} [Thu Jul 21 08:17:10 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 08:17:10.919112 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc90c569-1597-45d9-be93-7e95fbc35688 None None] Authenticating user token {{(pid=80245) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:17:10.982726 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae2b6c70-4a3e-4ed0-95c5-8c65507877ae None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:17:10.984929 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae2b6c70-4a3e-4ed0-95c5-8c65507877ae None admin] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:17:10.986128 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG dbcounter [-] [80245] Writing DB stats keystone:SELECT=100 {{(pid=80245) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:17:11.038144 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc90c569-1597-45d9-be93-7e95fbc35688 None None] Validating token access rules against request {{(pid=80245) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:17:11.083580 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aeeac17-10cf-4227-8d62-f07656176d19 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80245) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:17:11.084268 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aeeac17-10cf-4227-8d62-f07656176d19 None admin] REQUEST_METHOD: `GET` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:17:11.084560 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aeeac17-10cf-4227-8d62-f07656176d19 None admin] SCRIPT_NAME: `/identity` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:17:11.084843 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aeeac17-10cf-4227-8d62-f07656176d19 None admin] PATH_INFO: `/v3/roles` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:17:11.085864 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7aeeac17-10cf-4227-8d62-f07656176d19 None admin] RBAC: Adding query filter params (name=service) {{(pid=80245) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 08:17:11.086179 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7aeeac17-10cf-4227-8d62-f07656176d19 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:17:11.088414 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7aeeac17-10cf-4227-8d62-f07656176d19 None admin] RBAC: Authorization granted {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:17:11.100415 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: [pid: 80245|app: 0|req: 279/554] 10.222.0.47 () {56 vars in 1103 bytes} [Thu Jul 21 08:17:10 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 08:17:11.102162 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50db022f-e4a2-499b-b891-19eff8d407ef None None] REQUEST_METHOD: `POST` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:17:11.102479 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50db022f-e4a2-499b-b891-19eff8d407ef None None] SCRIPT_NAME: `/identity` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:17:11.103007 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50db022f-e4a2-499b-b891-19eff8d407ef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:17:11.109471 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 276/555] 10.222.0.47 () {58 vars in 1282 bytes} [Thu Jul 21 08:17:10 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 378 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Jul 21 08:17:11.110683 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae2b6c70-4a3e-4ed0-95c5-8c65507877ae None admin] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:17:11.122931 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.auth.core [None req-50db022f-e4a2-499b-b891-19eff8d407ef None None] MFA Rules not processed for user `2dce40b8d74c41ec94cc36e6ff05d412`. Rule list: `[]` (Enabled: `True`). {{(pid=80245) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 08:17:11.124641 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.fernet_utils [None req-50db022f-e4a2-499b-b891-19eff8d407ef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80245) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 08:17:11.175141 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG dbcounter [-] [80244] Writing DB stats keystone:SELECT=100 {{(pid=80244) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:17:11.219662 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: [pid: 80245|app: 0|req: 280/556] 10.222.0.47 () {58 vars in 1023 bytes} [Thu Jul 21 08:17:11 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 21 08:17:11.221030 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50db022f-e4a2-499b-b891-19eff8d407ef None None] Authenticating user token {{(pid=80245) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:17:11.237262 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae2b6c70-4a3e-4ed0-95c5-8c65507877ae None admin] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:17:11.278962 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f0009b4-1f30-487c-8a21-d824f47e5c62 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:17:11.279873 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f0009b4-1f30-487c-8a21-d824f47e5c62 None admin] REQUEST_METHOD: `GET` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:17:11.280160 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f0009b4-1f30-487c-8a21-d824f47e5c62 None admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:17:11.280439 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f0009b4-1f30-487c-8a21-d824f47e5c62 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:17:11.292778 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f0009b4-1f30-487c-8a21-d824f47e5c62 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:17:11.294014 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f0009b4-1f30-487c-8a21-d824f47e5c62 None admin] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:17:11.307939 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG dbcounter [-] [80245] Writing DB stats keystone:SELECT=100 {{(pid=80245) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:17:11.312764 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application [None req-1f0009b4-1f30-487c-8a21-d824f47e5c62 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 08:17:11.312764 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 08:17:11.312764 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 08:17:11.312764 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 08:17:11.312764 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 08:17:11.312764 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 08:17:11.312764 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 08:17:11.312764 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 08:17:11.312764 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 08:17:11.312764 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 08:17:11.312764 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 08:17:11.312764 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 08:17:11.312764 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 21 08:17:11.312764 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 21 08:17:11.312764 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 21 08:17:11.313740 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 21 08:17:11.313740 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 08:17:11.313740 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 08:17:11.313740 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 08:17:11.313740 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 08:17:11.313740 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 08:17:11.313740 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 08:17:11.313740 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 08:17:11.313740 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application with Lock( Jul 21 08:17:11.313740 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 08:17:11.313740 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self._enter() Jul 21 08:17:11.313740 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 08:17:11.313740 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 08:17:11.313740 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 08:17:11.313740 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self.creator() Jul 21 08:17:11.313740 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 08:17:11.313740 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application created_value = creator( Jul 21 08:17:11.315230 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 21 08:17:11.315230 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 21 08:17:11.315230 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 08:17:11.315230 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application Jul 21 08:17:11.315230 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 277/557] 10.222.0.47 () {56 vars in 1108 bytes} [Thu Jul 21 08:17:11 2022] GET /identity/v3/domains/Default => generated 87 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 08:17:11.315230 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f0009b4-1f30-487c-8a21-d824f47e5c62 None admin] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:17:11.362996 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50db022f-e4a2-499b-b891-19eff8d407ef None None] Validating token access rules against request {{(pid=80245) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:17:11.403090 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-762ccf2b-0154-4328-8488-7ca4a400b8e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80245) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:17:11.404448 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-762ccf2b-0154-4328-8488-7ca4a400b8e6 None admin] REQUEST_METHOD: `GET` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:17:11.404952 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-762ccf2b-0154-4328-8488-7ca4a400b8e6 None admin] SCRIPT_NAME: `/identity` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:17:11.405488 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-762ccf2b-0154-4328-8488-7ca4a400b8e6 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:17:11.417186 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-762ccf2b-0154-4328-8488-7ca4a400b8e6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:17:11.419059 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-762ccf2b-0154-4328-8488-7ca4a400b8e6 None admin] RBAC: Authorization granted {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:17:11.420326 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG dbcounter [-] [80244] Writing DB stats keystone:SELECT=100 {{(pid=80244) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:17:11.424064 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f0009b4-1f30-487c-8a21-d824f47e5c62 None admin] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:17:11.441511 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application [None req-762ccf2b-0154-4328-8488-7ca4a400b8e6 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 08:17:11.441511 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 08:17:11.441511 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 08:17:11.441511 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 08:17:11.441511 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 08:17:11.441511 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 08:17:11.441511 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 08:17:11.441511 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 08:17:11.441511 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 08:17:11.441511 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 08:17:11.441511 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 08:17:11.441511 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 08:17:11.441511 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 21 08:17:11.441511 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 21 08:17:11.441511 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 21 08:17:11.443517 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 21 08:17:11.443517 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 08:17:11.443517 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 08:17:11.443517 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 08:17:11.443517 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 08:17:11.443517 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 08:17:11.443517 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 08:17:11.443517 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 08:17:11.443517 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application with Lock( Jul 21 08:17:11.443517 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 08:17:11.443517 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application return self._enter() Jul 21 08:17:11.443517 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 08:17:11.443517 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 08:17:11.443517 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 08:17:11.443517 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application return self.creator() Jul 21 08:17:11.443517 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 08:17:11.443517 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application created_value = creator( Jul 21 08:17:11.444587 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 21 08:17:11.444587 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 21 08:17:11.444587 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 08:17:11.444587 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application Jul 21 08:17:11.446604 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: [pid: 80245|app: 0|req: 281/558] 10.222.0.47 () {56 vars in 1108 bytes} [Thu Jul 21 08:17:11 2022] GET /identity/v3/domains/Default => generated 87 bytes in 226 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 08:17:11.447670 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-762ccf2b-0154-4328-8488-7ca4a400b8e6 None admin] Authenticating user token {{(pid=80245) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:17:11.462545 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd9c4fc2-7646-4ff8-a836-caef6e57bb61 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:17:11.463600 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd9c4fc2-7646-4ff8-a836-caef6e57bb61 None admin] REQUEST_METHOD: `GET` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:17:11.464008 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd9c4fc2-7646-4ff8-a836-caef6e57bb61 None admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:17:11.464406 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd9c4fc2-7646-4ff8-a836-caef6e57bb61 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:17:11.476648 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd9c4fc2-7646-4ff8-a836-caef6e57bb61 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 21 08:17:11.484950 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application [None req-bd9c4fc2-7646-4ff8-a836-caef6e57bb61 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 21 08:17:11.484950 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 08:17:11.484950 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 08:17:11.484950 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 08:17:11.484950 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 08:17:11.484950 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 08:17:11.484950 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 08:17:11.484950 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 08:17:11.484950 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 08:17:11.484950 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 08:17:11.484950 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 08:17:11.484950 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 08:17:11.484950 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 21 08:17:11.484950 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 21 08:17:11.484950 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 21 08:17:11.484950 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 21 08:17:11.486024 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 21 08:17:11.486024 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 21 08:17:11.486024 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 21 08:17:11.486024 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 21 08:17:11.486024 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 08:17:11.486024 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 08:17:11.486024 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 08:17:11.486024 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 08:17:11.486024 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 08:17:11.486024 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 08:17:11.486024 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 08:17:11.486024 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application with Lock( Jul 21 08:17:11.486024 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 08:17:11.486024 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self._enter() Jul 21 08:17:11.486024 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 08:17:11.486024 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 08:17:11.486826 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 08:17:11.486826 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self.creator() Jul 21 08:17:11.486826 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 08:17:11.486826 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application created_value = creator( Jul 21 08:17:11.486826 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 21 08:17:11.486826 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 21 08:17:11.486826 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 21 08:17:11.486826 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 21 08:17:11.486826 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 21 08:17:11.486826 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 21 08:17:11.486826 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 21 08:17:11.486826 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application Jul 21 08:17:11.487594 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 278/559] 10.222.0.47 () {56 vars in 1102 bytes} [Thu Jul 21 08:17:11 2022] GET /identity/v3/roles/service => generated 85 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 08:17:11.488475 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd9c4fc2-7646-4ff8-a836-caef6e57bb61 None admin] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:17:11.566424 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG dbcounter [-] [80245] Writing DB stats keystone:SELECT=100 {{(pid=80245) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:17:11.574909 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-762ccf2b-0154-4328-8488-7ca4a400b8e6 None admin] Validating token access rules against request {{(pid=80245) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:17:11.612680 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce9902c3-9ee3-408a-bb19-79719413e564 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80245) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:17:11.614397 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce9902c3-9ee3-408a-bb19-79719413e564 None admin] REQUEST_METHOD: `GET` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:17:11.615181 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce9902c3-9ee3-408a-bb19-79719413e564 None admin] SCRIPT_NAME: `/identity` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:17:11.615936 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce9902c3-9ee3-408a-bb19-79719413e564 None admin] PATH_INFO: `/v3/domains` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:17:11.616846 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce9902c3-9ee3-408a-bb19-79719413e564 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80245) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 08:17:11.617335 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce9902c3-9ee3-408a-bb19-79719413e564 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:17:11.617879 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd9c4fc2-7646-4ff8-a836-caef6e57bb61 None admin] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:17:11.619154 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce9902c3-9ee3-408a-bb19-79719413e564 None admin] RBAC: Authorization granted {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:17:11.636202 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: [pid: 80245|app: 0|req: 282/560] 10.222.0.47 () {56 vars in 1109 bytes} [Thu Jul 21 08:17:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 08:17:11.637401 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce9902c3-9ee3-408a-bb19-79719413e564 None admin] Authenticating user token {{(pid=80245) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:17:11.658354 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c22c2814-7af4-4f0c-8507-9742f3b0989b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:17:11.659284 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c22c2814-7af4-4f0c-8507-9742f3b0989b None admin] REQUEST_METHOD: `GET` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:17:11.659593 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c22c2814-7af4-4f0c-8507-9742f3b0989b None admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:17:11.659868 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c22c2814-7af4-4f0c-8507-9742f3b0989b None admin] PATH_INFO: `/v3/domains` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:17:11.660586 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c22c2814-7af4-4f0c-8507-9742f3b0989b None admin] RBAC: Adding query filter params (name=Default) {{(pid=80244) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 08:17:11.661471 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c22c2814-7af4-4f0c-8507-9742f3b0989b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:17:11.662838 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c22c2814-7af4-4f0c-8507-9742f3b0989b None admin] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:17:11.675348 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 279/561] 10.222.0.47 () {56 vars in 1109 bytes} [Thu Jul 21 08:17:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 08:17:11.681873 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc15160f-e15e-42f9-ae6f-8752b5af25c7 None None] REQUEST_METHOD: `GET` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:17:11.682025 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc15160f-e15e-42f9-ae6f-8752b5af25c7 None None] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:17:11.682273 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc15160f-e15e-42f9-ae6f-8752b5af25c7 None None] PATH_INFO: `/` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:17:11.683635 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 280/562] 10.222.0.47 () {54 vars in 926 bytes} [Thu Jul 21 08:17:11 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 08:17:11.687539 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc15160f-e15e-42f9-ae6f-8752b5af25c7 None None] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:17:11.708786 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG dbcounter [-] [80244] Writing DB stats keystone:SELECT=100 {{(pid=80244) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:17:11.749219 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce9902c3-9ee3-408a-bb19-79719413e564 None admin] Validating token access rules against request {{(pid=80245) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:17:11.786705 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a674e6b-643c-4e76-a7c7-1da315355c35 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80245) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:17:11.787720 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a674e6b-643c-4e76-a7c7-1da315355c35 None admin] REQUEST_METHOD: `GET` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:17:11.788010 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a674e6b-643c-4e76-a7c7-1da315355c35 None admin] SCRIPT_NAME: `/identity` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:17:11.788287 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a674e6b-643c-4e76-a7c7-1da315355c35 None admin] PATH_INFO: `/v3/roles` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:17:11.789032 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a674e6b-643c-4e76-a7c7-1da315355c35 None admin] RBAC: Adding query filter params (name=service) {{(pid=80245) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 08:17:11.789338 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a674e6b-643c-4e76-a7c7-1da315355c35 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:17:11.790698 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a674e6b-643c-4e76-a7c7-1da315355c35 None admin] RBAC: Authorization granted {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:17:11.803092 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: [pid: 80245|app: 0|req: 283/563] 10.222.0.47 () {56 vars in 1103 bytes} [Thu Jul 21 08:17:11 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 08:17:11.803092 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a674e6b-643c-4e76-a7c7-1da315355c35 None admin] Authenticating user token {{(pid=80245) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:17:11.808580 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc15160f-e15e-42f9-ae6f-8752b5af25c7 None None] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:17:11.822517 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG dbcounter [-] [80245] Writing DB stats keystone:SELECT=100 {{(pid=80245) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:17:11.851972 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6627e198-fb1b-4cfa-bab9-976112705aa8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:17:11.852814 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6627e198-fb1b-4cfa-bab9-976112705aa8 None admin] REQUEST_METHOD: `GET` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:17:11.853148 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6627e198-fb1b-4cfa-bab9-976112705aa8 None admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:17:11.853426 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6627e198-fb1b-4cfa-bab9-976112705aa8 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:17:11.862066 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6627e198-fb1b-4cfa-bab9-976112705aa8 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:17:11.864486 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6627e198-fb1b-4cfa-bab9-976112705aa8 None admin] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:17:11.884049 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application [None req-6627e198-fb1b-4cfa-bab9-976112705aa8 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jul 21 08:17:11.884049 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 08:17:11.884049 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 08:17:11.884049 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 08:17:11.884049 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 08:17:11.884049 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 08:17:11.884049 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 08:17:11.884049 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 08:17:11.884049 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 08:17:11.884049 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 08:17:11.884049 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 08:17:11.884049 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 08:17:11.884049 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 21 08:17:11.884049 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 21 08:17:11.884049 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 21 08:17:11.884049 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 21 08:17:11.885066 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 08:17:11.885066 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 08:17:11.885066 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 21 08:17:11.885066 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 21 08:17:11.885066 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 21 08:17:11.885066 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 21 08:17:11.885066 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 08:17:11.885066 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 08:17:11.885066 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 08:17:11.885066 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 08:17:11.885066 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 08:17:11.885066 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application with Lock( Jul 21 08:17:11.885066 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 08:17:11.885066 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self._enter() Jul 21 08:17:11.885066 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 08:17:11.885066 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 08:17:11.885066 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 08:17:11.885066 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self.creator() Jul 21 08:17:11.885989 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 08:17:11.885989 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application created_value = creator( Jul 21 08:17:11.885989 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 21 08:17:11.885989 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 21 08:17:11.885989 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 21 08:17:11.885989 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 21 08:17:11.885989 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 21 08:17:11.885989 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 21 08:17:11.885989 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jul 21 08:17:11.885989 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application Jul 21 08:17:11.885989 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 281/564] 10.222.0.47 () {56 vars in 1093 bytes} [Thu Jul 21 08:17:11 2022] GET /identity/v3/users/nova => generated 82 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 08:17:11.887229 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89342c7e-2ca6-476c-9aef-a22136a49378 None None] REQUEST_METHOD: `POST` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:17:11.887229 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89342c7e-2ca6-476c-9aef-a22136a49378 None None] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:17:11.887513 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89342c7e-2ca6-476c-9aef-a22136a49378 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:17:11.905775 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.auth.core [None req-89342c7e-2ca6-476c-9aef-a22136a49378 None None] MFA Rules not processed for user `2dce40b8d74c41ec94cc36e6ff05d412`. Rule list: `[]` (Enabled: `True`). {{(pid=80244) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 08:17:11.906982 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.fernet_utils [None req-89342c7e-2ca6-476c-9aef-a22136a49378 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80244) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 08:17:11.913706 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a674e6b-643c-4e76-a7c7-1da315355c35 None admin] Validating token access rules against request {{(pid=80245) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:17:11.951516 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c756c536-489b-4bc1-bd7e-8c0a9dd04254 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80245) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:17:11.952418 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c756c536-489b-4bc1-bd7e-8c0a9dd04254 None admin] REQUEST_METHOD: `GET` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:17:11.952772 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c756c536-489b-4bc1-bd7e-8c0a9dd04254 None admin] SCRIPT_NAME: `/identity` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:17:11.953058 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c756c536-489b-4bc1-bd7e-8c0a9dd04254 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:17:11.960516 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c756c536-489b-4bc1-bd7e-8c0a9dd04254 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:17:11.961964 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c756c536-489b-4bc1-bd7e-8c0a9dd04254 None admin] RBAC: Authorization granted {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:17:11.976943 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 282/565] 10.222.0.47 () {58 vars in 1023 bytes} [Thu Jul 21 08:17:11 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 21 08:17:11.977708 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application [None req-c756c536-489b-4bc1-bd7e-8c0a9dd04254 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 21 08:17:11.977708 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 08:17:11.977708 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 08:17:11.977708 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 08:17:11.977708 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 08:17:11.977708 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 08:17:11.977708 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 08:17:11.977708 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 08:17:11.977708 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 08:17:11.977708 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 08:17:11.977708 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 08:17:11.977708 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 08:17:11.977708 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 21 08:17:11.977708 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 21 08:17:11.977708 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 21 08:17:11.978592 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 21 08:17:11.978592 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 08:17:11.978592 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 08:17:11.978592 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 08:17:11.978592 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 08:17:11.978592 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 08:17:11.978592 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 08:17:11.978592 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 08:17:11.978592 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application with Lock( Jul 21 08:17:11.978592 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 08:17:11.978592 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application return self._enter() Jul 21 08:17:11.978592 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 08:17:11.978592 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 08:17:11.978592 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 08:17:11.978592 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application return self.creator() Jul 21 08:17:11.978592 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 08:17:11.978592 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application created_value = creator( Jul 21 08:17:11.979521 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 21 08:17:11.979521 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 21 08:17:11.979521 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 21 08:17:11.979521 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 21 08:17:11.979521 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 21 08:17:11.979521 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 21 08:17:11.979521 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 21 08:17:11.979521 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application Jul 21 08:17:11.979521 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: [pid: 80245|app: 0|req: 284/566] 10.222.0.47 () {56 vars in 1111 bytes} [Thu Jul 21 08:17:11 2022] GET /identity/v3/projects/service => generated 88 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 08:17:11.979990 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c756c536-489b-4bc1-bd7e-8c0a9dd04254 None admin] Authenticating user token {{(pid=80245) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:17:11.984059 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89342c7e-2ca6-476c-9aef-a22136a49378 None None] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:17:11.995275 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG dbcounter [-] [80244] Writing DB stats keystone:SELECT=100 {{(pid=80244) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:17:12.074420 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG dbcounter [-] [80245] Writing DB stats keystone:SELECT=100 {{(pid=80245) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:17:12.094855 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89342c7e-2ca6-476c-9aef-a22136a49378 None None] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:17:12.095521 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c756c536-489b-4bc1-bd7e-8c0a9dd04254 None admin] Validating token access rules against request {{(pid=80245) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:17:12.130180 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-724f9009-eda7-4d8f-b3d2-97ec83fbd375 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:17:12.130388 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3163987-c265-4fef-bc73-8b559d1caaff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80245) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:17:12.130976 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3163987-c265-4fef-bc73-8b559d1caaff None admin] REQUEST_METHOD: `GET` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:17:12.131077 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-724f9009-eda7-4d8f-b3d2-97ec83fbd375 None admin] REQUEST_METHOD: `GET` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:17:12.131248 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3163987-c265-4fef-bc73-8b559d1caaff None admin] SCRIPT_NAME: `/identity` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:17:12.131437 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-724f9009-eda7-4d8f-b3d2-97ec83fbd375 None admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:17:12.131583 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3163987-c265-4fef-bc73-8b559d1caaff None admin] PATH_INFO: `/v3/users` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:17:12.131686 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-724f9009-eda7-4d8f-b3d2-97ec83fbd375 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:17:12.132381 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3163987-c265-4fef-bc73-8b559d1caaff None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80245) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 08:17:12.132684 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3163987-c265-4fef-bc73-8b559d1caaff None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:17:12.134750 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3163987-c265-4fef-bc73-8b559d1caaff None admin] RBAC: Authorization granted {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:17:12.143202 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-724f9009-eda7-4d8f-b3d2-97ec83fbd375 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:17:12.144690 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-724f9009-eda7-4d8f-b3d2-97ec83fbd375 None admin] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:17:12.163812 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application [None req-724f9009-eda7-4d8f-b3d2-97ec83fbd375 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 08:17:12.163812 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 08:17:12.163812 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 08:17:12.163812 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 08:17:12.163812 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 08:17:12.163812 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 08:17:12.163812 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 08:17:12.163812 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 08:17:12.163812 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 08:17:12.163812 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 08:17:12.163812 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 08:17:12.163812 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 08:17:12.163812 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 21 08:17:12.163812 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 21 08:17:12.163812 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 21 08:17:12.164959 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 21 08:17:12.164959 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 08:17:12.164959 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 08:17:12.164959 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 08:17:12.164959 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 08:17:12.164959 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 08:17:12.164959 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 08:17:12.164959 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 08:17:12.164959 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application with Lock( Jul 21 08:17:12.164959 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 08:17:12.164959 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self._enter() Jul 21 08:17:12.164959 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 08:17:12.164959 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 08:17:12.164959 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 08:17:12.164959 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self.creator() Jul 21 08:17:12.164959 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 08:17:12.164959 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application created_value = creator( Jul 21 08:17:12.165828 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 21 08:17:12.165828 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 21 08:17:12.165828 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 08:17:12.165828 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application Jul 21 08:17:12.167636 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 283/567] 10.222.0.47 () {56 vars in 1108 bytes} [Thu Jul 21 08:17:11 2022] GET /identity/v3/domains/Default => generated 87 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 08:17:12.168699 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-724f9009-eda7-4d8f-b3d2-97ec83fbd375 None admin] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:17:12.173854 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: [pid: 80245|app: 0|req: 285/568] 10.222.0.47 () {56 vars in 1133 bytes} [Thu Jul 21 08:17:11 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 08:17:12.174199 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3163987-c265-4fef-bc73-8b559d1caaff None admin] Authenticating user token {{(pid=80245) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:17:12.234543 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG dbcounter [-] [80244] Writing DB stats keystone:SELECT=100 {{(pid=80244) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:17:12.279354 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-724f9009-eda7-4d8f-b3d2-97ec83fbd375 None admin] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:17:12.281944 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3163987-c265-4fef-bc73-8b559d1caaff None admin] Validating token access rules against request {{(pid=80245) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:17:12.312859 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad4d0c97-4e0b-4ef7-a997-d2407b9231f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:17:12.313579 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad4d0c97-4e0b-4ef7-a997-d2407b9231f8 None admin] REQUEST_METHOD: `GET` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:17:12.313805 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad4d0c97-4e0b-4ef7-a997-d2407b9231f8 None admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:17:12.314076 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad4d0c97-4e0b-4ef7-a997-d2407b9231f8 None admin] PATH_INFO: `/v3/projects` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:17:12.314860 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad4d0c97-4e0b-4ef7-a997-d2407b9231f8 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80244) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 08:17:12.315170 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad4d0c97-4e0b-4ef7-a997-d2407b9231f8 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:17:12.316509 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c706fdb1-829c-472f-ab00-8e500451bc49 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80245) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:17:12.316624 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad4d0c97-4e0b-4ef7-a997-d2407b9231f8 None admin] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:17:12.317821 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c706fdb1-829c-472f-ab00-8e500451bc49 None admin] REQUEST_METHOD: `GET` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:17:12.318096 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c706fdb1-829c-472f-ab00-8e500451bc49 None admin] SCRIPT_NAME: `/identity` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:17:12.318364 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c706fdb1-829c-472f-ab00-8e500451bc49 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:17:12.325538 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG dbcounter [-] [80245] Writing DB stats keystone:SELECT=100 {{(pid=80245) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:17:12.330841 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c706fdb1-829c-472f-ab00-8e500451bc49 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 21 08:17:12.340548 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 284/569] 10.222.0.47 () {56 vars in 1148 bytes} [Thu Jul 21 08:17:12 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 08:17:12.340548 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad4d0c97-4e0b-4ef7-a997-d2407b9231f8 None admin] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:17:12.340795 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application [None req-c706fdb1-829c-472f-ab00-8e500451bc49 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 21 08:17:12.340795 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 08:17:12.340795 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 08:17:12.340795 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 08:17:12.340795 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 08:17:12.340795 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 08:17:12.340795 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 08:17:12.340795 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 08:17:12.340795 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 08:17:12.340795 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 08:17:12.340795 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 08:17:12.340795 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 08:17:12.340795 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 21 08:17:12.340795 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 21 08:17:12.340795 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 21 08:17:12.340795 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 21 08:17:12.341631 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 21 08:17:12.341631 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 21 08:17:12.341631 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 21 08:17:12.341631 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 21 08:17:12.341631 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 08:17:12.341631 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 08:17:12.341631 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 08:17:12.341631 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 08:17:12.341631 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 08:17:12.341631 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 08:17:12.341631 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 08:17:12.341631 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application with Lock( Jul 21 08:17:12.341631 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 08:17:12.341631 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application return self._enter() Jul 21 08:17:12.341631 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 08:17:12.341631 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 08:17:12.342464 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 08:17:12.342464 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application return self.creator() Jul 21 08:17:12.342464 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 08:17:12.342464 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application created_value = creator( Jul 21 08:17:12.342464 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 21 08:17:12.342464 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 21 08:17:12.342464 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 21 08:17:12.342464 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 21 08:17:12.342464 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 21 08:17:12.342464 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 21 08:17:12.342464 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 21 08:17:12.342464 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application Jul 21 08:17:12.342464 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: [pid: 80245|app: 0|req: 286/570] 10.222.0.47 () {56 vars in 1102 bytes} [Thu Jul 21 08:17:12 2022] GET /identity/v3/roles/service => generated 85 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 08:17:12.342464 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c706fdb1-829c-472f-ab00-8e500451bc49 None admin] Authenticating user token {{(pid=80245) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:17:12.451949 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad4d0c97-4e0b-4ef7-a997-d2407b9231f8 None admin] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:17:12.452192 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c706fdb1-829c-472f-ab00-8e500451bc49 None admin] Validating token access rules against request {{(pid=80245) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:17:12.478062 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG dbcounter [-] [80244] Writing DB stats keystone:SELECT=100 {{(pid=80244) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:17:12.493825 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf744b7b-43d2-4c92-b469-03352b0204c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:17:12.494496 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf744b7b-43d2-4c92-b469-03352b0204c1 None admin] REQUEST_METHOD: `GET` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:17:12.494778 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf744b7b-43d2-4c92-b469-03352b0204c1 None admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:17:12.494877 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30300ef8-9257-4e86-81eb-83d727540bd2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80245) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:17:12.495054 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf744b7b-43d2-4c92-b469-03352b0204c1 None admin] PATH_INFO: `/v3/domains` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:17:12.495638 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30300ef8-9257-4e86-81eb-83d727540bd2 None admin] REQUEST_METHOD: `GET` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:17:12.495802 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf744b7b-43d2-4c92-b469-03352b0204c1 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80244) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 08:17:12.495978 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30300ef8-9257-4e86-81eb-83d727540bd2 None admin] SCRIPT_NAME: `/identity` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:17:12.496125 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf744b7b-43d2-4c92-b469-03352b0204c1 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:17:12.496213 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30300ef8-9257-4e86-81eb-83d727540bd2 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:17:12.497464 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf744b7b-43d2-4c92-b469-03352b0204c1 None admin] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:17:12.507010 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30300ef8-9257-4e86-81eb-83d727540bd2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:17:12.508623 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30300ef8-9257-4e86-81eb-83d727540bd2 None admin] RBAC: Authorization granted {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:17:12.514066 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 285/571] 10.222.0.47 () {56 vars in 1109 bytes} [Thu Jul 21 08:17:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 08:17:12.514066 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf744b7b-43d2-4c92-b469-03352b0204c1 None admin] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:17:12.530845 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application [None req-30300ef8-9257-4e86-81eb-83d727540bd2 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 08:17:12.530845 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 08:17:12.530845 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 08:17:12.530845 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 08:17:12.530845 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 08:17:12.530845 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 08:17:12.530845 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 08:17:12.530845 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 08:17:12.530845 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 08:17:12.530845 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 08:17:12.530845 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 08:17:12.530845 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 08:17:12.530845 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 21 08:17:12.530845 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 21 08:17:12.530845 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 21 08:17:12.531987 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 21 08:17:12.531987 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 08:17:12.531987 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 08:17:12.531987 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 08:17:12.531987 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 08:17:12.531987 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 08:17:12.531987 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 08:17:12.531987 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 08:17:12.531987 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application with Lock( Jul 21 08:17:12.531987 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 08:17:12.531987 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application return self._enter() Jul 21 08:17:12.531987 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 08:17:12.531987 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 08:17:12.531987 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 08:17:12.531987 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application return self.creator() Jul 21 08:17:12.531987 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 08:17:12.531987 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application created_value = creator( Jul 21 08:17:12.532888 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 21 08:17:12.532888 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 21 08:17:12.532888 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 08:17:12.532888 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application Jul 21 08:17:12.532888 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: [pid: 80245|app: 0|req: 287/572] 10.222.0.47 () {56 vars in 1108 bytes} [Thu Jul 21 08:17:12 2022] GET /identity/v3/domains/Default => generated 87 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 08:17:12.532888 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30300ef8-9257-4e86-81eb-83d727540bd2 None admin] Authenticating user token {{(pid=80245) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:17:12.585987 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG dbcounter [-] [80245] Writing DB stats keystone:SELECT=100 {{(pid=80245) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:17:12.628804 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf744b7b-43d2-4c92-b469-03352b0204c1 None admin] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:17:12.639348 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30300ef8-9257-4e86-81eb-83d727540bd2 None admin] Validating token access rules against request {{(pid=80245) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:17:12.666499 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9be7d82-4037-4315-857d-e880d90d617b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:17:12.667398 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9be7d82-4037-4315-857d-e880d90d617b None admin] REQUEST_METHOD: `GET` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:17:12.667542 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9be7d82-4037-4315-857d-e880d90d617b None admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:17:12.667717 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9be7d82-4037-4315-857d-e880d90d617b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:17:12.668556 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9be7d82-4037-4315-857d-e880d90d617b None admin] RBAC: Adding query filter params (role.id=50d1ec7f947f42acb65c57e07f2269d6, scope.project.id=4faef13aa50e4e36b6c265fb1398fa14, user.id=302351411fc84248a6b166b25d21609f) {{(pid=80244) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 08:17:12.668870 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9be7d82-4037-4315-857d-e880d90d617b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:17:12.670319 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9be7d82-4037-4315-857d-e880d90d617b None admin] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:17:12.674592 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0518a2fa-6457-46b8-80cb-7a58b27f53c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80245) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:17:12.675254 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0518a2fa-6457-46b8-80cb-7a58b27f53c1 None admin] REQUEST_METHOD: `GET` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:17:12.675565 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0518a2fa-6457-46b8-80cb-7a58b27f53c1 None admin] SCRIPT_NAME: `/identity` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:17:12.675839 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0518a2fa-6457-46b8-80cb-7a58b27f53c1 None admin] PATH_INFO: `/v3/roles` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:17:12.676064 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 286/573] 10.222.0.47 () {56 vars in 1374 bytes} [Thu Jul 21 08:17:12 2022] GET /identity/v3/role_assignments?user.id=302351411fc84248a6b166b25d21609f&scope.project.id=4faef13aa50e4e36b6c265fb1398fa14&role.id=50d1ec7f947f42acb65c57e07f2269d6 => generated 259 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 08:17:12.676712 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0518a2fa-6457-46b8-80cb-7a58b27f53c1 None admin] RBAC: Adding query filter params (name=service) {{(pid=80245) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 08:17:12.676897 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9be7d82-4037-4315-857d-e880d90d617b None admin] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:17:12.677075 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0518a2fa-6457-46b8-80cb-7a58b27f53c1 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:17:12.678447 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0518a2fa-6457-46b8-80cb-7a58b27f53c1 None admin] RBAC: Authorization granted {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:17:12.687595 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: [pid: 80245|app: 0|req: 288/574] 10.222.0.47 () {56 vars in 1103 bytes} [Thu Jul 21 08:17:12 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 08:17:12.688382 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0518a2fa-6457-46b8-80cb-7a58b27f53c1 None admin] Authenticating user token {{(pid=80245) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:17:12.737196 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG dbcounter [-] [80244] Writing DB stats keystone:SELECT=100 {{(pid=80244) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:17:12.805299 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9be7d82-4037-4315-857d-e880d90d617b None admin] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:17:12.808397 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0518a2fa-6457-46b8-80cb-7a58b27f53c1 None admin] Validating token access rules against request {{(pid=80245) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:17:12.822764 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG dbcounter [-] [80245] Writing DB stats keystone:SELECT=100 {{(pid=80245) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:17:12.848158 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f971e3e7-edff-4504-8423-1397bf91c9cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:17:12.848831 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07f5e2d4-d3f0-4214-a007-b0d39f787c34 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80245) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:17:12.848949 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f971e3e7-edff-4504-8423-1397bf91c9cc None admin] REQUEST_METHOD: `GET` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:17:12.849187 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f971e3e7-edff-4504-8423-1397bf91c9cc None admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:17:12.849461 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f971e3e7-edff-4504-8423-1397bf91c9cc None admin] PATH_INFO: `/v3/users/glance` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:17:12.849557 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07f5e2d4-d3f0-4214-a007-b0d39f787c34 None admin] REQUEST_METHOD: `GET` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:17:12.849776 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07f5e2d4-d3f0-4214-a007-b0d39f787c34 None admin] SCRIPT_NAME: `/identity` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:17:12.850055 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07f5e2d4-d3f0-4214-a007-b0d39f787c34 None admin] PATH_INFO: `/v3/domains` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:17:12.850779 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07f5e2d4-d3f0-4214-a007-b0d39f787c34 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80245) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 08:17:12.851090 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07f5e2d4-d3f0-4214-a007-b0d39f787c34 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:17:12.852791 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07f5e2d4-d3f0-4214-a007-b0d39f787c34 None admin] RBAC: Authorization granted {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:17:12.857656 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f971e3e7-edff-4504-8423-1397bf91c9cc None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:17:12.859104 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f971e3e7-edff-4504-8423-1397bf91c9cc None admin] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:17:12.865698 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: [pid: 80245|app: 0|req: 289/575] 10.222.0.47 () {56 vars in 1109 bytes} [Thu Jul 21 08:17:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 08:17:12.867050 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d5d7cc2-ec79-4a86-9e19-c4a205b614f2 None None] REQUEST_METHOD: `GET` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:17:12.867571 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d5d7cc2-ec79-4a86-9e19-c4a205b614f2 None None] SCRIPT_NAME: `/identity` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:17:12.867677 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d5d7cc2-ec79-4a86-9e19-c4a205b614f2 None None] PATH_INFO: `/` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:17:12.869006 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: [pid: 80245|app: 0|req: 290/576] 10.222.0.47 () {54 vars in 926 bytes} [Thu Jul 21 08:17:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 08:17:12.869584 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d5d7cc2-ec79-4a86-9e19-c4a205b614f2 None None] Authenticating user token {{(pid=80245) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:17:12.881251 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application [None req-f971e3e7-edff-4504-8423-1397bf91c9cc None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Jul 21 08:17:12.881251 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 08:17:12.881251 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 08:17:12.881251 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 08:17:12.881251 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 08:17:12.881251 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 08:17:12.881251 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 08:17:12.881251 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 08:17:12.881251 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 08:17:12.881251 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 08:17:12.881251 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 08:17:12.881251 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 08:17:12.881251 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 21 08:17:12.881251 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 21 08:17:12.881251 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 21 08:17:12.881251 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 21 08:17:12.882199 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 08:17:12.882199 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 08:17:12.882199 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 21 08:17:12.882199 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 21 08:17:12.882199 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 21 08:17:12.882199 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 21 08:17:12.882199 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 08:17:12.882199 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 08:17:12.882199 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 08:17:12.882199 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 08:17:12.882199 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 08:17:12.882199 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application with Lock( Jul 21 08:17:12.882199 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 08:17:12.882199 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self._enter() Jul 21 08:17:12.882199 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 08:17:12.882199 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 08:17:12.882199 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 08:17:12.882199 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self.creator() Jul 21 08:17:12.883469 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 08:17:12.883469 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application created_value = creator( Jul 21 08:17:12.883469 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 21 08:17:12.883469 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 21 08:17:12.883469 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 21 08:17:12.883469 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 21 08:17:12.883469 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 21 08:17:12.883469 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 21 08:17:12.883469 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Jul 21 08:17:12.883469 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application Jul 21 08:17:12.883469 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 287/577] 10.222.0.47 () {56 vars in 1099 bytes} [Thu Jul 21 08:17:12 2022] GET /identity/v3/users/glance => generated 84 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 08:17:12.884071 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f971e3e7-edff-4504-8423-1397bf91c9cc None admin] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:17:12.990560 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f971e3e7-edff-4504-8423-1397bf91c9cc None admin] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:17:12.991565 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d5d7cc2-ec79-4a86-9e19-c4a205b614f2 None None] Validating token access rules against request {{(pid=80245) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:17:12.993095 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG dbcounter [-] [80244] Writing DB stats keystone:SELECT=100 {{(pid=80244) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:17:13.026650 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d56a54a6-dd39-418a-a3e9-6dfd9578d9fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:17:13.027219 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31e12d95-ede8-44b3-bcff-d04150ebf73d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80245) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:17:13.027737 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d56a54a6-dd39-418a-a3e9-6dfd9578d9fa None admin] REQUEST_METHOD: `GET` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:17:13.028039 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31e12d95-ede8-44b3-bcff-d04150ebf73d None admin] REQUEST_METHOD: `GET` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:17:13.028195 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d56a54a6-dd39-418a-a3e9-6dfd9578d9fa None admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:17:13.028319 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31e12d95-ede8-44b3-bcff-d04150ebf73d None admin] SCRIPT_NAME: `/identity` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:17:13.028590 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31e12d95-ede8-44b3-bcff-d04150ebf73d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:17:13.028693 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d56a54a6-dd39-418a-a3e9-6dfd9578d9fa None admin] PATH_INFO: `/v3/projects/service` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:17:13.036165 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d56a54a6-dd39-418a-a3e9-6dfd9578d9fa None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:17:13.037691 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d56a54a6-dd39-418a-a3e9-6dfd9578d9fa None admin] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:17:13.039876 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31e12d95-ede8-44b3-bcff-d04150ebf73d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:17:13.041303 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31e12d95-ede8-44b3-bcff-d04150ebf73d None admin] RBAC: Authorization granted {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:17:13.052813 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application [None req-d56a54a6-dd39-418a-a3e9-6dfd9578d9fa None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 21 08:17:13.052813 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 08:17:13.052813 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 08:17:13.052813 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 08:17:13.052813 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 08:17:13.052813 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 08:17:13.052813 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 08:17:13.052813 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 08:17:13.052813 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 08:17:13.052813 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 08:17:13.052813 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 08:17:13.052813 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 08:17:13.052813 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 21 08:17:13.052813 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 21 08:17:13.052813 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 21 08:17:13.053868 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 21 08:17:13.053868 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 08:17:13.053868 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 08:17:13.053868 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 08:17:13.053868 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 08:17:13.053868 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 08:17:13.053868 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 08:17:13.053868 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 08:17:13.053868 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application with Lock( Jul 21 08:17:13.053868 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 08:17:13.053868 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self._enter() Jul 21 08:17:13.053868 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 08:17:13.053868 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 08:17:13.053868 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 08:17:13.053868 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self.creator() Jul 21 08:17:13.053868 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 08:17:13.053868 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application created_value = creator( Jul 21 08:17:13.054784 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 21 08:17:13.054784 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 21 08:17:13.054784 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 21 08:17:13.054784 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 21 08:17:13.054784 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 21 08:17:13.054784 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 21 08:17:13.054784 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 21 08:17:13.054784 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application Jul 21 08:17:13.059377 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 288/578] 10.222.0.47 () {56 vars in 1111 bytes} [Thu Jul 21 08:17:12 2022] GET /identity/v3/projects/service => generated 88 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 08:17:13.059587 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application [None req-31e12d95-ede8-44b3-bcff-d04150ebf73d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 08:17:13.059587 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 08:17:13.059587 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 08:17:13.059587 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 08:17:13.059587 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 08:17:13.059587 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 08:17:13.059587 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 08:17:13.059587 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 08:17:13.059587 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 08:17:13.059587 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 08:17:13.059587 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 08:17:13.059587 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 08:17:13.059587 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 21 08:17:13.059587 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 21 08:17:13.059587 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 21 08:17:13.060410 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 21 08:17:13.060410 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 08:17:13.060410 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 08:17:13.060410 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 08:17:13.060410 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 08:17:13.060410 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 08:17:13.060410 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 08:17:13.060410 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 08:17:13.060410 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application with Lock( Jul 21 08:17:13.060410 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 08:17:13.060410 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application return self._enter() Jul 21 08:17:13.060410 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 08:17:13.060410 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 08:17:13.060410 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 08:17:13.060410 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application return self.creator() Jul 21 08:17:13.060410 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 08:17:13.060410 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application created_value = creator( Jul 21 08:17:13.067170 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 21 08:17:13.067170 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 21 08:17:13.067170 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 08:17:13.067170 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application Jul 21 08:17:13.067942 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-727e43a7-323e-4b48-b51a-e764a34a5068 None None] REQUEST_METHOD: `POST` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:17:13.067942 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-727e43a7-323e-4b48-b51a-e764a34a5068 None None] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:17:13.068230 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: [pid: 80245|app: 0|req: 291/579] 10.222.0.47 () {56 vars in 1108 bytes} [Thu Jul 21 08:17:12 2022] GET /identity/v3/domains/Default => generated 87 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 08:17:13.069264 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-727e43a7-323e-4b48-b51a-e764a34a5068 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:17:13.069374 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31e12d95-ede8-44b3-bcff-d04150ebf73d None admin] Authenticating user token {{(pid=80245) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:17:13.080065 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.auth.core [None req-727e43a7-323e-4b48-b51a-e764a34a5068 None None] MFA Rules not processed for user `2dce40b8d74c41ec94cc36e6ff05d412`. Rule list: `[]` (Enabled: `True`). {{(pid=80244) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 08:17:13.081282 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.fernet_utils [None req-727e43a7-323e-4b48-b51a-e764a34a5068 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80244) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 08:17:13.092349 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG dbcounter [-] [80245] Writing DB stats keystone:SELECT=100 {{(pid=80245) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:17:13.154420 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 289/580] 10.222.0.47 () {58 vars in 1023 bytes} [Thu Jul 21 08:17:13 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 21 08:17:13.155536 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-727e43a7-323e-4b48-b51a-e764a34a5068 None None] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:17:13.183236 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31e12d95-ede8-44b3-bcff-d04150ebf73d None admin] Validating token access rules against request {{(pid=80245) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:17:13.221165 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97aac825-4324-472a-830e-dc1dafbfe073 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80245) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:17:13.221848 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97aac825-4324-472a-830e-dc1dafbfe073 None admin] REQUEST_METHOD: `GET` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:17:13.222130 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97aac825-4324-472a-830e-dc1dafbfe073 None admin] SCRIPT_NAME: `/identity` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:17:13.222406 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97aac825-4324-472a-830e-dc1dafbfe073 None admin] PATH_INFO: `/v3/users` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:17:13.223819 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97aac825-4324-472a-830e-dc1dafbfe073 None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=80245) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 08:17:13.224132 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97aac825-4324-472a-830e-dc1dafbfe073 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:17:13.225583 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97aac825-4324-472a-830e-dc1dafbfe073 None admin] RBAC: Authorization granted {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:17:13.243416 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: [pid: 80245|app: 0|req: 292/581] 10.222.0.47 () {56 vars in 1137 bytes} [Thu Jul 21 08:17:13 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 08:17:13.244234 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97aac825-4324-472a-830e-dc1dafbfe073 None admin] Authenticating user token {{(pid=80245) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:17:13.273634 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG dbcounter [-] [80244] Writing DB stats keystone:SELECT=100 {{(pid=80244) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:17:13.282329 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-727e43a7-323e-4b48-b51a-e764a34a5068 None None] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:17:13.323718 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90a5a820-06eb-4f52-a38d-6373ca68fde0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:17:13.324401 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90a5a820-06eb-4f52-a38d-6373ca68fde0 None admin] REQUEST_METHOD: `GET` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:17:13.325024 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90a5a820-06eb-4f52-a38d-6373ca68fde0 None admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:17:13.325312 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90a5a820-06eb-4f52-a38d-6373ca68fde0 None admin] PATH_INFO: `/v3/projects` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:17:13.326105 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90a5a820-06eb-4f52-a38d-6373ca68fde0 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80244) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 08:17:13.326725 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90a5a820-06eb-4f52-a38d-6373ca68fde0 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:17:13.327183 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG dbcounter [-] [80245] Writing DB stats keystone:SELECT=100 {{(pid=80245) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:17:13.327886 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90a5a820-06eb-4f52-a38d-6373ca68fde0 None admin] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:17:13.341156 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 290/582] 10.222.0.47 () {56 vars in 1148 bytes} [Thu Jul 21 08:17:13 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 08:17:13.341912 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90a5a820-06eb-4f52-a38d-6373ca68fde0 None admin] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:17:13.352633 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97aac825-4324-472a-830e-dc1dafbfe073 None admin] Validating token access rules against request {{(pid=80245) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:17:13.388479 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89bead1b-2373-4db4-946b-6f0bac464277 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80245) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:17:13.389150 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89bead1b-2373-4db4-946b-6f0bac464277 None admin] REQUEST_METHOD: `GET` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:17:13.389435 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89bead1b-2373-4db4-946b-6f0bac464277 None admin] SCRIPT_NAME: `/identity` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:17:13.389709 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89bead1b-2373-4db4-946b-6f0bac464277 None admin] PATH_INFO: `/v3/domains` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:17:13.390426 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89bead1b-2373-4db4-946b-6f0bac464277 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80245) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 08:17:13.390731 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89bead1b-2373-4db4-946b-6f0bac464277 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:17:13.392132 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89bead1b-2373-4db4-946b-6f0bac464277 None admin] RBAC: Authorization granted {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:17:13.404687 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: [pid: 80245|app: 0|req: 293/583] 10.222.0.47 () {56 vars in 1109 bytes} [Thu Jul 21 08:17:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 08:17:13.405728 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89bead1b-2373-4db4-946b-6f0bac464277 None admin] Authenticating user token {{(pid=80245) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:17:13.454594 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90a5a820-06eb-4f52-a38d-6373ca68fde0 None admin] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:17:13.497494 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80e94541-086a-4cc9-bc54-1cbaad50dfdf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:17:13.498295 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80e94541-086a-4cc9-bc54-1cbaad50dfdf None admin] REQUEST_METHOD: `GET` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:17:13.498581 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80e94541-086a-4cc9-bc54-1cbaad50dfdf None admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:17:13.498866 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80e94541-086a-4cc9-bc54-1cbaad50dfdf None admin] PATH_INFO: `/v3/roles/service` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:17:13.510243 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80e94541-086a-4cc9-bc54-1cbaad50dfdf None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 21 08:17:13.514474 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89bead1b-2373-4db4-946b-6f0bac464277 None admin] Validating token access rules against request {{(pid=80245) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:17:13.517871 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application [None req-80e94541-086a-4cc9-bc54-1cbaad50dfdf None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 21 08:17:13.517871 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 08:17:13.517871 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 08:17:13.517871 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 08:17:13.517871 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 08:17:13.517871 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 08:17:13.517871 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 08:17:13.517871 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 08:17:13.517871 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 08:17:13.517871 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 08:17:13.517871 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 08:17:13.517871 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 08:17:13.517871 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 21 08:17:13.517871 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 21 08:17:13.517871 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 21 08:17:13.517871 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 21 08:17:13.518794 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 21 08:17:13.518794 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 21 08:17:13.518794 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 21 08:17:13.518794 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 21 08:17:13.518794 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 08:17:13.518794 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 08:17:13.518794 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 08:17:13.518794 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 08:17:13.518794 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 08:17:13.518794 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 08:17:13.518794 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 08:17:13.518794 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application with Lock( Jul 21 08:17:13.518794 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 08:17:13.518794 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self._enter() Jul 21 08:17:13.518794 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 08:17:13.518794 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 08:17:13.519623 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 08:17:13.519623 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self.creator() Jul 21 08:17:13.519623 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 08:17:13.519623 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application created_value = creator( Jul 21 08:17:13.519623 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 21 08:17:13.519623 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 21 08:17:13.519623 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 21 08:17:13.519623 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 21 08:17:13.519623 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 21 08:17:13.519623 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 21 08:17:13.519623 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 21 08:17:13.519623 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application Jul 21 08:17:13.519623 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 291/584] 10.222.0.47 () {56 vars in 1102 bytes} [Thu Jul 21 08:17:13 2022] GET /identity/v3/roles/service => generated 85 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 08:17:13.520875 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80e94541-086a-4cc9-bc54-1cbaad50dfdf None admin] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:17:13.538485 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG dbcounter [-] [80244] Writing DB stats keystone:SELECT=100 {{(pid=80244) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:17:13.553770 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69888950-409e-4ab7-be48-48978db102ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80245) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:17:13.554574 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69888950-409e-4ab7-be48-48978db102ca None admin] REQUEST_METHOD: `GET` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:17:13.554856 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69888950-409e-4ab7-be48-48978db102ca None admin] SCRIPT_NAME: `/identity` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:17:13.555131 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69888950-409e-4ab7-be48-48978db102ca None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:17:13.566113 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69888950-409e-4ab7-be48-48978db102ca None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:17:13.568310 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69888950-409e-4ab7-be48-48978db102ca None admin] RBAC: Authorization granted {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:17:13.570268 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG dbcounter [-] [80245] Writing DB stats keystone:SELECT=100 {{(pid=80245) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:17:13.587696 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application [None req-69888950-409e-4ab7-be48-48978db102ca None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 08:17:13.587696 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 08:17:13.587696 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 08:17:13.587696 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 08:17:13.587696 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 08:17:13.587696 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 08:17:13.587696 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 08:17:13.587696 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 08:17:13.587696 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 08:17:13.587696 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 08:17:13.587696 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 08:17:13.587696 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 08:17:13.587696 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 21 08:17:13.587696 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 21 08:17:13.587696 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 21 08:17:13.588621 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 21 08:17:13.588621 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 08:17:13.588621 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 08:17:13.588621 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 08:17:13.588621 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 08:17:13.588621 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 08:17:13.588621 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 08:17:13.588621 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 08:17:13.588621 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application with Lock( Jul 21 08:17:13.588621 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 08:17:13.588621 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application return self._enter() Jul 21 08:17:13.588621 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 08:17:13.588621 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 08:17:13.588621 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 08:17:13.588621 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application return self.creator() Jul 21 08:17:13.588621 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 08:17:13.588621 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application created_value = creator( Jul 21 08:17:13.589521 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 21 08:17:13.589521 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 21 08:17:13.589521 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 08:17:13.589521 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application Jul 21 08:17:13.589521 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: [pid: 80245|app: 0|req: 294/585] 10.222.0.47 () {56 vars in 1108 bytes} [Thu Jul 21 08:17:13 2022] GET /identity/v3/domains/Default => generated 87 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 08:17:13.590351 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69888950-409e-4ab7-be48-48978db102ca None admin] Authenticating user token {{(pid=80245) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:17:13.630551 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80e94541-086a-4cc9-bc54-1cbaad50dfdf None admin] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:17:13.670415 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d932a337-225f-4f2b-a868-0f1e486288ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:17:13.671627 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d932a337-225f-4f2b-a868-0f1e486288ef None admin] REQUEST_METHOD: `PUT` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:17:13.672088 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d932a337-225f-4f2b-a868-0f1e486288ef None admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:17:13.672539 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d932a337-225f-4f2b-a868-0f1e486288ef None admin] PATH_INFO: `/v3/projects/4faef13aa50e4e36b6c265fb1398fa14/users/ac2b5b99a36442148fe743269054e991/roles/50d1ec7f947f42acb65c57e07f2269d6` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:17:13.688429 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d932a337-225f-4f2b-a868-0f1e486288ef None admin] RBAC: Authorizing `identity:create_grant(project_id=4faef13aa50e4e36b6c265fb1398fa14, user_id=ac2b5b99a36442148fe743269054e991, role_id=50d1ec7f947f42acb65c57e07f2269d6)` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:17:13.689906 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d932a337-225f-4f2b-a868-0f1e486288ef None admin] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:17:13.700802 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69888950-409e-4ab7-be48-48978db102ca None admin] Validating token access rules against request {{(pid=80245) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:17:13.703240 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 292/586] 10.222.0.47 () {58 vars in 1439 bytes} [Thu Jul 21 08:17:13 2022] PUT /identity/v3/projects/4faef13aa50e4e36b6c265fb1398fa14/users/ac2b5b99a36442148fe743269054e991/roles/50d1ec7f947f42acb65c57e07f2269d6 => generated 0 bytes in 184 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 08:17:13.704073 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d932a337-225f-4f2b-a868-0f1e486288ef None admin] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:17:13.764872 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34e65329-b405-48c7-a926-fa23c2528e93 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80245) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:17:13.766562 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34e65329-b405-48c7-a926-fa23c2528e93 None admin] REQUEST_METHOD: `GET` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:17:13.767140 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34e65329-b405-48c7-a926-fa23c2528e93 None admin] SCRIPT_NAME: `/identity` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:17:13.767973 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34e65329-b405-48c7-a926-fa23c2528e93 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:17:13.780824 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34e65329-b405-48c7-a926-fa23c2528e93 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:17:13.783863 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34e65329-b405-48c7-a926-fa23c2528e93 None admin] RBAC: Authorization granted {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:17:13.790094 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG dbcounter [-] [80244] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80244) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:17:13.806460 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application [None req-34e65329-b405-48c7-a926-fa23c2528e93 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Jul 21 08:17:13.806460 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 08:17:13.806460 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 08:17:13.806460 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 08:17:13.806460 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 08:17:13.806460 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 08:17:13.806460 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 08:17:13.806460 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 08:17:13.806460 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 08:17:13.806460 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 08:17:13.806460 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 08:17:13.806460 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 08:17:13.806460 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 21 08:17:13.806460 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 21 08:17:13.806460 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 21 08:17:13.806460 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 21 08:17:13.807505 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 08:17:13.807505 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 08:17:13.807505 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 21 08:17:13.807505 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 21 08:17:13.807505 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 21 08:17:13.807505 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 21 08:17:13.807505 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 08:17:13.807505 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 08:17:13.807505 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 08:17:13.807505 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 08:17:13.807505 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 08:17:13.807505 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application with Lock( Jul 21 08:17:13.807505 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 08:17:13.807505 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application return self._enter() Jul 21 08:17:13.807505 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 08:17:13.807505 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 08:17:13.807505 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 08:17:13.807505 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application return self.creator() Jul 21 08:17:13.808430 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 08:17:13.808430 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application created_value = creator( Jul 21 08:17:13.808430 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 21 08:17:13.808430 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 21 08:17:13.808430 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 21 08:17:13.808430 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 21 08:17:13.808430 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 21 08:17:13.808430 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 21 08:17:13.808430 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Jul 21 08:17:13.808430 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application Jul 21 08:17:13.817676 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: [pid: 80245|app: 0|req: 295/587] 10.222.0.47 () {56 vars in 1099 bytes} [Thu Jul 21 08:17:13 2022] GET /identity/v3/users/cinder => generated 84 bytes in 222 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 08:17:13.817676 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34e65329-b405-48c7-a926-fa23c2528e93 None admin] Authenticating user token {{(pid=80245) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:17:13.853845 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d932a337-225f-4f2b-a868-0f1e486288ef None admin] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:17:13.894150 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG dbcounter [-] [80245] Writing DB stats keystone:SELECT=100 {{(pid=80245) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:17:13.903478 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f23c7b8e-2720-4bf8-971b-2b63bb53065f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:17:13.904200 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f23c7b8e-2720-4bf8-971b-2b63bb53065f None admin] REQUEST_METHOD: `GET` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:17:13.904489 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f23c7b8e-2720-4bf8-971b-2b63bb53065f None admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:17:13.904815 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f23c7b8e-2720-4bf8-971b-2b63bb53065f None admin] PATH_INFO: `/v3/roles` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:17:13.905586 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f23c7b8e-2720-4bf8-971b-2b63bb53065f None admin] RBAC: Adding query filter params (name=service) {{(pid=80244) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 08:17:13.905894 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f23c7b8e-2720-4bf8-971b-2b63bb53065f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:17:13.907290 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f23c7b8e-2720-4bf8-971b-2b63bb53065f None admin] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:17:13.924533 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 293/588] 10.222.0.47 () {56 vars in 1103 bytes} [Thu Jul 21 08:17:13 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 221 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 08:17:13.925399 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f23c7b8e-2720-4bf8-971b-2b63bb53065f None admin] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:17:13.964702 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34e65329-b405-48c7-a926-fa23c2528e93 None admin] Validating token access rules against request {{(pid=80245) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:17:14.012710 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe742255-f5af-4f82-9402-2603377f3ec8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80245) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:17:14.013587 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe742255-f5af-4f82-9402-2603377f3ec8 None admin] REQUEST_METHOD: `GET` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:17:14.014392 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe742255-f5af-4f82-9402-2603377f3ec8 None admin] SCRIPT_NAME: `/identity` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:17:14.014786 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe742255-f5af-4f82-9402-2603377f3ec8 None admin] PATH_INFO: `/v3/domains` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:17:14.016049 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe742255-f5af-4f82-9402-2603377f3ec8 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80245) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 08:17:14.016724 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe742255-f5af-4f82-9402-2603377f3ec8 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:17:14.018460 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe742255-f5af-4f82-9402-2603377f3ec8 None admin] RBAC: Authorization granted {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:17:14.036539 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: [pid: 80245|app: 0|req: 296/589] 10.222.0.47 () {56 vars in 1109 bytes} [Thu Jul 21 08:17:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 224 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 08:17:14.037339 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe742255-f5af-4f82-9402-2603377f3ec8 None admin] Authenticating user token {{(pid=80245) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:17:14.037872 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f23c7b8e-2720-4bf8-971b-2b63bb53065f None admin] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:17:14.066608 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG dbcounter [-] [80244] Writing DB stats keystone:SELECT=100 {{(pid=80244) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:17:14.081727 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f9db7b1-a1ca-4545-8116-91121dc9f986 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:17:14.082570 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f9db7b1-a1ca-4545-8116-91121dc9f986 None admin] REQUEST_METHOD: `GET` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:17:14.082916 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f9db7b1-a1ca-4545-8116-91121dc9f986 None admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:17:14.083517 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f9db7b1-a1ca-4545-8116-91121dc9f986 None admin] PATH_INFO: `/v3/users` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:17:14.085324 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f9db7b1-a1ca-4545-8116-91121dc9f986 None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=80244) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 08:17:14.085703 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f9db7b1-a1ca-4545-8116-91121dc9f986 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:17:14.087259 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f9db7b1-a1ca-4545-8116-91121dc9f986 None admin] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:17:14.116106 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 294/590] 10.222.0.47 () {56 vars in 1137 bytes} [Thu Jul 21 08:17:13 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 08:17:14.117019 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f9db7b1-a1ca-4545-8116-91121dc9f986 None admin] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:17:14.158641 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe742255-f5af-4f82-9402-2603377f3ec8 None admin] Validating token access rules against request {{(pid=80245) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:17:14.173660 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG dbcounter [-] [80245] Writing DB stats keystone:SELECT=100 {{(pid=80245) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:17:14.196726 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-411255fc-8a9e-4408-9416-42ef51bf1700 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80245) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:17:14.197806 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-411255fc-8a9e-4408-9416-42ef51bf1700 None admin] REQUEST_METHOD: `GET` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:17:14.198252 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-411255fc-8a9e-4408-9416-42ef51bf1700 None admin] SCRIPT_NAME: `/identity` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:17:14.198672 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-411255fc-8a9e-4408-9416-42ef51bf1700 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:17:14.210311 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-411255fc-8a9e-4408-9416-42ef51bf1700 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:17:14.212049 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-411255fc-8a9e-4408-9416-42ef51bf1700 None admin] RBAC: Authorization granted {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:17:14.231960 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application [None req-411255fc-8a9e-4408-9416-42ef51bf1700 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 08:17:14.231960 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 08:17:14.231960 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 08:17:14.231960 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 08:17:14.231960 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 08:17:14.231960 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 08:17:14.231960 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 08:17:14.231960 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 08:17:14.231960 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 08:17:14.231960 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 08:17:14.231960 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 08:17:14.231960 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 08:17:14.231960 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 21 08:17:14.231960 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 21 08:17:14.231960 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 21 08:17:14.233040 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 21 08:17:14.233040 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 08:17:14.233040 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 08:17:14.233040 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 08:17:14.233040 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 08:17:14.233040 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 08:17:14.233040 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 08:17:14.233040 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 08:17:14.233040 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application with Lock( Jul 21 08:17:14.233040 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 08:17:14.233040 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application return self._enter() Jul 21 08:17:14.233040 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 08:17:14.233040 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 08:17:14.233040 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 08:17:14.233040 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application return self.creator() Jul 21 08:17:14.233040 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 08:17:14.233040 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application created_value = creator( Jul 21 08:17:14.233900 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 21 08:17:14.233900 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 21 08:17:14.233900 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 08:17:14.233900 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application Jul 21 08:17:14.234112 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f9db7b1-a1ca-4545-8116-91121dc9f986 None admin] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:17:14.240186 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: [pid: 80245|app: 0|req: 297/591] 10.222.0.47 () {56 vars in 1108 bytes} [Thu Jul 21 08:17:14 2022] GET /identity/v3/domains/Default => generated 87 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 08:17:14.241298 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-411255fc-8a9e-4408-9416-42ef51bf1700 None admin] Authenticating user token {{(pid=80245) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:17:14.273667 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ae23cfd-0f30-46a1-bbbe-d075aa77fbde None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:17:14.274546 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ae23cfd-0f30-46a1-bbbe-d075aa77fbde None admin] REQUEST_METHOD: `GET` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:17:14.274768 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ae23cfd-0f30-46a1-bbbe-d075aa77fbde None admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:17:14.275041 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ae23cfd-0f30-46a1-bbbe-d075aa77fbde None admin] PATH_INFO: `/v3/projects/service` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:17:14.283180 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ae23cfd-0f30-46a1-bbbe-d075aa77fbde None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:17:14.284657 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ae23cfd-0f30-46a1-bbbe-d075aa77fbde None admin] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:17:14.299502 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application [None req-5ae23cfd-0f30-46a1-bbbe-d075aa77fbde None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 21 08:17:14.299502 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 08:17:14.299502 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 08:17:14.299502 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 08:17:14.299502 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 08:17:14.299502 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 08:17:14.299502 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 08:17:14.299502 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 08:17:14.299502 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 08:17:14.299502 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 08:17:14.299502 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 08:17:14.299502 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 08:17:14.299502 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 21 08:17:14.299502 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 21 08:17:14.299502 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 21 08:17:14.300591 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 21 08:17:14.300591 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 08:17:14.300591 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 08:17:14.300591 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 08:17:14.300591 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 08:17:14.300591 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 08:17:14.300591 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 08:17:14.300591 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 08:17:14.300591 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application with Lock( Jul 21 08:17:14.300591 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 08:17:14.300591 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self._enter() Jul 21 08:17:14.300591 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 08:17:14.300591 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 08:17:14.300591 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 08:17:14.300591 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self.creator() Jul 21 08:17:14.300591 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 08:17:14.300591 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application created_value = creator( Jul 21 08:17:14.301519 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 21 08:17:14.301519 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 21 08:17:14.301519 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 21 08:17:14.301519 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 21 08:17:14.301519 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 21 08:17:14.301519 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 21 08:17:14.301519 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 21 08:17:14.301519 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application Jul 21 08:17:14.301519 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 295/592] 10.222.0.47 () {56 vars in 1111 bytes} [Thu Jul 21 08:17:14 2022] GET /identity/v3/projects/service => generated 88 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 08:17:14.301987 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ae23cfd-0f30-46a1-bbbe-d075aa77fbde None admin] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:17:14.351370 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG dbcounter [-] [80244] Writing DB stats keystone:SELECT=100 {{(pid=80244) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:17:14.358313 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-411255fc-8a9e-4408-9416-42ef51bf1700 None admin] Validating token access rules against request {{(pid=80245) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:17:14.395715 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4f85bd9-7b27-47cd-afc7-3d23dbe59dcf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80245) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:17:14.396904 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4f85bd9-7b27-47cd-afc7-3d23dbe59dcf None admin] REQUEST_METHOD: `GET` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:17:14.397372 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4f85bd9-7b27-47cd-afc7-3d23dbe59dcf None admin] SCRIPT_NAME: `/identity` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:17:14.397824 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4f85bd9-7b27-47cd-afc7-3d23dbe59dcf None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:17:14.409193 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4f85bd9-7b27-47cd-afc7-3d23dbe59dcf None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:17:14.410994 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4f85bd9-7b27-47cd-afc7-3d23dbe59dcf None admin] RBAC: Authorization granted {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:17:14.414159 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ae23cfd-0f30-46a1-bbbe-d075aa77fbde None admin] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:17:14.429693 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application [None req-a4f85bd9-7b27-47cd-afc7-3d23dbe59dcf None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 08:17:14.429693 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 08:17:14.429693 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 08:17:14.429693 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 08:17:14.429693 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 08:17:14.429693 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 08:17:14.429693 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 08:17:14.429693 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 08:17:14.429693 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 08:17:14.429693 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 08:17:14.429693 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 08:17:14.429693 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 08:17:14.429693 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 21 08:17:14.429693 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 21 08:17:14.429693 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 21 08:17:14.430589 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 21 08:17:14.430589 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 08:17:14.430589 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 08:17:14.430589 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 08:17:14.430589 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 08:17:14.430589 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 08:17:14.430589 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 08:17:14.430589 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 08:17:14.430589 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application with Lock( Jul 21 08:17:14.430589 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 08:17:14.430589 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application return self._enter() Jul 21 08:17:14.430589 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 08:17:14.430589 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 08:17:14.430589 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 08:17:14.430589 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application return self.creator() Jul 21 08:17:14.430589 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 08:17:14.430589 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application created_value = creator( Jul 21 08:17:14.431501 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 21 08:17:14.431501 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 21 08:17:14.431501 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 08:17:14.431501 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application Jul 21 08:17:14.433753 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: [pid: 80245|app: 0|req: 298/593] 10.222.0.47 () {56 vars in 1108 bytes} [Thu Jul 21 08:17:14 2022] GET /identity/v3/domains/Default => generated 87 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 08:17:14.434704 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4f85bd9-7b27-47cd-afc7-3d23dbe59dcf None admin] Authenticating user token {{(pid=80245) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:17:14.454742 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6ada0bd-24ae-4653-81ac-e9c20dc7f2c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:17:14.455052 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG dbcounter [-] [80245] Writing DB stats keystone:SELECT=100 {{(pid=80245) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:17:14.455398 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6ada0bd-24ae-4653-81ac-e9c20dc7f2c1 None admin] REQUEST_METHOD: `GET` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:17:14.455686 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6ada0bd-24ae-4653-81ac-e9c20dc7f2c1 None admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:17:14.456173 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6ada0bd-24ae-4653-81ac-e9c20dc7f2c1 None admin] PATH_INFO: `/v3/domains` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:17:14.456682 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6ada0bd-24ae-4653-81ac-e9c20dc7f2c1 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80244) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 08:17:14.456988 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6ada0bd-24ae-4653-81ac-e9c20dc7f2c1 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:17:14.459090 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6ada0bd-24ae-4653-81ac-e9c20dc7f2c1 None admin] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:17:14.477983 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 296/594] 10.222.0.47 () {56 vars in 1109 bytes} [Thu Jul 21 08:17:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 08:17:14.478962 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6ada0bd-24ae-4653-81ac-e9c20dc7f2c1 None admin] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:17:14.553953 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4f85bd9-7b27-47cd-afc7-3d23dbe59dcf None admin] Validating token access rules against request {{(pid=80245) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:17:14.585700 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6ada0bd-24ae-4653-81ac-e9c20dc7f2c1 None admin] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:17:14.588201 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG dbcounter [-] [80244] Writing DB stats keystone:SELECT=100 {{(pid=80244) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:17:14.596123 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d3fadff-b853-4680-a773-334a64ff109c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80245) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:17:14.597084 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d3fadff-b853-4680-a773-334a64ff109c None admin] REQUEST_METHOD: `GET` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:17:14.597548 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d3fadff-b853-4680-a773-334a64ff109c None admin] SCRIPT_NAME: `/identity` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:17:14.598499 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d3fadff-b853-4680-a773-334a64ff109c None admin] PATH_INFO: `/v3/projects` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:17:14.599493 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d3fadff-b853-4680-a773-334a64ff109c None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80245) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 08:17:14.599997 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d3fadff-b853-4680-a773-334a64ff109c None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:17:14.601617 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d3fadff-b853-4680-a773-334a64ff109c None admin] RBAC: Authorization granted {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:17:14.614786 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: [pid: 80245|app: 0|req: 299/595] 10.222.0.47 () {56 vars in 1148 bytes} [Thu Jul 21 08:17:14 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 08:17:14.615933 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d3fadff-b853-4680-a773-334a64ff109c None admin] Authenticating user token {{(pid=80245) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:17:14.624824 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3052569d-257b-40f7-92af-b4997495062f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:17:14.625493 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3052569d-257b-40f7-92af-b4997495062f None admin] REQUEST_METHOD: `GET` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:17:14.625775 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3052569d-257b-40f7-92af-b4997495062f None admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:17:14.626055 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3052569d-257b-40f7-92af-b4997495062f None admin] PATH_INFO: `/v3/domains` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:17:14.626775 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3052569d-257b-40f7-92af-b4997495062f None admin] RBAC: Adding query filter params (name=Default) {{(pid=80244) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 08:17:14.627079 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3052569d-257b-40f7-92af-b4997495062f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:17:14.628483 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3052569d-257b-40f7-92af-b4997495062f None admin] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:17:14.645882 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 297/596] 10.222.0.47 () {56 vars in 1109 bytes} [Thu Jul 21 08:17:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 08:17:14.646815 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3052569d-257b-40f7-92af-b4997495062f None admin] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:17:14.700425 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG dbcounter [-] [80245] Writing DB stats keystone:SELECT=100 {{(pid=80245) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:17:14.724701 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d3fadff-b853-4680-a773-334a64ff109c None admin] Validating token access rules against request {{(pid=80245) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:17:14.758693 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3052569d-257b-40f7-92af-b4997495062f None admin] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:17:14.762123 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c91427a7-105b-430f-97a1-441a7dd9e0b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80245) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:17:14.762925 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c91427a7-105b-430f-97a1-441a7dd9e0b0 None admin] REQUEST_METHOD: `GET` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:17:14.763201 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c91427a7-105b-430f-97a1-441a7dd9e0b0 None admin] SCRIPT_NAME: `/identity` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:17:14.763501 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c91427a7-105b-430f-97a1-441a7dd9e0b0 None admin] PATH_INFO: `/v3/users/neutron` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:17:14.772096 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c91427a7-105b-430f-97a1-441a7dd9e0b0 None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:17:14.773665 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c91427a7-105b-430f-97a1-441a7dd9e0b0 None admin] RBAC: Authorization granted {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:17:14.789830 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application [None req-c91427a7-105b-430f-97a1-441a7dd9e0b0 None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Jul 21 08:17:14.789830 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 08:17:14.789830 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 08:17:14.789830 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 08:17:14.789830 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 08:17:14.789830 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 08:17:14.789830 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 08:17:14.789830 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 08:17:14.789830 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 08:17:14.789830 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 08:17:14.789830 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 08:17:14.789830 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 08:17:14.789830 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 21 08:17:14.789830 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 21 08:17:14.789830 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 21 08:17:14.789830 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 21 08:17:14.790752 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 08:17:14.790752 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 08:17:14.790752 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 21 08:17:14.790752 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 21 08:17:14.790752 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 21 08:17:14.790752 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 21 08:17:14.790752 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 08:17:14.790752 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 08:17:14.790752 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 08:17:14.790752 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 08:17:14.790752 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 08:17:14.790752 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application with Lock( Jul 21 08:17:14.790752 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 08:17:14.790752 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application return self._enter() Jul 21 08:17:14.790752 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 08:17:14.790752 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 08:17:14.790752 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 08:17:14.790752 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application return self.creator() Jul 21 08:17:14.792122 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 08:17:14.792122 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application created_value = creator( Jul 21 08:17:14.792122 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 21 08:17:14.792122 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 21 08:17:14.792122 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 21 08:17:14.792122 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 21 08:17:14.792122 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 21 08:17:14.792122 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 21 08:17:14.792122 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Jul 21 08:17:14.792122 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application Jul 21 08:17:14.792122 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: [pid: 80245|app: 0|req: 300/597] 10.222.0.47 () {56 vars in 1102 bytes} [Thu Jul 21 08:17:14 2022] GET /identity/v3/users/neutron => generated 85 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 08:17:14.792122 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c91427a7-105b-430f-97a1-441a7dd9e0b0 None admin] Authenticating user token {{(pid=80245) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:17:14.797898 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78c264e2-7a8b-40f2-8958-48e84d674eb1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:17:14.798804 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78c264e2-7a8b-40f2-8958-48e84d674eb1 None admin] REQUEST_METHOD: `PUT` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:17:14.799084 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78c264e2-7a8b-40f2-8958-48e84d674eb1 None admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:17:14.799384 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78c264e2-7a8b-40f2-8958-48e84d674eb1 None admin] PATH_INFO: `/v3/projects/4faef13aa50e4e36b6c265fb1398fa14/users/9b0524aed8c74abd8949b860d6b93936/roles/50d1ec7f947f42acb65c57e07f2269d6` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:17:14.814904 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78c264e2-7a8b-40f2-8958-48e84d674eb1 None admin] RBAC: Authorizing `identity:create_grant(project_id=4faef13aa50e4e36b6c265fb1398fa14, user_id=9b0524aed8c74abd8949b860d6b93936, role_id=50d1ec7f947f42acb65c57e07f2269d6)` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:17:14.816945 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78c264e2-7a8b-40f2-8958-48e84d674eb1 None admin] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:17:14.835810 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 298/598] 10.222.0.47 () {58 vars in 1439 bytes} [Thu Jul 21 08:17:14 2022] PUT /identity/v3/projects/4faef13aa50e4e36b6c265fb1398fa14/users/9b0524aed8c74abd8949b860d6b93936/roles/50d1ec7f947f42acb65c57e07f2269d6 => generated 0 bytes in 185 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 08:17:14.835810 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cf788ac-e48c-46e4-be59-492a95ba2124 None None] REQUEST_METHOD: `GET` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:17:14.835810 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cf788ac-e48c-46e4-be59-492a95ba2124 None None] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:17:14.835810 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cf788ac-e48c-46e4-be59-492a95ba2124 None None] PATH_INFO: `/` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:17:14.835810 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 299/599] 10.222.0.47 () {54 vars in 926 bytes} [Thu Jul 21 08:17:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 08:17:14.835810 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cf788ac-e48c-46e4-be59-492a95ba2124 None None] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:17:14.856457 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG dbcounter [-] [80244] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80244) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:17:14.925078 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c91427a7-105b-430f-97a1-441a7dd9e0b0 None admin] Validating token access rules against request {{(pid=80245) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:17:14.969462 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dba0381e-02df-43bd-ae56-acce4334ca27 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80245) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:17:14.970292 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dba0381e-02df-43bd-ae56-acce4334ca27 None admin] REQUEST_METHOD: `GET` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:17:14.970571 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dba0381e-02df-43bd-ae56-acce4334ca27 None admin] SCRIPT_NAME: `/identity` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:17:14.970847 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dba0381e-02df-43bd-ae56-acce4334ca27 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:17:14.972438 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cf788ac-e48c-46e4-be59-492a95ba2124 None None] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:17:14.981074 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dba0381e-02df-43bd-ae56-acce4334ca27 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:17:14.982541 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dba0381e-02df-43bd-ae56-acce4334ca27 None admin] RBAC: Authorization granted {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:17:14.985457 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG dbcounter [-] [80245] Writing DB stats keystone:SELECT=100 {{(pid=80245) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:17:15.003166 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application [None req-dba0381e-02df-43bd-ae56-acce4334ca27 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 21 08:17:15.003166 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 08:17:15.003166 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 08:17:15.003166 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 08:17:15.003166 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 08:17:15.003166 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 08:17:15.003166 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 08:17:15.003166 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 08:17:15.003166 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 08:17:15.003166 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 08:17:15.003166 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 08:17:15.003166 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 08:17:15.003166 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 21 08:17:15.003166 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 21 08:17:15.003166 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 21 08:17:15.004832 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 21 08:17:15.004832 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 08:17:15.004832 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 08:17:15.004832 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 08:17:15.004832 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 08:17:15.004832 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 08:17:15.004832 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 08:17:15.004832 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 08:17:15.004832 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application with Lock( Jul 21 08:17:15.004832 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 08:17:15.004832 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application return self._enter() Jul 21 08:17:15.004832 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 08:17:15.004832 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 08:17:15.004832 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 08:17:15.004832 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application return self.creator() Jul 21 08:17:15.004832 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 08:17:15.004832 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application created_value = creator( Jul 21 08:17:15.005758 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 21 08:17:15.005758 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 21 08:17:15.005758 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 21 08:17:15.005758 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 21 08:17:15.005758 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 21 08:17:15.005758 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 21 08:17:15.005758 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 21 08:17:15.005758 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application Jul 21 08:17:15.005758 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: [pid: 80245|app: 0|req: 301/600] 10.222.0.47 () {56 vars in 1111 bytes} [Thu Jul 21 08:17:14 2022] GET /identity/v3/projects/service => generated 88 bytes in 213 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 08:17:15.006300 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3d67e93-b2fe-4db5-b668-23058550d006 None None] REQUEST_METHOD: `POST` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:17:15.006795 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3d67e93-b2fe-4db5-b668-23058550d006 None None] SCRIPT_NAME: `/identity` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:17:15.007091 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3d67e93-b2fe-4db5-b668-23058550d006 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:17:15.020982 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90deba76-275e-4cb1-a651-9ce58b5d8f73 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:17:15.021712 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90deba76-275e-4cb1-a651-9ce58b5d8f73 None admin] REQUEST_METHOD: `GET` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:17:15.022046 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90deba76-275e-4cb1-a651-9ce58b5d8f73 None admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:17:15.022317 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90deba76-275e-4cb1-a651-9ce58b5d8f73 None admin] PATH_INFO: `/v3/users` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:17:15.023159 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90deba76-275e-4cb1-a651-9ce58b5d8f73 None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=80244) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 08:17:15.023550 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90deba76-275e-4cb1-a651-9ce58b5d8f73 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:17:15.025060 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90deba76-275e-4cb1-a651-9ce58b5d8f73 None admin] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:17:15.027881 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.auth.core [None req-f3d67e93-b2fe-4db5-b668-23058550d006 None None] MFA Rules not processed for user `2dce40b8d74c41ec94cc36e6ff05d412`. Rule list: `[]` (Enabled: `True`). {{(pid=80245) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 08:17:15.029091 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.fernet_utils [None req-f3d67e93-b2fe-4db5-b668-23058550d006 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80245) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 08:17:15.046203 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 300/601] 10.222.0.47 () {56 vars in 1139 bytes} [Thu Jul 21 08:17:14 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 08:17:15.046910 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90deba76-275e-4cb1-a651-9ce58b5d8f73 None admin] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:17:15.117043 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: [pid: 80245|app: 0|req: 302/602] 10.222.0.47 () {58 vars in 1023 bytes} [Thu Jul 21 08:17:15 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 21 08:17:15.118567 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3d67e93-b2fe-4db5-b668-23058550d006 None None] Authenticating user token {{(pid=80245) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:17:15.139479 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG dbcounter [-] [80244] Writing DB stats keystone:SELECT=100 {{(pid=80244) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:17:15.175630 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90deba76-275e-4cb1-a651-9ce58b5d8f73 None admin] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:17:15.212673 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a9561ac-ebe1-45af-8b3c-1193932d450b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:17:15.213492 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a9561ac-ebe1-45af-8b3c-1193932d450b None admin] REQUEST_METHOD: `GET` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:17:15.213902 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a9561ac-ebe1-45af-8b3c-1193932d450b None admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:17:15.214523 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a9561ac-ebe1-45af-8b3c-1193932d450b None admin] PATH_INFO: `/v3/projects` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:17:15.215277 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a9561ac-ebe1-45af-8b3c-1193932d450b None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80244) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 08:17:15.215730 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a9561ac-ebe1-45af-8b3c-1193932d450b None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:17:15.217267 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a9561ac-ebe1-45af-8b3c-1193932d450b None admin] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:17:15.231038 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 301/603] 10.222.0.47 () {56 vars in 1148 bytes} [Thu Jul 21 08:17:15 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 08:17:15.232327 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a9561ac-ebe1-45af-8b3c-1193932d450b None admin] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:17:15.255301 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3d67e93-b2fe-4db5-b668-23058550d006 None None] Validating token access rules against request {{(pid=80245) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:17:15.295421 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG dbcounter [-] [80245] Writing DB stats keystone:SELECT=100 {{(pid=80245) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:17:15.306654 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a826bffc-7b8a-43fe-82e2-e3cea7d992e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80245) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:17:15.307929 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a826bffc-7b8a-43fe-82e2-e3cea7d992e3 None admin] REQUEST_METHOD: `GET` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:17:15.308837 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a826bffc-7b8a-43fe-82e2-e3cea7d992e3 None admin] SCRIPT_NAME: `/identity` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:17:15.309264 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a826bffc-7b8a-43fe-82e2-e3cea7d992e3 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:17:15.322796 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a826bffc-7b8a-43fe-82e2-e3cea7d992e3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:17:15.324831 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a826bffc-7b8a-43fe-82e2-e3cea7d992e3 None admin] RBAC: Authorization granted {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:17:15.345213 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application [None req-a826bffc-7b8a-43fe-82e2-e3cea7d992e3 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 08:17:15.345213 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 08:17:15.345213 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 08:17:15.345213 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 08:17:15.345213 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 08:17:15.345213 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 08:17:15.345213 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 08:17:15.345213 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 08:17:15.345213 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 08:17:15.345213 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 08:17:15.345213 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 08:17:15.345213 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 08:17:15.345213 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 21 08:17:15.345213 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 21 08:17:15.345213 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 21 08:17:15.346315 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 21 08:17:15.346315 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 08:17:15.346315 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 08:17:15.346315 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 08:17:15.346315 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 08:17:15.346315 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 08:17:15.346315 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 08:17:15.346315 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 08:17:15.346315 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application with Lock( Jul 21 08:17:15.346315 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 08:17:15.346315 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application return self._enter() Jul 21 08:17:15.346315 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 08:17:15.346315 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 08:17:15.346315 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 08:17:15.346315 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application return self.creator() Jul 21 08:17:15.346315 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 08:17:15.346315 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application created_value = creator( Jul 21 08:17:15.347364 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 21 08:17:15.347364 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 21 08:17:15.347364 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 08:17:15.347364 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application Jul 21 08:17:15.349537 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: [pid: 80245|app: 0|req: 303/604] 10.222.0.47 () {56 vars in 1108 bytes} [Thu Jul 21 08:17:15 2022] GET /identity/v3/domains/Default => generated 87 bytes in 231 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 08:17:15.350517 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a826bffc-7b8a-43fe-82e2-e3cea7d992e3 None admin] Authenticating user token {{(pid=80245) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:17:15.355635 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a9561ac-ebe1-45af-8b3c-1193932d450b None admin] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:17:15.406221 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG dbcounter [-] [80244] Writing DB stats keystone:SELECT=100 {{(pid=80244) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:17:15.412184 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa89ce7d-6bc6-462f-8b18-1ed0c21690fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:17:15.414308 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa89ce7d-6bc6-462f-8b18-1ed0c21690fa None admin] REQUEST_METHOD: `GET` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:17:15.414776 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa89ce7d-6bc6-462f-8b18-1ed0c21690fa None admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:17:15.415337 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa89ce7d-6bc6-462f-8b18-1ed0c21690fa None admin] PATH_INFO: `/v3/roles/service` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:17:15.430464 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa89ce7d-6bc6-462f-8b18-1ed0c21690fa None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 21 08:17:15.440630 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application [None req-fa89ce7d-6bc6-462f-8b18-1ed0c21690fa None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 21 08:17:15.440630 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 08:17:15.440630 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 08:17:15.440630 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 08:17:15.440630 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 08:17:15.440630 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 08:17:15.440630 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 08:17:15.440630 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 08:17:15.440630 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 08:17:15.440630 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 08:17:15.440630 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 08:17:15.440630 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 08:17:15.440630 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 21 08:17:15.440630 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 21 08:17:15.440630 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 21 08:17:15.440630 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 21 08:17:15.441771 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 21 08:17:15.441771 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 21 08:17:15.441771 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 21 08:17:15.441771 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 21 08:17:15.441771 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 08:17:15.441771 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 08:17:15.441771 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 08:17:15.441771 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 08:17:15.441771 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 08:17:15.441771 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 08:17:15.441771 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 08:17:15.441771 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application with Lock( Jul 21 08:17:15.441771 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 08:17:15.441771 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self._enter() Jul 21 08:17:15.441771 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 08:17:15.441771 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 08:17:15.442807 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 08:17:15.442807 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self.creator() Jul 21 08:17:15.442807 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 08:17:15.442807 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application created_value = creator( Jul 21 08:17:15.442807 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 21 08:17:15.442807 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 21 08:17:15.442807 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 21 08:17:15.442807 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 21 08:17:15.442807 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 21 08:17:15.442807 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 21 08:17:15.442807 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 21 08:17:15.442807 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application Jul 21 08:17:15.445852 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 302/605] 10.222.0.47 () {56 vars in 1102 bytes} [Thu Jul 21 08:17:15 2022] GET /identity/v3/roles/service => generated 85 bytes in 214 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 08:17:15.446834 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa89ce7d-6bc6-462f-8b18-1ed0c21690fa None admin] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:17:15.472276 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a826bffc-7b8a-43fe-82e2-e3cea7d992e3 None admin] Validating token access rules against request {{(pid=80245) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:17:15.518372 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4334221c-9c4c-4aa7-8895-7b9832ac0657 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80245) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:17:15.519824 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4334221c-9c4c-4aa7-8895-7b9832ac0657 None admin] REQUEST_METHOD: `PUT` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:17:15.521133 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4334221c-9c4c-4aa7-8895-7b9832ac0657 None admin] SCRIPT_NAME: `/identity` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:17:15.521555 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4334221c-9c4c-4aa7-8895-7b9832ac0657 None admin] PATH_INFO: `/v3/projects/4faef13aa50e4e36b6c265fb1398fa14/users/b4f40410b7674687ab5362ac982248ef/roles/50d1ec7f947f42acb65c57e07f2269d6` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:17:15.544747 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4334221c-9c4c-4aa7-8895-7b9832ac0657 None admin] RBAC: Authorizing `identity:create_grant(project_id=4faef13aa50e4e36b6c265fb1398fa14, user_id=b4f40410b7674687ab5362ac982248ef, role_id=50d1ec7f947f42acb65c57e07f2269d6)` {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:17:15.547430 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4334221c-9c4c-4aa7-8895-7b9832ac0657 None admin] RBAC: Authorization granted {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:17:15.566873 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: [pid: 80245|app: 0|req: 304/606] 10.222.0.47 () {58 vars in 1439 bytes} [Thu Jul 21 08:17:15 2022] PUT /identity/v3/projects/4faef13aa50e4e36b6c265fb1398fa14/users/b4f40410b7674687ab5362ac982248ef/roles/50d1ec7f947f42acb65c57e07f2269d6 => generated 0 bytes in 216 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 08:17:15.568180 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4334221c-9c4c-4aa7-8895-7b9832ac0657 None admin] Authenticating user token {{(pid=80245) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:17:15.588854 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa89ce7d-6bc6-462f-8b18-1ed0c21690fa None admin] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:17:15.608425 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG dbcounter [-] [80245] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80245) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:17:15.628166 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d529662-729f-4ca2-9916-b5cbe3a6107e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:17:15.629010 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d529662-729f-4ca2-9916-b5cbe3a6107e None admin] REQUEST_METHOD: `GET` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:17:15.629445 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d529662-729f-4ca2-9916-b5cbe3a6107e None admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:17:15.629843 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d529662-729f-4ca2-9916-b5cbe3a6107e None admin] PATH_INFO: `/v3/domains` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:17:15.630727 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d529662-729f-4ca2-9916-b5cbe3a6107e None admin] RBAC: Adding query filter params (name=Default) {{(pid=80244) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 08:17:15.631145 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d529662-729f-4ca2-9916-b5cbe3a6107e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:17:15.632745 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d529662-729f-4ca2-9916-b5cbe3a6107e None admin] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:17:15.647361 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 303/607] 10.222.0.47 () {56 vars in 1109 bytes} [Thu Jul 21 08:17:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 08:17:15.652781 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d529662-729f-4ca2-9916-b5cbe3a6107e None admin] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:17:15.696105 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4334221c-9c4c-4aa7-8895-7b9832ac0657 None admin] Validating token access rules against request {{(pid=80245) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:17:15.718690 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG dbcounter [-] [80244] Writing DB stats keystone:SELECT=100 {{(pid=80244) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:17:15.735865 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0474fd2-80bd-4d13-b45e-00d52ede8ee0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80245) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:17:15.736873 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0474fd2-80bd-4d13-b45e-00d52ede8ee0 None admin] REQUEST_METHOD: `GET` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:17:15.737310 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0474fd2-80bd-4d13-b45e-00d52ede8ee0 None admin] SCRIPT_NAME: `/identity` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:17:15.737750 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0474fd2-80bd-4d13-b45e-00d52ede8ee0 None admin] PATH_INFO: `/v3/roles` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:17:15.738991 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0474fd2-80bd-4d13-b45e-00d52ede8ee0 None admin] RBAC: Adding query filter params (name=service) {{(pid=80245) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 08:17:15.739511 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0474fd2-80bd-4d13-b45e-00d52ede8ee0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:17:15.741062 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0474fd2-80bd-4d13-b45e-00d52ede8ee0 None admin] RBAC: Authorization granted {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:17:15.750676 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: [pid: 80245|app: 0|req: 305/608] 10.222.0.47 () {56 vars in 1103 bytes} [Thu Jul 21 08:17:15 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 08:17:15.760136 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0474fd2-80bd-4d13-b45e-00d52ede8ee0 None admin] Authenticating user token {{(pid=80245) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:17:15.780006 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d529662-729f-4ca2-9916-b5cbe3a6107e None admin] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:17:15.828483 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12803b7e-de3a-4c8b-8007-d7581778f9f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:17:15.829347 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12803b7e-de3a-4c8b-8007-d7581778f9f5 None admin] REQUEST_METHOD: `GET` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:17:15.829636 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12803b7e-de3a-4c8b-8007-d7581778f9f5 None admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:17:15.829913 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12803b7e-de3a-4c8b-8007-d7581778f9f5 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:17:15.838032 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12803b7e-de3a-4c8b-8007-d7581778f9f5 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:17:15.839539 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12803b7e-de3a-4c8b-8007-d7581778f9f5 None admin] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:17:15.853924 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application [None req-12803b7e-de3a-4c8b-8007-d7581778f9f5 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 21 08:17:15.853924 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 08:17:15.853924 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 08:17:15.853924 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 08:17:15.853924 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 08:17:15.853924 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 08:17:15.853924 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 08:17:15.853924 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 08:17:15.853924 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 08:17:15.853924 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 08:17:15.853924 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 08:17:15.853924 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 08:17:15.853924 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 21 08:17:15.853924 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 21 08:17:15.853924 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 21 08:17:15.854985 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 21 08:17:15.854985 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 08:17:15.854985 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 08:17:15.854985 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 08:17:15.854985 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 08:17:15.854985 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 08:17:15.854985 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 08:17:15.854985 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 08:17:15.854985 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application with Lock( Jul 21 08:17:15.854985 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 08:17:15.854985 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self._enter() Jul 21 08:17:15.854985 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 08:17:15.854985 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 08:17:15.854985 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 08:17:15.854985 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self.creator() Jul 21 08:17:15.854985 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 08:17:15.854985 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application created_value = creator( Jul 21 08:17:15.855886 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 21 08:17:15.855886 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 21 08:17:15.855886 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 21 08:17:15.855886 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 21 08:17:15.855886 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 21 08:17:15.855886 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 21 08:17:15.855886 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 21 08:17:15.855886 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application Jul 21 08:17:15.855886 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 304/609] 10.222.0.47 () {56 vars in 1111 bytes} [Thu Jul 21 08:17:15 2022] GET /identity/v3/projects/service => generated 88 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 08:17:15.861205 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12803b7e-de3a-4c8b-8007-d7581778f9f5 None admin] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:17:15.883014 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG dbcounter [-] [80245] Writing DB stats keystone:SELECT=100 {{(pid=80245) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:17:15.894433 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0474fd2-80bd-4d13-b45e-00d52ede8ee0 None admin] Validating token access rules against request {{(pid=80245) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:17:15.933906 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e34b26d6-2bcd-4815-941f-2ab4665e2767 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80245) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:17:15.934860 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e34b26d6-2bcd-4815-941f-2ab4665e2767 None admin] REQUEST_METHOD: `GET` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:17:15.935112 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e34b26d6-2bcd-4815-941f-2ab4665e2767 None admin] SCRIPT_NAME: `/identity` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:17:15.935429 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e34b26d6-2bcd-4815-941f-2ab4665e2767 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:17:15.948358 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e34b26d6-2bcd-4815-941f-2ab4665e2767 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:17:15.949718 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e34b26d6-2bcd-4815-941f-2ab4665e2767 None admin] RBAC: Authorization granted {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:17:15.969264 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application [None req-e34b26d6-2bcd-4815-941f-2ab4665e2767 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 08:17:15.969264 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 08:17:15.969264 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 08:17:15.969264 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 08:17:15.969264 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 08:17:15.969264 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 08:17:15.969264 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 08:17:15.969264 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 08:17:15.969264 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 08:17:15.969264 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 08:17:15.969264 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 08:17:15.969264 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 08:17:15.969264 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 21 08:17:15.969264 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 21 08:17:15.969264 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 21 08:17:15.970179 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 21 08:17:15.970179 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 08:17:15.970179 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 08:17:15.970179 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 08:17:15.970179 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 08:17:15.970179 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 08:17:15.970179 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 08:17:15.970179 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 08:17:15.970179 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application with Lock( Jul 21 08:17:15.970179 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 08:17:15.970179 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application return self._enter() Jul 21 08:17:15.970179 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 08:17:15.970179 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 08:17:15.970179 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 08:17:15.970179 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application return self.creator() Jul 21 08:17:15.970179 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 08:17:15.970179 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application created_value = creator( Jul 21 08:17:15.971039 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 21 08:17:15.971039 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 21 08:17:15.971039 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 08:17:15.971039 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application Jul 21 08:17:15.971039 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: [pid: 80245|app: 0|req: 306/610] 10.222.0.47 () {56 vars in 1108 bytes} [Thu Jul 21 08:17:15 2022] GET /identity/v3/domains/Default => generated 87 bytes in 212 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 21 08:17:15.976684 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e34b26d6-2bcd-4815-941f-2ab4665e2767 None admin] Authenticating user token {{(pid=80245) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:17:15.987127 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12803b7e-de3a-4c8b-8007-d7581778f9f5 None admin] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:17:16.000919 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG dbcounter [-] [80244] Writing DB stats keystone:SELECT=100 {{(pid=80244) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:17:16.036531 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daa26643-5183-4292-8fa1-101d261a241c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:17:16.037206 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daa26643-5183-4292-8fa1-101d261a241c None admin] REQUEST_METHOD: `GET` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:17:16.037488 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daa26643-5183-4292-8fa1-101d261a241c None admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:17:16.037760 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daa26643-5183-4292-8fa1-101d261a241c None admin] PATH_INFO: `/v3/projects` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:17:16.038551 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daa26643-5183-4292-8fa1-101d261a241c None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80244) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 08:17:16.038854 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daa26643-5183-4292-8fa1-101d261a241c None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:17:16.041308 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daa26643-5183-4292-8fa1-101d261a241c None admin] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:17:16.061254 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 305/611] 10.222.0.47 () {56 vars in 1148 bytes} [Thu Jul 21 08:17:15 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 21 08:17:16.062604 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83f7cc71-1751-455d-9f2b-1292e7de70d5 None None] REQUEST_METHOD: `GET` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:17:16.062893 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83f7cc71-1751-455d-9f2b-1292e7de70d5 None None] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:17:16.063170 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83f7cc71-1751-455d-9f2b-1292e7de70d5 None None] PATH_INFO: `/` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:17:16.064861 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 306/612] 10.222.0.47 () {54 vars in 926 bytes} [Thu Jul 21 08:17:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 08:17:16.069864 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83f7cc71-1751-455d-9f2b-1292e7de70d5 None None] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:17:16.121690 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e34b26d6-2bcd-4815-941f-2ab4665e2767 None admin] Validating token access rules against request {{(pid=80245) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:17:16.158898 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-731066ad-e3c9-4480-9aa5-9165025a6f61 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80245) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:17:16.159633 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-731066ad-e3c9-4480-9aa5-9165025a6f61 None admin] REQUEST_METHOD: `GET` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:17:16.159933 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-731066ad-e3c9-4480-9aa5-9165025a6f61 None admin] SCRIPT_NAME: `/identity` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:17:16.160190 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-731066ad-e3c9-4480-9aa5-9165025a6f61 None admin] PATH_INFO: `/v3/domains` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:17:16.160977 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-731066ad-e3c9-4480-9aa5-9165025a6f61 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80245) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 08:17:16.161286 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-731066ad-e3c9-4480-9aa5-9165025a6f61 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:17:16.162719 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-731066ad-e3c9-4480-9aa5-9165025a6f61 None admin] RBAC: Authorization granted {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:17:16.181603 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: [pid: 80245|app: 0|req: 307/613] 10.222.0.47 () {56 vars in 1109 bytes} [Thu Jul 21 08:17:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 08:17:16.183150 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87748277-8419-4157-a28c-9aa768959c83 None None] REQUEST_METHOD: `POST` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:17:16.183372 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87748277-8419-4157-a28c-9aa768959c83 None None] SCRIPT_NAME: `/identity` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:17:16.183671 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87748277-8419-4157-a28c-9aa768959c83 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:17:16.187594 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83f7cc71-1751-455d-9f2b-1292e7de70d5 None None] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:17:16.189661 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG dbcounter [-] [80245] Writing DB stats keystone:SELECT=100 {{(pid=80245) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:17:16.203024 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.auth.core [None req-87748277-8419-4157-a28c-9aa768959c83 None None] MFA Rules not processed for user `2dce40b8d74c41ec94cc36e6ff05d412`. Rule list: `[]` (Enabled: `True`). {{(pid=80245) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 08:17:16.204382 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.fernet_utils [None req-87748277-8419-4157-a28c-9aa768959c83 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80245) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 08:17:16.226699 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f785f47-edd5-4b25-8719-f3951f8746c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:17:16.228045 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f785f47-edd5-4b25-8719-f3951f8746c8 None admin] REQUEST_METHOD: `PUT` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:17:16.228560 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f785f47-edd5-4b25-8719-f3951f8746c8 None admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:17:16.229042 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f785f47-edd5-4b25-8719-f3951f8746c8 None admin] PATH_INFO: `/v3/projects/4faef13aa50e4e36b6c265fb1398fa14/users/20ca3c3bd3774691b3374cfbfec6e494/roles/50d1ec7f947f42acb65c57e07f2269d6` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:17:16.248382 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f785f47-edd5-4b25-8719-f3951f8746c8 None admin] RBAC: Authorizing `identity:create_grant(project_id=4faef13aa50e4e36b6c265fb1398fa14, user_id=20ca3c3bd3774691b3374cfbfec6e494, role_id=50d1ec7f947f42acb65c57e07f2269d6)` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:17:16.250193 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f785f47-edd5-4b25-8719-f3951f8746c8 None admin] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:17:16.268795 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 307/614] 10.222.0.47 () {58 vars in 1439 bytes} [Thu Jul 21 08:17:16 2022] PUT /identity/v3/projects/4faef13aa50e4e36b6c265fb1398fa14/users/20ca3c3bd3774691b3374cfbfec6e494/roles/50d1ec7f947f42acb65c57e07f2269d6 => generated 0 bytes in 199 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 08:17:16.270139 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f785f47-edd5-4b25-8719-f3951f8746c8 None admin] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:17:16.300787 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG dbcounter [-] [80244] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80244) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:17:16.309509 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: [pid: 80245|app: 0|req: 308/615] 10.222.0.47 () {58 vars in 1023 bytes} [Thu Jul 21 08:17:16 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 21 08:17:16.343662 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87748277-8419-4157-a28c-9aa768959c83 None None] Authenticating user token {{(pid=80245) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:17:16.400026 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f785f47-edd5-4b25-8719-f3951f8746c8 None admin] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:17:16.463363 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb7b9ee2-3b16-49de-8a75-5b2c04b2c4e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:17:16.464192 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb7b9ee2-3b16-49de-8a75-5b2c04b2c4e1 None admin] REQUEST_METHOD: `GET` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:17:16.464477 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb7b9ee2-3b16-49de-8a75-5b2c04b2c4e1 None admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:17:16.464753 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb7b9ee2-3b16-49de-8a75-5b2c04b2c4e1 None admin] PATH_INFO: `/v3/users/swift` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:17:16.486648 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb7b9ee2-3b16-49de-8a75-5b2c04b2c4e1 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:17:16.489316 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb7b9ee2-3b16-49de-8a75-5b2c04b2c4e1 None admin] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:17:16.510381 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application [None req-cb7b9ee2-3b16-49de-8a75-5b2c04b2c4e1 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Jul 21 08:17:16.510381 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 08:17:16.510381 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 08:17:16.510381 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 08:17:16.510381 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 08:17:16.510381 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 08:17:16.510381 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 08:17:16.510381 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 08:17:16.510381 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 08:17:16.510381 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 08:17:16.510381 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 08:17:16.510381 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 08:17:16.510381 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 21 08:17:16.510381 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 21 08:17:16.510381 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 21 08:17:16.510381 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 21 08:17:16.511340 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 08:17:16.511340 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 08:17:16.511340 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 21 08:17:16.511340 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 21 08:17:16.511340 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 21 08:17:16.511340 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 21 08:17:16.511340 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 08:17:16.511340 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 08:17:16.511340 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 08:17:16.511340 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 08:17:16.511340 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 08:17:16.511340 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application with Lock( Jul 21 08:17:16.511340 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 08:17:16.511340 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self._enter() Jul 21 08:17:16.511340 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 08:17:16.511340 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 08:17:16.511340 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 08:17:16.511340 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self.creator() Jul 21 08:17:16.512425 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 08:17:16.512425 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application created_value = creator( Jul 21 08:17:16.512425 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 21 08:17:16.512425 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 21 08:17:16.512425 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 21 08:17:16.512425 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 21 08:17:16.512425 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 21 08:17:16.512425 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 21 08:17:16.512425 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Jul 21 08:17:16.512425 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application Jul 21 08:17:16.512425 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 308/616] 10.222.0.47 () {56 vars in 1096 bytes} [Thu Jul 21 08:17:16 2022] GET /identity/v3/users/swift => generated 83 bytes in 242 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 08:17:16.518798 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb7b9ee2-3b16-49de-8a75-5b2c04b2c4e1 None admin] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:17:16.539297 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87748277-8419-4157-a28c-9aa768959c83 None None] Validating token access rules against request {{(pid=80245) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:17:16.582515 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2b00ef0-4575-4f4d-848b-7ed9a2e274ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80245) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:17:16.583638 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2b00ef0-4575-4f4d-848b-7ed9a2e274ca None admin] REQUEST_METHOD: `GET` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:17:16.584067 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2b00ef0-4575-4f4d-848b-7ed9a2e274ca None admin] SCRIPT_NAME: `/identity` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:17:16.584487 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2b00ef0-4575-4f4d-848b-7ed9a2e274ca None admin] PATH_INFO: `/v3/roles/service` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:17:16.587594 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG dbcounter [-] [80245] Writing DB stats keystone:SELECT=100 {{(pid=80245) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:17:16.598949 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2b00ef0-4575-4f4d-848b-7ed9a2e274ca None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 21 08:17:16.607091 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application [None req-a2b00ef0-4575-4f4d-848b-7ed9a2e274ca None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 21 08:17:16.607091 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 08:17:16.607091 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 08:17:16.607091 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 08:17:16.607091 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 08:17:16.607091 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 08:17:16.607091 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 08:17:16.607091 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 08:17:16.607091 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 08:17:16.607091 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 08:17:16.607091 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 08:17:16.607091 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 08:17:16.607091 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 21 08:17:16.607091 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 21 08:17:16.607091 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 21 08:17:16.607091 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 21 08:17:16.608234 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 21 08:17:16.608234 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 21 08:17:16.608234 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 21 08:17:16.608234 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 21 08:17:16.608234 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 08:17:16.608234 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 08:17:16.608234 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 08:17:16.608234 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 08:17:16.608234 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 08:17:16.608234 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 08:17:16.608234 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 08:17:16.608234 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application with Lock( Jul 21 08:17:16.608234 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 08:17:16.608234 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application return self._enter() Jul 21 08:17:16.608234 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 08:17:16.608234 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 08:17:16.609254 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 08:17:16.609254 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application return self.creator() Jul 21 08:17:16.609254 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 08:17:16.609254 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application created_value = creator( Jul 21 08:17:16.609254 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 21 08:17:16.609254 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 21 08:17:16.609254 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 21 08:17:16.609254 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 21 08:17:16.609254 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 21 08:17:16.609254 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 21 08:17:16.609254 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 21 08:17:16.609254 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application Jul 21 08:17:16.611696 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: [pid: 80245|app: 0|req: 309/617] 10.222.0.47 () {56 vars in 1102 bytes} [Thu Jul 21 08:17:16 2022] GET /identity/v3/roles/service => generated 85 bytes in 268 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 21 08:17:16.617378 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2b00ef0-4575-4f4d-848b-7ed9a2e274ca None admin] Authenticating user token {{(pid=80245) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:17:16.627968 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG dbcounter [-] [80244] Writing DB stats keystone:SELECT=100 {{(pid=80244) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:17:16.656441 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb7b9ee2-3b16-49de-8a75-5b2c04b2c4e1 None admin] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:17:16.713145 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1340f2e-d2dd-47dc-8da7-10aa97d23851 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:17:16.713837 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1340f2e-d2dd-47dc-8da7-10aa97d23851 None admin] REQUEST_METHOD: `GET` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:17:16.714126 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1340f2e-d2dd-47dc-8da7-10aa97d23851 None admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:17:16.714406 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1340f2e-d2dd-47dc-8da7-10aa97d23851 None admin] PATH_INFO: `/v3/users` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:17:16.715260 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1340f2e-d2dd-47dc-8da7-10aa97d23851 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80244) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 08:17:16.715599 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1340f2e-d2dd-47dc-8da7-10aa97d23851 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:17:16.717057 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1340f2e-d2dd-47dc-8da7-10aa97d23851 None admin] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:17:16.730990 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2b00ef0-4575-4f4d-848b-7ed9a2e274ca None admin] Validating token access rules against request {{(pid=80245) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:17:16.757176 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 309/618] 10.222.0.47 () {56 vars in 1135 bytes} [Thu Jul 21 08:17:16 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 234 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 08:17:16.758547 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1340f2e-d2dd-47dc-8da7-10aa97d23851 None admin] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:17:16.766833 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca5b79c9-1945-40a4-b0cf-dba4b1703288 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80245) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:17:16.767495 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca5b79c9-1945-40a4-b0cf-dba4b1703288 None admin] REQUEST_METHOD: `GET` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:17:16.767810 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca5b79c9-1945-40a4-b0cf-dba4b1703288 None admin] SCRIPT_NAME: `/identity` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:17:16.768066 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca5b79c9-1945-40a4-b0cf-dba4b1703288 None admin] PATH_INFO: `/v3/roles` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:17:16.768823 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca5b79c9-1945-40a4-b0cf-dba4b1703288 None admin] RBAC: Adding query filter params (name=service) {{(pid=80245) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 08:17:16.769181 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca5b79c9-1945-40a4-b0cf-dba4b1703288 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:17:16.771164 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca5b79c9-1945-40a4-b0cf-dba4b1703288 None admin] RBAC: Authorization granted {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:17:16.780221 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: [pid: 80245|app: 0|req: 310/619] 10.222.0.47 () {56 vars in 1103 bytes} [Thu Jul 21 08:17:16 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 08:17:16.785664 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca5b79c9-1945-40a4-b0cf-dba4b1703288 None admin] Authenticating user token {{(pid=80245) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:17:16.861002 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG dbcounter [-] [80245] Writing DB stats keystone:SELECT=100 {{(pid=80245) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:17:16.907530 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca5b79c9-1945-40a4-b0cf-dba4b1703288 None admin] Validating token access rules against request {{(pid=80245) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:17:16.952329 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26312dca-90ee-4456-b925-f694a5147809 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80245) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:17:16.953249 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26312dca-90ee-4456-b925-f694a5147809 None admin] REQUEST_METHOD: `GET` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:17:16.953541 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26312dca-90ee-4456-b925-f694a5147809 None admin] SCRIPT_NAME: `/identity` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:17:16.953821 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26312dca-90ee-4456-b925-f694a5147809 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:17:16.965336 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26312dca-90ee-4456-b925-f694a5147809 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:17:16.966827 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26312dca-90ee-4456-b925-f694a5147809 None admin] RBAC: Authorization granted {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:17:16.985017 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application [None req-26312dca-90ee-4456-b925-f694a5147809 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 08:17:16.985017 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 08:17:16.985017 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 08:17:16.985017 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 08:17:16.985017 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 08:17:16.985017 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 08:17:16.985017 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 08:17:16.985017 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 08:17:16.985017 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 08:17:16.985017 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 08:17:16.985017 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 08:17:16.985017 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 08:17:16.985017 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 21 08:17:16.985017 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 21 08:17:16.985017 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 21 08:17:16.986142 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 21 08:17:16.986142 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 08:17:16.986142 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 08:17:16.986142 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 08:17:16.986142 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 08:17:16.986142 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 08:17:16.986142 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 08:17:16.986142 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 08:17:16.986142 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application with Lock( Jul 21 08:17:16.986142 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 08:17:16.986142 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application return self._enter() Jul 21 08:17:16.986142 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 08:17:16.986142 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 08:17:16.986142 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 08:17:16.986142 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application return self.creator() Jul 21 08:17:16.986142 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 08:17:16.986142 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application created_value = creator( Jul 21 08:17:16.987054 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 21 08:17:16.987054 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 21 08:17:16.987054 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 08:17:16.987054 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application Jul 21 08:17:16.987054 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: [pid: 80245|app: 0|req: 311/620] 10.222.0.47 () {56 vars in 1108 bytes} [Thu Jul 21 08:17:16 2022] GET /identity/v3/domains/Default => generated 87 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 21 08:17:16.996238 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26312dca-90ee-4456-b925-f694a5147809 None admin] Authenticating user token {{(pid=80245) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:17:17.033387 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1340f2e-d2dd-47dc-8da7-10aa97d23851 None admin] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:17:17.111427 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26312dca-90ee-4456-b925-f694a5147809 None admin] Validating token access rules against request {{(pid=80245) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:17:17.124052 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG dbcounter [-] [80245] Writing DB stats keystone:SELECT=100 {{(pid=80245) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:17:17.141025 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6a460bb-f895-493f-9f9d-6f0bf72f997b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:17:17.141828 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6a460bb-f895-493f-9f9d-6f0bf72f997b None admin] REQUEST_METHOD: `GET` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:17:17.142112 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6a460bb-f895-493f-9f9d-6f0bf72f997b None admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:17:17.142391 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6a460bb-f895-493f-9f9d-6f0bf72f997b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:17:17.159819 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG dbcounter [-] [80244] Writing DB stats keystone:SELECT=100 {{(pid=80244) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:17:17.161367 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1bdfd29-8cde-4e1f-b3e7-00d00dda536f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80245) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:17:17.162068 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1bdfd29-8cde-4e1f-b3e7-00d00dda536f None admin] REQUEST_METHOD: `GET` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:17:17.162355 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1bdfd29-8cde-4e1f-b3e7-00d00dda536f None admin] SCRIPT_NAME: `/identity` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:17:17.162630 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1bdfd29-8cde-4e1f-b3e7-00d00dda536f None admin] PATH_INFO: `/v3/domains` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:17:17.163409 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1bdfd29-8cde-4e1f-b3e7-00d00dda536f None admin] RBAC: Adding query filter params (name=Default) {{(pid=80245) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 08:17:17.163769 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1bdfd29-8cde-4e1f-b3e7-00d00dda536f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:17:17.165167 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1bdfd29-8cde-4e1f-b3e7-00d00dda536f None admin] RBAC: Authorization granted {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:17:17.167919 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6a460bb-f895-493f-9f9d-6f0bf72f997b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:17:17.169415 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6a460bb-f895-493f-9f9d-6f0bf72f997b None admin] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:17:17.192419 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: [pid: 80245|app: 0|req: 312/621] 10.222.0.47 () {56 vars in 1109 bytes} [Thu Jul 21 08:17:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 21 08:17:17.200422 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application [None req-c6a460bb-f895-493f-9f9d-6f0bf72f997b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 08:17:17.200422 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 08:17:17.200422 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 08:17:17.200422 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 08:17:17.200422 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 08:17:17.200422 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 08:17:17.200422 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 08:17:17.200422 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 08:17:17.200422 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 08:17:17.200422 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 08:17:17.200422 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 08:17:17.200422 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 08:17:17.200422 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 21 08:17:17.200422 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 21 08:17:17.200422 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 21 08:17:17.201282 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 21 08:17:17.201282 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 08:17:17.201282 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 08:17:17.201282 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 08:17:17.201282 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 08:17:17.201282 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 08:17:17.201282 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 08:17:17.201282 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 08:17:17.201282 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application with Lock( Jul 21 08:17:17.201282 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 08:17:17.201282 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self._enter() Jul 21 08:17:17.201282 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 08:17:17.201282 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 08:17:17.201282 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 08:17:17.201282 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self.creator() Jul 21 08:17:17.201282 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 08:17:17.201282 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application created_value = creator( Jul 21 08:17:17.202145 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 21 08:17:17.202145 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 21 08:17:17.202145 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 08:17:17.202145 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application Jul 21 08:17:17.202145 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 310/622] 10.222.0.47 () {56 vars in 1108 bytes} [Thu Jul 21 08:17:16 2022] GET /identity/v3/domains/Default => generated 87 bytes in 444 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 21 08:17:17.202863 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1bdfd29-8cde-4e1f-b3e7-00d00dda536f None admin] Authenticating user token {{(pid=80245) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:17:17.213133 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6a460bb-f895-493f-9f9d-6f0bf72f997b None admin] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:17:17.325686 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1bdfd29-8cde-4e1f-b3e7-00d00dda536f None admin] Validating token access rules against request {{(pid=80245) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:17:17.356070 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6a460bb-f895-493f-9f9d-6f0bf72f997b None admin] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:17:17.362251 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fb9aa74-cddd-4f1a-a9cf-e2bc9858508a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80245) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:17:17.363062 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fb9aa74-cddd-4f1a-a9cf-e2bc9858508a None admin] REQUEST_METHOD: `GET` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:17:17.363371 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fb9aa74-cddd-4f1a-a9cf-e2bc9858508a None admin] SCRIPT_NAME: `/identity` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:17:17.364192 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fb9aa74-cddd-4f1a-a9cf-e2bc9858508a None admin] PATH_INFO: `/v3/users/nova` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:17:17.371914 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fb9aa74-cddd-4f1a-a9cf-e2bc9858508a None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:17:17.373362 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fb9aa74-cddd-4f1a-a9cf-e2bc9858508a None admin] RBAC: Authorization granted {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:17:17.388281 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application [None req-4fb9aa74-cddd-4f1a-a9cf-e2bc9858508a None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jul 21 08:17:17.388281 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 08:17:17.388281 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 08:17:17.388281 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 08:17:17.388281 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 08:17:17.388281 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 08:17:17.388281 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 08:17:17.388281 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 08:17:17.388281 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 08:17:17.388281 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 08:17:17.388281 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 08:17:17.388281 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 08:17:17.388281 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 21 08:17:17.388281 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 21 08:17:17.388281 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 21 08:17:17.388281 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 21 08:17:17.390381 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 08:17:17.390381 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 08:17:17.390381 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 21 08:17:17.390381 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 21 08:17:17.390381 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 21 08:17:17.390381 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 21 08:17:17.390381 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 08:17:17.390381 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 08:17:17.390381 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 08:17:17.390381 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 08:17:17.390381 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 08:17:17.390381 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application with Lock( Jul 21 08:17:17.390381 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 08:17:17.390381 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application return self._enter() Jul 21 08:17:17.390381 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 08:17:17.390381 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 08:17:17.390381 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 08:17:17.390381 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application return self.creator() Jul 21 08:17:17.392619 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 08:17:17.392619 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application created_value = creator( Jul 21 08:17:17.392619 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 21 08:17:17.392619 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 21 08:17:17.392619 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 21 08:17:17.392619 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 21 08:17:17.392619 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 21 08:17:17.392619 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 21 08:17:17.392619 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jul 21 08:17:17.392619 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application Jul 21 08:17:17.392619 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: [pid: 80245|app: 0|req: 313/623] 10.222.0.47 () {56 vars in 1093 bytes} [Thu Jul 21 08:17:17 2022] GET /identity/v3/users/nova => generated 82 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 08:17:17.392619 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01b05de1-2c57-47c7-ac2e-1269c542dd23 None None] REQUEST_METHOD: `GET` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:17:17.392619 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01b05de1-2c57-47c7-ac2e-1269c542dd23 None None] SCRIPT_NAME: `/identity` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:17:17.393749 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01b05de1-2c57-47c7-ac2e-1269c542dd23 None None] PATH_INFO: `/` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:17:17.393749 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: [pid: 80245|app: 0|req: 314/624] 10.222.0.47 () {54 vars in 926 bytes} [Thu Jul 21 08:17:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 08:17:17.393912 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37876a44-3060-4a87-9744-fc749e91037e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:17:17.403445 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37876a44-3060-4a87-9744-fc749e91037e None admin] REQUEST_METHOD: `GET` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:17:17.403445 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37876a44-3060-4a87-9744-fc749e91037e None admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:17:17.403778 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01b05de1-2c57-47c7-ac2e-1269c542dd23 None None] Authenticating user token {{(pid=80245) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:17:17.403926 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37876a44-3060-4a87-9744-fc749e91037e None admin] PATH_INFO: `/v3/domains` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:17:17.403926 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37876a44-3060-4a87-9744-fc749e91037e None admin] RBAC: Adding query filter params (name=Default) {{(pid=80244) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 08:17:17.403926 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37876a44-3060-4a87-9744-fc749e91037e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:17:17.403926 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37876a44-3060-4a87-9744-fc749e91037e None admin] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:17:17.412245 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 311/625] 10.222.0.47 () {56 vars in 1109 bytes} [Thu Jul 21 08:17:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 08:17:17.413635 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12275320-1228-4213-ac4e-23828c3811ec None None] REQUEST_METHOD: `POST` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:17:17.414063 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12275320-1228-4213-ac4e-23828c3811ec None None] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:17:17.414399 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12275320-1228-4213-ac4e-23828c3811ec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:17:17.424310 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG dbcounter [-] [80245] Writing DB stats keystone:SELECT=100 {{(pid=80245) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:17:17.435290 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.auth.core [None req-12275320-1228-4213-ac4e-23828c3811ec None None] MFA Rules not processed for user `2dce40b8d74c41ec94cc36e6ff05d412`. Rule list: `[]` (Enabled: `True`). {{(pid=80244) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 08:17:17.436754 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.fernet_utils [None req-12275320-1228-4213-ac4e-23828c3811ec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80244) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 08:17:17.500395 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG dbcounter [-] [80244] Writing DB stats keystone:SELECT=100 {{(pid=80244) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:17:17.501234 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01b05de1-2c57-47c7-ac2e-1269c542dd23 None None] Validating token access rules against request {{(pid=80245) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:17:17.512478 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 312/626] 10.222.0.47 () {58 vars in 1023 bytes} [Thu Jul 21 08:17:17 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 21 08:17:17.513667 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12275320-1228-4213-ac4e-23828c3811ec None None] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:17:17.537355 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74596062-bf3c-4500-baca-5deff341b311 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80245) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:17:17.538003 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74596062-bf3c-4500-baca-5deff341b311 None admin] REQUEST_METHOD: `GET` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:17:17.538342 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74596062-bf3c-4500-baca-5deff341b311 None admin] SCRIPT_NAME: `/identity` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:17:17.538621 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74596062-bf3c-4500-baca-5deff341b311 None admin] PATH_INFO: `/v3/users` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:17:17.539536 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74596062-bf3c-4500-baca-5deff341b311 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80245) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 08:17:17.539840 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74596062-bf3c-4500-baca-5deff341b311 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:17:17.541349 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74596062-bf3c-4500-baca-5deff341b311 None admin] RBAC: Authorization granted {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:17:17.561629 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: [pid: 80245|app: 0|req: 315/627] 10.222.0.47 () {56 vars in 1133 bytes} [Thu Jul 21 08:17:17 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 08:17:17.562583 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74596062-bf3c-4500-baca-5deff341b311 None admin] Authenticating user token {{(pid=80245) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:17:17.624723 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12275320-1228-4213-ac4e-23828c3811ec None None] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:17:17.664403 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG dbcounter [-] [80245] Writing DB stats keystone:SELECT=100 {{(pid=80245) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:17:17.667315 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2f8aa7c-8a97-4ea7-9b19-9bd4671f9c57 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:17:17.668769 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2f8aa7c-8a97-4ea7-9b19-9bd4671f9c57 None admin] REQUEST_METHOD: `GET` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:17:17.670100 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2f8aa7c-8a97-4ea7-9b19-9bd4671f9c57 None admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:17:17.670600 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2f8aa7c-8a97-4ea7-9b19-9bd4671f9c57 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:17:17.680167 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2f8aa7c-8a97-4ea7-9b19-9bd4671f9c57 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:17:17.681802 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74596062-bf3c-4500-baca-5deff341b311 None admin] Validating token access rules against request {{(pid=80245) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:17:17.682193 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2f8aa7c-8a97-4ea7-9b19-9bd4671f9c57 None admin] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:17:17.698642 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application [None req-b2f8aa7c-8a97-4ea7-9b19-9bd4671f9c57 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 21 08:17:17.698642 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 08:17:17.698642 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 08:17:17.698642 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 08:17:17.698642 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 08:17:17.698642 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 08:17:17.698642 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 08:17:17.698642 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 08:17:17.698642 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 08:17:17.698642 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 08:17:17.698642 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 08:17:17.698642 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 08:17:17.698642 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 21 08:17:17.698642 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 21 08:17:17.698642 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 21 08:17:17.699643 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 21 08:17:17.699643 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 08:17:17.699643 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 08:17:17.699643 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 08:17:17.699643 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 08:17:17.699643 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 08:17:17.699643 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 08:17:17.699643 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 08:17:17.699643 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application with Lock( Jul 21 08:17:17.699643 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 08:17:17.699643 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self._enter() Jul 21 08:17:17.699643 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 08:17:17.699643 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 08:17:17.699643 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 08:17:17.699643 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self.creator() Jul 21 08:17:17.699643 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 08:17:17.699643 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application created_value = creator( Jul 21 08:17:17.700616 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 21 08:17:17.700616 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 21 08:17:17.700616 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 21 08:17:17.700616 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 21 08:17:17.700616 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 21 08:17:17.700616 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 21 08:17:17.700616 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 21 08:17:17.700616 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application Jul 21 08:17:17.703564 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 313/628] 10.222.0.47 () {56 vars in 1111 bytes} [Thu Jul 21 08:17:17 2022] GET /identity/v3/projects/service => generated 88 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 08:17:17.704805 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2f8aa7c-8a97-4ea7-9b19-9bd4671f9c57 None admin] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:17:17.730036 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea7f33f4-f00b-4718-805f-38505876aae6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80245) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:17:17.730849 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea7f33f4-f00b-4718-805f-38505876aae6 None admin] REQUEST_METHOD: `GET` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:17:17.731180 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea7f33f4-f00b-4718-805f-38505876aae6 None admin] SCRIPT_NAME: `/identity` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:17:17.731510 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea7f33f4-f00b-4718-805f-38505876aae6 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:17:17.743300 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea7f33f4-f00b-4718-805f-38505876aae6 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 21 08:17:17.751358 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application [None req-ea7f33f4-f00b-4718-805f-38505876aae6 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 21 08:17:17.751358 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 08:17:17.751358 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 08:17:17.751358 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 08:17:17.751358 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 08:17:17.751358 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 08:17:17.751358 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 08:17:17.751358 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 08:17:17.751358 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 08:17:17.751358 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 08:17:17.751358 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 08:17:17.751358 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 08:17:17.751358 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 21 08:17:17.751358 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 21 08:17:17.751358 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 21 08:17:17.751358 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 21 08:17:17.752329 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 21 08:17:17.752329 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 21 08:17:17.752329 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 21 08:17:17.752329 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 21 08:17:17.752329 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 08:17:17.752329 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 08:17:17.752329 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 08:17:17.752329 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 08:17:17.752329 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 08:17:17.752329 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 08:17:17.752329 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 08:17:17.752329 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application with Lock( Jul 21 08:17:17.752329 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 08:17:17.752329 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application return self._enter() Jul 21 08:17:17.752329 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 08:17:17.752329 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 08:17:17.753142 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 08:17:17.753142 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application return self.creator() Jul 21 08:17:17.753142 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 08:17:17.753142 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application created_value = creator( Jul 21 08:17:17.753142 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 21 08:17:17.753142 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 21 08:17:17.753142 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 21 08:17:17.753142 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 21 08:17:17.753142 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 21 08:17:17.753142 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 21 08:17:17.753142 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 21 08:17:17.753142 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application Jul 21 08:17:17.753142 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: [pid: 80245|app: 0|req: 316/629] 10.222.0.47 () {56 vars in 1102 bytes} [Thu Jul 21 08:17:17 2022] GET /identity/v3/roles/service => generated 85 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 08:17:17.753859 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea7f33f4-f00b-4718-805f-38505876aae6 None admin] Authenticating user token {{(pid=80245) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:17:17.759691 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG dbcounter [-] [80244] Writing DB stats keystone:SELECT=100 {{(pid=80244) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:17:17.837112 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2f8aa7c-8a97-4ea7-9b19-9bd4671f9c57 None admin] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:17:17.868396 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea7f33f4-f00b-4718-805f-38505876aae6 None admin] Validating token access rules against request {{(pid=80245) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:17:17.885834 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1efe08f3-ff84-4934-a6be-987b01995c87 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:17:17.886452 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1efe08f3-ff84-4934-a6be-987b01995c87 None admin] REQUEST_METHOD: `GET` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:17:17.886741 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1efe08f3-ff84-4934-a6be-987b01995c87 None admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:17:17.887114 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1efe08f3-ff84-4934-a6be-987b01995c87 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:17:17.906708 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6655659-769d-47a8-affc-4ca170ec41f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80245) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:17:17.907777 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6655659-769d-47a8-affc-4ca170ec41f5 None admin] REQUEST_METHOD: `GET` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:17:17.908271 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6655659-769d-47a8-affc-4ca170ec41f5 None admin] SCRIPT_NAME: `/identity` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:17:17.908739 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6655659-769d-47a8-affc-4ca170ec41f5 None admin] PATH_INFO: `/v3/projects` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:17:17.909735 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6655659-769d-47a8-affc-4ca170ec41f5 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80245) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 08:17:17.910244 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6655659-769d-47a8-affc-4ca170ec41f5 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:17:17.912454 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6655659-769d-47a8-affc-4ca170ec41f5 None admin] RBAC: Authorization granted {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:17:17.925595 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: [pid: 80245|app: 0|req: 317/630] 10.222.0.47 () {56 vars in 1148 bytes} [Thu Jul 21 08:17:17 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 08:17:17.927834 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6655659-769d-47a8-affc-4ca170ec41f5 None admin] Authenticating user token {{(pid=80245) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:17:17.936827 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG dbcounter [-] [80245] Writing DB stats keystone:SELECT=100 {{(pid=80245) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:17:17.978695 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1efe08f3-ff84-4934-a6be-987b01995c87 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:17:17.980352 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1efe08f3-ff84-4934-a6be-987b01995c87 None admin] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:17:18.025356 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG dbcounter [-] [80244] Writing DB stats keystone:SELECT=100 {{(pid=80244) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:17:18.040029 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6655659-769d-47a8-affc-4ca170ec41f5 None admin] Validating token access rules against request {{(pid=80245) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:17:18.076298 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-138c06cb-da79-4cab-9778-42f804d4c6be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80245) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:17:18.077350 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-138c06cb-da79-4cab-9778-42f804d4c6be None admin] REQUEST_METHOD: `GET` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:17:18.077828 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-138c06cb-da79-4cab-9778-42f804d4c6be None admin] SCRIPT_NAME: `/identity` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:17:18.078313 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-138c06cb-da79-4cab-9778-42f804d4c6be None admin] PATH_INFO: `/v3/roles` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:17:18.079244 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-138c06cb-da79-4cab-9778-42f804d4c6be None admin] RBAC: Adding query filter params (name=service) {{(pid=80245) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 08:17:18.079795 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-138c06cb-da79-4cab-9778-42f804d4c6be None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:17:18.081363 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-138c06cb-da79-4cab-9778-42f804d4c6be None admin] RBAC: Authorization granted {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:17:18.088177 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 314/631] 10.222.0.47 () {58 vars in 1282 bytes} [Thu Jul 21 08:17:17 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 384 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 21 08:17:18.089095 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1efe08f3-ff84-4934-a6be-987b01995c87 None admin] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:17:18.091935 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: [pid: 80245|app: 0|req: 318/632] 10.222.0.47 () {56 vars in 1103 bytes} [Thu Jul 21 08:17:17 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 08:17:18.093204 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-138c06cb-da79-4cab-9778-42f804d4c6be None admin] Authenticating user token {{(pid=80245) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:17:18.167131 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG dbcounter [-] [80245] Writing DB stats keystone:SELECT=100 {{(pid=80245) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:17:18.193072 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1efe08f3-ff84-4934-a6be-987b01995c87 None admin] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:17:18.208473 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-138c06cb-da79-4cab-9778-42f804d4c6be None admin] Validating token access rules against request {{(pid=80245) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:17:18.230000 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2860c05c-fa3e-419d-9fd4-70a1359766fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:17:18.230997 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2860c05c-fa3e-419d-9fd4-70a1359766fa None admin] REQUEST_METHOD: `PUT` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:17:18.231288 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2860c05c-fa3e-419d-9fd4-70a1359766fa None admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:17:18.232019 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2860c05c-fa3e-419d-9fd4-70a1359766fa None admin] PATH_INFO: `/v3/projects/4faef13aa50e4e36b6c265fb1398fa14/users/302351411fc84248a6b166b25d21609f/roles/50d1ec7f947f42acb65c57e07f2269d6` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:17:18.245376 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG dbcounter [-] [80244] Writing DB stats keystone:SELECT=100 {{(pid=80244) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:17:18.247785 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd6a1da7-4106-4a21-8bc1-b2254dc24bfc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80245) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:17:18.248586 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd6a1da7-4106-4a21-8bc1-b2254dc24bfc None admin] REQUEST_METHOD: `GET` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:17:18.248898 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd6a1da7-4106-4a21-8bc1-b2254dc24bfc None admin] SCRIPT_NAME: `/identity` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:17:18.249136 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd6a1da7-4106-4a21-8bc1-b2254dc24bfc None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:17:18.250039 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2860c05c-fa3e-419d-9fd4-70a1359766fa None admin] RBAC: Authorizing `identity:create_grant(project_id=4faef13aa50e4e36b6c265fb1398fa14, user_id=302351411fc84248a6b166b25d21609f, role_id=50d1ec7f947f42acb65c57e07f2269d6)` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:17:18.252070 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2860c05c-fa3e-419d-9fd4-70a1359766fa None admin] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:17:18.260877 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd6a1da7-4106-4a21-8bc1-b2254dc24bfc None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:17:18.262328 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd6a1da7-4106-4a21-8bc1-b2254dc24bfc None admin] RBAC: Authorization granted {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:17:18.268130 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 315/633] 10.222.0.47 () {58 vars in 1439 bytes} [Thu Jul 21 08:17:18 2022] PUT /identity/v3/projects/4faef13aa50e4e36b6c265fb1398fa14/users/302351411fc84248a6b166b25d21609f/roles/50d1ec7f947f42acb65c57e07f2269d6 => generated 0 bytes in 179 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 08:17:18.269392 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2860c05c-fa3e-419d-9fd4-70a1359766fa None admin] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:17:18.288438 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application [None req-dd6a1da7-4106-4a21-8bc1-b2254dc24bfc None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 08:17:18.288438 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 08:17:18.288438 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 08:17:18.288438 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 08:17:18.288438 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 08:17:18.288438 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 08:17:18.288438 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 08:17:18.288438 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 08:17:18.288438 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 08:17:18.288438 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 08:17:18.288438 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 08:17:18.288438 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 08:17:18.288438 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 21 08:17:18.288438 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 21 08:17:18.288438 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 21 08:17:18.289579 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 21 08:17:18.289579 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 08:17:18.289579 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 08:17:18.289579 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 08:17:18.289579 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 08:17:18.289579 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 08:17:18.289579 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 08:17:18.289579 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 08:17:18.289579 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application with Lock( Jul 21 08:17:18.289579 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 08:17:18.289579 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application return self._enter() Jul 21 08:17:18.289579 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 08:17:18.289579 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 08:17:18.289579 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 08:17:18.289579 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application return self.creator() Jul 21 08:17:18.289579 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 08:17:18.289579 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application created_value = creator( Jul 21 08:17:18.290672 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 21 08:17:18.290672 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 21 08:17:18.290672 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 08:17:18.290672 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application Jul 21 08:17:18.292751 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: [pid: 80245|app: 0|req: 319/634] 10.222.0.47 () {56 vars in 1108 bytes} [Thu Jul 21 08:17:18 2022] GET /identity/v3/domains/Default => generated 87 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 21 08:17:18.294449 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7d9db6a-92be-4b5d-b5c5-f8eae6122e44 None None] REQUEST_METHOD: `GET` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:17:18.294920 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7d9db6a-92be-4b5d-b5c5-f8eae6122e44 None None] SCRIPT_NAME: `/identity` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:17:18.295443 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7d9db6a-92be-4b5d-b5c5-f8eae6122e44 None None] PATH_INFO: `/` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:17:18.300870 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: [pid: 80245|app: 0|req: 320/635] 10.222.0.47 () {54 vars in 926 bytes} [Thu Jul 21 08:17:18 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 08:17:18.300870 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7d9db6a-92be-4b5d-b5c5-f8eae6122e44 None None] Authenticating user token {{(pid=80245) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:17:18.425443 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7d9db6a-92be-4b5d-b5c5-f8eae6122e44 None None] Validating token access rules against request {{(pid=80245) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:17:18.436912 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2860c05c-fa3e-419d-9fd4-70a1359766fa None admin] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:17:18.456859 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG dbcounter [-] [80245] Writing DB stats keystone:SELECT=100 {{(pid=80245) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:17:18.480614 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-895b0e9c-675d-4d7d-9e7d-a3a31cf007a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80245) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:17:18.480614 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-895b0e9c-675d-4d7d-9e7d-a3a31cf007a2 None admin] REQUEST_METHOD: `GET` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:17:18.480614 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-895b0e9c-675d-4d7d-9e7d-a3a31cf007a2 None admin] SCRIPT_NAME: `/identity` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:17:18.483537 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78fe41ec-4b43-4950-af9f-fb01b4c44173 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:17:18.484339 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78fe41ec-4b43-4950-af9f-fb01b4c44173 None admin] REQUEST_METHOD: `GET` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:17:18.484620 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78fe41ec-4b43-4950-af9f-fb01b4c44173 None admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:17:18.484890 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78fe41ec-4b43-4950-af9f-fb01b4c44173 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:17:18.485891 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-895b0e9c-675d-4d7d-9e7d-a3a31cf007a2 None admin] PATH_INFO: `/v3/domains` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:17:18.486481 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-895b0e9c-675d-4d7d-9e7d-a3a31cf007a2 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80245) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 08:17:18.486905 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-895b0e9c-675d-4d7d-9e7d-a3a31cf007a2 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:17:18.488467 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-895b0e9c-675d-4d7d-9e7d-a3a31cf007a2 None admin] RBAC: Authorization granted {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:17:18.495674 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78fe41ec-4b43-4950-af9f-fb01b4c44173 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:17:18.497128 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78fe41ec-4b43-4950-af9f-fb01b4c44173 None admin] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:17:18.508576 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: [pid: 80245|app: 0|req: 321/636] 10.222.0.47 () {56 vars in 1109 bytes} [Thu Jul 21 08:17:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 08:17:18.510260 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1d27b40-ac96-4ee3-81ab-452584a6a962 None None] REQUEST_METHOD: `POST` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:17:18.510773 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1d27b40-ac96-4ee3-81ab-452584a6a962 None None] SCRIPT_NAME: `/identity` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:17:18.511269 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1d27b40-ac96-4ee3-81ab-452584a6a962 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:17:18.516529 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application [None req-78fe41ec-4b43-4950-af9f-fb01b4c44173 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 08:17:18.516529 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 08:17:18.516529 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 08:17:18.516529 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 08:17:18.516529 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 08:17:18.516529 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 08:17:18.516529 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 08:17:18.516529 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 08:17:18.516529 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 08:17:18.516529 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 08:17:18.516529 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 08:17:18.516529 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 08:17:18.516529 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 21 08:17:18.516529 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 21 08:17:18.516529 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 21 08:17:18.517411 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 21 08:17:18.517411 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 08:17:18.517411 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 08:17:18.517411 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 08:17:18.517411 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 08:17:18.517411 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 08:17:18.517411 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 08:17:18.517411 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 08:17:18.517411 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application with Lock( Jul 21 08:17:18.517411 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 08:17:18.517411 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self._enter() Jul 21 08:17:18.517411 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 08:17:18.517411 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 08:17:18.517411 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 08:17:18.517411 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self.creator() Jul 21 08:17:18.517411 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 08:17:18.517411 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application created_value = creator( Jul 21 08:17:18.518877 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 21 08:17:18.518877 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 21 08:17:18.518877 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 08:17:18.518877 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application Jul 21 08:17:18.518877 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 316/637] 10.222.0.47 () {56 vars in 1108 bytes} [Thu Jul 21 08:17:18 2022] GET /identity/v3/domains/Default => generated 87 bytes in 249 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 08:17:18.518877 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78fe41ec-4b43-4950-af9f-fb01b4c44173 None admin] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:17:18.534975 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.auth.core [None req-b1d27b40-ac96-4ee3-81ab-452584a6a962 None None] MFA Rules not processed for user `2dce40b8d74c41ec94cc36e6ff05d412`. Rule list: `[]` (Enabled: `True`). {{(pid=80245) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 08:17:18.537852 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.fernet_utils [None req-b1d27b40-ac96-4ee3-81ab-452584a6a962 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80245) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 08:17:18.569012 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG dbcounter [-] [80244] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80244) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:17:18.632276 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: [pid: 80245|app: 0|req: 322/638] 10.222.0.47 () {58 vars in 1023 bytes} [Thu Jul 21 08:17:18 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 21 08:17:18.634495 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1d27b40-ac96-4ee3-81ab-452584a6a962 None None] Authenticating user token {{(pid=80245) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:17:18.636265 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78fe41ec-4b43-4950-af9f-fb01b4c44173 None admin] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:17:18.684676 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a907cb95-609d-474d-b49f-cb22e6cae0a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:17:18.686186 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a907cb95-609d-474d-b49f-cb22e6cae0a4 None admin] REQUEST_METHOD: `GET` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:17:18.686778 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a907cb95-609d-474d-b49f-cb22e6cae0a4 None admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:17:18.687617 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a907cb95-609d-474d-b49f-cb22e6cae0a4 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:17:18.697358 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a907cb95-609d-474d-b49f-cb22e6cae0a4 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:17:18.699822 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a907cb95-609d-474d-b49f-cb22e6cae0a4 None admin] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:17:18.715823 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application [None req-a907cb95-609d-474d-b49f-cb22e6cae0a4 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 21 08:17:18.715823 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 08:17:18.715823 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 08:17:18.715823 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 08:17:18.715823 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 08:17:18.715823 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 08:17:18.715823 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 08:17:18.715823 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 08:17:18.715823 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 08:17:18.715823 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 08:17:18.715823 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 08:17:18.715823 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 08:17:18.715823 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 21 08:17:18.715823 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 21 08:17:18.715823 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 21 08:17:18.716912 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 21 08:17:18.716912 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 08:17:18.716912 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 08:17:18.716912 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 08:17:18.716912 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 08:17:18.716912 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 08:17:18.716912 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 08:17:18.716912 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 08:17:18.716912 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application with Lock( Jul 21 08:17:18.716912 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 08:17:18.716912 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self._enter() Jul 21 08:17:18.716912 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 08:17:18.716912 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 08:17:18.716912 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 08:17:18.716912 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self.creator() Jul 21 08:17:18.716912 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 08:17:18.716912 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application created_value = creator( Jul 21 08:17:18.717999 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 21 08:17:18.717999 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 21 08:17:18.717999 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 21 08:17:18.717999 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 21 08:17:18.717999 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 21 08:17:18.717999 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 21 08:17:18.717999 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 21 08:17:18.717999 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application Jul 21 08:17:18.718910 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 317/639] 10.222.0.47 () {56 vars in 1111 bytes} [Thu Jul 21 08:17:18 2022] GET /identity/v3/projects/service => generated 88 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 08:17:18.719746 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a907cb95-609d-474d-b49f-cb22e6cae0a4 None admin] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:17:18.753841 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1d27b40-ac96-4ee3-81ab-452584a6a962 None None] Validating token access rules against request {{(pid=80245) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:17:18.765705 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG dbcounter [-] [80245] Writing DB stats keystone:SELECT=100 {{(pid=80245) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:17:18.795698 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b221128e-063e-4966-9333-5daa2b7cae9e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80245) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:17:18.796615 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b221128e-063e-4966-9333-5daa2b7cae9e None admin] REQUEST_METHOD: `GET` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:17:18.797048 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b221128e-063e-4966-9333-5daa2b7cae9e None admin] SCRIPT_NAME: `/identity` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:17:18.797522 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b221128e-063e-4966-9333-5daa2b7cae9e None admin] PATH_INFO: `/v3/domains` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:17:18.798252 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b221128e-063e-4966-9333-5daa2b7cae9e None admin] RBAC: Adding query filter params (name=Default) {{(pid=80245) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 08:17:18.798559 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b221128e-063e-4966-9333-5daa2b7cae9e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:17:18.799963 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b221128e-063e-4966-9333-5daa2b7cae9e None admin] RBAC: Authorization granted {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:17:18.816756 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: [pid: 80245|app: 0|req: 323/640] 10.222.0.47 () {56 vars in 1109 bytes} [Thu Jul 21 08:17:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 08:17:18.816756 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b16fa938-a1af-4c7a-be67-aa01fb4aef58 None None] REQUEST_METHOD: `GET` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:17:18.816756 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b16fa938-a1af-4c7a-be67-aa01fb4aef58 None None] SCRIPT_NAME: `/identity` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:17:18.816756 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b16fa938-a1af-4c7a-be67-aa01fb4aef58 None None] PATH_INFO: `/` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:17:18.816756 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: [pid: 80245|app: 0|req: 324/641] 10.222.0.47 () {54 vars in 926 bytes} [Thu Jul 21 08:17:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 08:17:18.816756 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b16fa938-a1af-4c7a-be67-aa01fb4aef58 None None] Authenticating user token {{(pid=80245) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:17:18.836337 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a907cb95-609d-474d-b49f-cb22e6cae0a4 None admin] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:17:18.838686 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG dbcounter [-] [80244] Writing DB stats keystone:SELECT=100 {{(pid=80244) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:17:18.874519 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8f26d35-4e36-4aa3-98e8-cdb8b0009dd8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:17:18.875772 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8f26d35-4e36-4aa3-98e8-cdb8b0009dd8 None admin] REQUEST_METHOD: `GET` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:17:18.876076 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8f26d35-4e36-4aa3-98e8-cdb8b0009dd8 None admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:17:18.876350 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8f26d35-4e36-4aa3-98e8-cdb8b0009dd8 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:17:18.887238 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8f26d35-4e36-4aa3-98e8-cdb8b0009dd8 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 21 08:17:18.895095 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application [None req-d8f26d35-4e36-4aa3-98e8-cdb8b0009dd8 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 21 08:17:18.895095 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 08:17:18.895095 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 08:17:18.895095 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 08:17:18.895095 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 08:17:18.895095 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 08:17:18.895095 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 08:17:18.895095 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 08:17:18.895095 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 08:17:18.895095 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 08:17:18.895095 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 08:17:18.895095 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 08:17:18.895095 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 21 08:17:18.895095 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 21 08:17:18.895095 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 21 08:17:18.895095 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 21 08:17:18.896324 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 21 08:17:18.896324 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 21 08:17:18.896324 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 21 08:17:18.896324 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 21 08:17:18.896324 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 08:17:18.896324 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 08:17:18.896324 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 08:17:18.896324 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 08:17:18.896324 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 08:17:18.896324 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 08:17:18.896324 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 08:17:18.896324 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application with Lock( Jul 21 08:17:18.896324 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 08:17:18.896324 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self._enter() Jul 21 08:17:18.896324 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 08:17:18.896324 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 08:17:18.897148 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 08:17:18.897148 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self.creator() Jul 21 08:17:18.897148 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 08:17:18.897148 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application created_value = creator( Jul 21 08:17:18.897148 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 21 08:17:18.897148 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 21 08:17:18.897148 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 21 08:17:18.897148 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 21 08:17:18.897148 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 21 08:17:18.897148 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 21 08:17:18.897148 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 21 08:17:18.897148 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application Jul 21 08:17:18.901284 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 318/642] 10.222.0.47 () {56 vars in 1102 bytes} [Thu Jul 21 08:17:18 2022] GET /identity/v3/roles/service => generated 85 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 08:17:18.902154 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8f26d35-4e36-4aa3-98e8-cdb8b0009dd8 None admin] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:17:18.929370 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b16fa938-a1af-4c7a-be67-aa01fb4aef58 None None] Validating token access rules against request {{(pid=80245) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:17:18.965841 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df722160-65d5-4b6f-a8dc-a15c389dfbfe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80245) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:17:18.966730 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df722160-65d5-4b6f-a8dc-a15c389dfbfe None admin] REQUEST_METHOD: `GET` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:17:18.966730 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df722160-65d5-4b6f-a8dc-a15c389dfbfe None admin] SCRIPT_NAME: `/identity` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:17:18.967030 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df722160-65d5-4b6f-a8dc-a15c389dfbfe None admin] PATH_INFO: `/v3/projects` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:17:18.967896 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df722160-65d5-4b6f-a8dc-a15c389dfbfe None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80245) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 08:17:18.968130 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df722160-65d5-4b6f-a8dc-a15c389dfbfe None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:17:18.969567 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df722160-65d5-4b6f-a8dc-a15c389dfbfe None admin] RBAC: Authorization granted {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:17:18.982702 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: [pid: 80245|app: 0|req: 325/643] 10.222.0.47 () {56 vars in 1148 bytes} [Thu Jul 21 08:17:18 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 08:17:18.984220 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8cf4889-1a5e-44df-a017-30848465249f None None] REQUEST_METHOD: `POST` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:17:18.984513 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8cf4889-1a5e-44df-a017-30848465249f None None] SCRIPT_NAME: `/identity` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:17:18.984788 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8cf4889-1a5e-44df-a017-30848465249f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:17:19.006757 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.auth.core [None req-b8cf4889-1a5e-44df-a017-30848465249f None None] MFA Rules not processed for user `2dce40b8d74c41ec94cc36e6ff05d412`. Rule list: `[]` (Enabled: `True`). {{(pid=80245) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 08:17:19.008677 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.fernet_utils [None req-b8cf4889-1a5e-44df-a017-30848465249f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80245) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 08:17:19.021223 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8f26d35-4e36-4aa3-98e8-cdb8b0009dd8 None admin] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:17:19.039196 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG dbcounter [-] [80245] Writing DB stats keystone:SELECT=100 {{(pid=80245) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:17:19.058738 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6ff2f4a-4f4e-4817-8877-42ef845c23bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:17:19.059567 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6ff2f4a-4f4e-4817-8877-42ef845c23bf None admin] REQUEST_METHOD: `GET` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:17:19.059848 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6ff2f4a-4f4e-4817-8877-42ef845c23bf None admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:17:19.060182 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6ff2f4a-4f4e-4817-8877-42ef845c23bf None admin] PATH_INFO: `/v3/users/glance` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:17:19.068122 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6ff2f4a-4f4e-4817-8877-42ef845c23bf None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:17:19.069581 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6ff2f4a-4f4e-4817-8877-42ef845c23bf None admin] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:17:19.083969 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application [None req-d6ff2f4a-4f4e-4817-8877-42ef845c23bf None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Jul 21 08:17:19.083969 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 08:17:19.083969 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 08:17:19.083969 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 08:17:19.083969 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 08:17:19.083969 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 08:17:19.083969 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 08:17:19.083969 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 08:17:19.083969 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 08:17:19.083969 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 08:17:19.083969 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 08:17:19.083969 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 08:17:19.083969 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 21 08:17:19.083969 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 21 08:17:19.083969 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 21 08:17:19.083969 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 21 08:17:19.084912 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 08:17:19.084912 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 08:17:19.084912 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 21 08:17:19.084912 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 21 08:17:19.084912 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 21 08:17:19.084912 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 21 08:17:19.084912 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 08:17:19.084912 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 08:17:19.084912 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 08:17:19.084912 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 08:17:19.084912 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 08:17:19.084912 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application with Lock( Jul 21 08:17:19.084912 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 08:17:19.084912 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self._enter() Jul 21 08:17:19.084912 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 08:17:19.084912 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 08:17:19.084912 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 08:17:19.084912 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self.creator() Jul 21 08:17:19.085984 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 08:17:19.085984 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application created_value = creator( Jul 21 08:17:19.085984 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 21 08:17:19.085984 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 21 08:17:19.085984 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 21 08:17:19.085984 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 21 08:17:19.085984 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 21 08:17:19.085984 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 21 08:17:19.085984 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Jul 21 08:17:19.085984 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application Jul 21 08:17:19.085984 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 319/644] 10.222.0.47 () {56 vars in 1099 bytes} [Thu Jul 21 08:17:18 2022] GET /identity/v3/users/glance => generated 84 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 08:17:19.085984 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6ff2f4a-4f4e-4817-8877-42ef845c23bf None admin] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:17:19.093799 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: [pid: 80245|app: 0|req: 326/645] 10.222.0.47 () {58 vars in 1023 bytes} [Thu Jul 21 08:17:18 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 21 08:17:19.094846 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8cf4889-1a5e-44df-a017-30848465249f None None] Authenticating user token {{(pid=80245) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:17:19.119582 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG dbcounter [-] [80244] Writing DB stats keystone:SELECT=100 {{(pid=80244) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:17:19.206259 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6ff2f4a-4f4e-4817-8877-42ef845c23bf None admin] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:17:19.206881 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8cf4889-1a5e-44df-a017-30848465249f None None] Validating token access rules against request {{(pid=80245) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:17:19.243221 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c99abf0-6b34-4956-a959-458951f54c61 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:17:19.243845 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c99abf0-6b34-4956-a959-458951f54c61 None admin] REQUEST_METHOD: `GET` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:17:19.244133 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c99abf0-6b34-4956-a959-458951f54c61 None admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:17:19.244412 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c99abf0-6b34-4956-a959-458951f54c61 None admin] PATH_INFO: `/v3/roles` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:17:19.245161 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c99abf0-6b34-4956-a959-458951f54c61 None admin] RBAC: Adding query filter params (name=service) {{(pid=80244) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 08:17:19.245470 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c99abf0-6b34-4956-a959-458951f54c61 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:17:19.246859 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c99abf0-6b34-4956-a959-458951f54c61 None admin] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:17:19.247781 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5812d2ce-4d4c-4620-bb73-474f822391ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80245) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:17:19.248444 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5812d2ce-4d4c-4620-bb73-474f822391ae None admin] REQUEST_METHOD: `GET` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:17:19.248726 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5812d2ce-4d4c-4620-bb73-474f822391ae None admin] SCRIPT_NAME: `/identity` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:17:19.249248 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5812d2ce-4d4c-4620-bb73-474f822391ae None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:17:19.249830 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5812d2ce-4d4c-4620-bb73-474f822391ae None admin] RBAC: Adding query filter params (role.id=50d1ec7f947f42acb65c57e07f2269d6, scope.project.id=4faef13aa50e4e36b6c265fb1398fa14, user.id=ac2b5b99a36442148fe743269054e991) {{(pid=80245) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 08:17:19.250140 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5812d2ce-4d4c-4620-bb73-474f822391ae None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:17:19.251598 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5812d2ce-4d4c-4620-bb73-474f822391ae None admin] RBAC: Authorization granted {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:17:19.255468 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 320/646] 10.222.0.47 () {56 vars in 1103 bytes} [Thu Jul 21 08:17:19 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 08:17:19.256150 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c99abf0-6b34-4956-a959-458951f54c61 None admin] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:17:19.267708 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: [pid: 80245|app: 0|req: 327/647] 10.222.0.47 () {56 vars in 1374 bytes} [Thu Jul 21 08:17:19 2022] GET /identity/v3/role_assignments?user.id=ac2b5b99a36442148fe743269054e991&scope.project.id=4faef13aa50e4e36b6c265fb1398fa14&role.id=50d1ec7f947f42acb65c57e07f2269d6 => generated 608 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 08:17:19.269101 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5812d2ce-4d4c-4620-bb73-474f822391ae None admin] Authenticating user token {{(pid=80245) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:17:19.313547 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG dbcounter [-] [80245] Writing DB stats keystone:SELECT=100 {{(pid=80245) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:17:19.398515 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG dbcounter [-] [80244] Writing DB stats keystone:SELECT=100 {{(pid=80244) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:17:19.414133 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5812d2ce-4d4c-4620-bb73-474f822391ae None admin] Validating token access rules against request {{(pid=80245) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:17:19.419146 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c99abf0-6b34-4956-a959-458951f54c61 None admin] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:17:19.467649 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3a74d92-9625-42e3-9f98-db41e50963f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:17:19.468337 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3a74d92-9625-42e3-9f98-db41e50963f7 None admin] REQUEST_METHOD: `GET` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:17:19.468618 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3a74d92-9625-42e3-9f98-db41e50963f7 None admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:17:19.468878 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b087ac9-174c-4c13-90e9-37c56edf93c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80245) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:17:19.469316 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3a74d92-9625-42e3-9f98-db41e50963f7 None admin] PATH_INFO: `/v3/users` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:17:19.469724 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3a74d92-9625-42e3-9f98-db41e50963f7 None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=80244) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 08:17:19.470026 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3a74d92-9625-42e3-9f98-db41e50963f7 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:17:19.470585 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b087ac9-174c-4c13-90e9-37c56edf93c9 None admin] REQUEST_METHOD: `GET` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:17:19.471021 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b087ac9-174c-4c13-90e9-37c56edf93c9 None admin] SCRIPT_NAME: `/identity` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:17:19.471526 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b087ac9-174c-4c13-90e9-37c56edf93c9 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:17:19.473380 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3a74d92-9625-42e3-9f98-db41e50963f7 None admin] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:17:19.490328 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b087ac9-174c-4c13-90e9-37c56edf93c9 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 21 08:17:19.498466 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 321/648] 10.222.0.47 () {56 vars in 1137 bytes} [Thu Jul 21 08:17:19 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 242 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 08:17:19.498906 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application [None req-0b087ac9-174c-4c13-90e9-37c56edf93c9 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 21 08:17:19.498906 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 08:17:19.498906 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 08:17:19.498906 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 08:17:19.498906 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 08:17:19.498906 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 08:17:19.498906 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 08:17:19.498906 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 08:17:19.498906 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 08:17:19.498906 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 08:17:19.498906 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 08:17:19.498906 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 08:17:19.498906 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 21 08:17:19.498906 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 21 08:17:19.498906 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 21 08:17:19.498906 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 21 08:17:19.499849 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 21 08:17:19.499849 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 21 08:17:19.499849 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 21 08:17:19.499849 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 21 08:17:19.499849 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 08:17:19.499849 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 08:17:19.499849 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 08:17:19.499849 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 08:17:19.499849 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 08:17:19.499849 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 08:17:19.499849 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 08:17:19.499849 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application with Lock( Jul 21 08:17:19.499849 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 08:17:19.499849 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application return self._enter() Jul 21 08:17:19.499849 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 08:17:19.499849 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 08:17:19.500663 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 08:17:19.500663 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application return self.creator() Jul 21 08:17:19.500663 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 08:17:19.500663 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application created_value = creator( Jul 21 08:17:19.500663 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 21 08:17:19.500663 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 21 08:17:19.500663 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 21 08:17:19.500663 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 21 08:17:19.500663 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 21 08:17:19.500663 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 21 08:17:19.500663 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 21 08:17:19.500663 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application Jul 21 08:17:19.501290 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: [pid: 80245|app: 0|req: 328/649] 10.222.0.47 () {56 vars in 1102 bytes} [Thu Jul 21 08:17:19 2022] GET /identity/v3/roles/service => generated 85 bytes in 232 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 08:17:19.503100 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3a74d92-9625-42e3-9f98-db41e50963f7 None admin] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:17:19.507111 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b087ac9-174c-4c13-90e9-37c56edf93c9 None admin] Authenticating user token {{(pid=80245) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:17:19.619744 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG dbcounter [-] [80245] Writing DB stats keystone:SELECT=100 {{(pid=80245) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:17:19.642803 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b087ac9-174c-4c13-90e9-37c56edf93c9 None admin] Validating token access rules against request {{(pid=80245) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:17:19.650685 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3a74d92-9625-42e3-9f98-db41e50963f7 None admin] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:17:19.692098 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf5b78d2-7d09-4c0b-8777-1f888f8f0f97 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80245) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:17:19.692768 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf5b78d2-7d09-4c0b-8777-1f888f8f0f97 None admin] REQUEST_METHOD: `GET` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:17:19.693055 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf5b78d2-7d09-4c0b-8777-1f888f8f0f97 None admin] SCRIPT_NAME: `/identity` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:17:19.693334 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf5b78d2-7d09-4c0b-8777-1f888f8f0f97 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:17:19.694417 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4d621ca-3dad-4f97-93c0-706b2f1c19b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:17:19.695471 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4d621ca-3dad-4f97-93c0-706b2f1c19b4 None admin] REQUEST_METHOD: `GET` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:17:19.695954 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4d621ca-3dad-4f97-93c0-706b2f1c19b4 None admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:17:19.696482 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4d621ca-3dad-4f97-93c0-706b2f1c19b4 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:17:19.708632 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4d621ca-3dad-4f97-93c0-706b2f1c19b4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:17:19.711601 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4d621ca-3dad-4f97-93c0-706b2f1c19b4 None admin] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:17:19.714885 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG dbcounter [-] [80244] Writing DB stats keystone:SELECT=100 {{(pid=80244) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:17:19.736377 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application [None req-a4d621ca-3dad-4f97-93c0-706b2f1c19b4 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 08:17:19.736377 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 08:17:19.736377 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 08:17:19.736377 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 08:17:19.736377 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 08:17:19.736377 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 08:17:19.736377 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 08:17:19.736377 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 08:17:19.736377 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 08:17:19.736377 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 08:17:19.736377 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 08:17:19.736377 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 08:17:19.736377 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 21 08:17:19.736377 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 21 08:17:19.736377 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 21 08:17:19.737341 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 21 08:17:19.737341 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 08:17:19.737341 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 08:17:19.737341 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 08:17:19.737341 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 08:17:19.737341 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 08:17:19.737341 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 08:17:19.737341 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 08:17:19.737341 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application with Lock( Jul 21 08:17:19.737341 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 08:17:19.737341 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self._enter() Jul 21 08:17:19.737341 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 08:17:19.737341 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 08:17:19.737341 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 08:17:19.737341 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self.creator() Jul 21 08:17:19.737341 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 08:17:19.737341 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application created_value = creator( Jul 21 08:17:19.738216 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 21 08:17:19.738216 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 21 08:17:19.738216 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 08:17:19.738216 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application Jul 21 08:17:19.745645 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 322/650] 10.222.0.47 () {56 vars in 1108 bytes} [Thu Jul 21 08:17:19 2022] GET /identity/v3/domains/Default => generated 87 bytes in 242 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 08:17:19.746910 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4d621ca-3dad-4f97-93c0-706b2f1c19b4 None admin] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:17:19.793533 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf5b78d2-7d09-4c0b-8777-1f888f8f0f97 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:17:19.795708 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf5b78d2-7d09-4c0b-8777-1f888f8f0f97 None admin] RBAC: Authorization granted {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:17:19.873053 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4d621ca-3dad-4f97-93c0-706b2f1c19b4 None admin] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:17:19.886211 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG dbcounter [-] [80245] Writing DB stats keystone:SELECT=100 {{(pid=80245) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:17:19.906378 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: [pid: 80245|app: 0|req: 329/651] 10.222.0.47 () {58 vars in 1282 bytes} [Thu Jul 21 08:17:19 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 401 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 21 08:17:19.907583 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf5b78d2-7d09-4c0b-8777-1f888f8f0f97 None admin] Authenticating user token {{(pid=80245) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:17:19.921509 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ad4ad4f-d9c5-4bfd-a5a7-28775ef3f940 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:17:19.922203 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ad4ad4f-d9c5-4bfd-a5a7-28775ef3f940 None admin] REQUEST_METHOD: `GET` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:17:19.922490 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ad4ad4f-d9c5-4bfd-a5a7-28775ef3f940 None admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:17:19.922770 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ad4ad4f-d9c5-4bfd-a5a7-28775ef3f940 None admin] PATH_INFO: `/v3/roles` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:17:19.923572 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ad4ad4f-d9c5-4bfd-a5a7-28775ef3f940 None admin] RBAC: Adding query filter params (name=service) {{(pid=80244) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 08:17:19.923879 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ad4ad4f-d9c5-4bfd-a5a7-28775ef3f940 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:17:19.926000 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ad4ad4f-d9c5-4bfd-a5a7-28775ef3f940 None admin] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:17:19.940541 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 323/652] 10.222.0.47 () {56 vars in 1103 bytes} [Thu Jul 21 08:17:19 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 08:17:19.940541 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ad4ad4f-d9c5-4bfd-a5a7-28775ef3f940 None admin] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:17:20.005543 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG dbcounter [-] [80244] Writing DB stats keystone:SELECT=100 {{(pid=80244) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:17:20.031586 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf5b78d2-7d09-4c0b-8777-1f888f8f0f97 None admin] Validating token access rules against request {{(pid=80245) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:17:20.055854 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ad4ad4f-d9c5-4bfd-a5a7-28775ef3f940 None admin] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:17:20.067968 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f374319-4679-407e-aa91-4a4fdde11d9d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80245) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:17:20.068809 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f374319-4679-407e-aa91-4a4fdde11d9d None admin] REQUEST_METHOD: `GET` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:17:20.068921 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f374319-4679-407e-aa91-4a4fdde11d9d None admin] SCRIPT_NAME: `/identity` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:17:20.069190 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f374319-4679-407e-aa91-4a4fdde11d9d None admin] PATH_INFO: `/v3/domains` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:17:20.069918 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f374319-4679-407e-aa91-4a4fdde11d9d None admin] RBAC: Adding query filter params (name=Default) {{(pid=80245) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 08:17:20.070281 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f374319-4679-407e-aa91-4a4fdde11d9d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:17:20.071627 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f374319-4679-407e-aa91-4a4fdde11d9d None admin] RBAC: Authorization granted {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:17:20.087360 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: [pid: 80245|app: 0|req: 330/653] 10.222.0.47 () {56 vars in 1109 bytes} [Thu Jul 21 08:17:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 08:17:20.087360 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f374319-4679-407e-aa91-4a4fdde11d9d None admin] Authenticating user token {{(pid=80245) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:17:20.102267 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-628517e5-38c4-43ac-b441-2fc5f10c03a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:17:20.103463 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-628517e5-38c4-43ac-b441-2fc5f10c03a6 None admin] REQUEST_METHOD: `GET` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:17:20.104085 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-628517e5-38c4-43ac-b441-2fc5f10c03a6 None admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:17:20.104721 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-628517e5-38c4-43ac-b441-2fc5f10c03a6 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:17:20.117368 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-628517e5-38c4-43ac-b441-2fc5f10c03a6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:17:20.119210 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-628517e5-38c4-43ac-b441-2fc5f10c03a6 None admin] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:17:20.136466 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG dbcounter [-] [80245] Writing DB stats keystone:SELECT=100 {{(pid=80245) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:17:20.139867 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application [None req-628517e5-38c4-43ac-b441-2fc5f10c03a6 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 08:17:20.139867 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 08:17:20.139867 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 08:17:20.139867 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 08:17:20.139867 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 08:17:20.139867 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 08:17:20.139867 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 08:17:20.139867 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 08:17:20.139867 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 08:17:20.139867 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 08:17:20.139867 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 08:17:20.139867 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 08:17:20.139867 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 21 08:17:20.139867 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 21 08:17:20.139867 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 21 08:17:20.140733 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 21 08:17:20.140733 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 08:17:20.140733 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 08:17:20.140733 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 08:17:20.140733 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 08:17:20.140733 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 08:17:20.140733 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 08:17:20.140733 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 08:17:20.140733 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application with Lock( Jul 21 08:17:20.140733 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 08:17:20.140733 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self._enter() Jul 21 08:17:20.140733 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 08:17:20.140733 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 08:17:20.140733 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 08:17:20.140733 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self.creator() Jul 21 08:17:20.140733 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 08:17:20.140733 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application created_value = creator( Jul 21 08:17:20.141694 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 21 08:17:20.141694 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 21 08:17:20.141694 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 08:17:20.141694 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application Jul 21 08:17:20.143943 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 324/654] 10.222.0.47 () {56 vars in 1108 bytes} [Thu Jul 21 08:17:19 2022] GET /identity/v3/domains/Default => generated 87 bytes in 207 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 08:17:20.146344 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-628517e5-38c4-43ac-b441-2fc5f10c03a6 None admin] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:17:20.198967 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f374319-4679-407e-aa91-4a4fdde11d9d None admin] Validating token access rules against request {{(pid=80245) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:17:20.234810 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12887eb8-1ab9-4d03-81ba-2d7d1ad2cf92 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80245) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:17:20.235667 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12887eb8-1ab9-4d03-81ba-2d7d1ad2cf92 None admin] REQUEST_METHOD: `GET` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:17:20.236253 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12887eb8-1ab9-4d03-81ba-2d7d1ad2cf92 None admin] SCRIPT_NAME: `/identity` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:17:20.236394 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12887eb8-1ab9-4d03-81ba-2d7d1ad2cf92 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:17:20.249287 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12887eb8-1ab9-4d03-81ba-2d7d1ad2cf92 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:17:20.250866 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12887eb8-1ab9-4d03-81ba-2d7d1ad2cf92 None admin] RBAC: Authorization granted {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:17:20.262326 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-628517e5-38c4-43ac-b441-2fc5f10c03a6 None admin] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:17:20.269385 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application [None req-12887eb8-1ab9-4d03-81ba-2d7d1ad2cf92 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 08:17:20.269385 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 08:17:20.269385 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 08:17:20.269385 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 08:17:20.269385 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 08:17:20.269385 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 08:17:20.269385 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 08:17:20.269385 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 08:17:20.269385 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 08:17:20.269385 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 08:17:20.269385 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 08:17:20.269385 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 08:17:20.269385 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 21 08:17:20.269385 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 21 08:17:20.269385 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 21 08:17:20.270819 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 21 08:17:20.270819 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 08:17:20.270819 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 08:17:20.270819 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 08:17:20.270819 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 08:17:20.270819 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 08:17:20.270819 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 08:17:20.270819 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 08:17:20.270819 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application with Lock( Jul 21 08:17:20.270819 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 08:17:20.270819 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application return self._enter() Jul 21 08:17:20.270819 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 08:17:20.270819 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 08:17:20.270819 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 08:17:20.270819 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application return self.creator() Jul 21 08:17:20.270819 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 08:17:20.270819 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application created_value = creator( Jul 21 08:17:20.271730 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 21 08:17:20.271730 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 21 08:17:20.271730 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 08:17:20.271730 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application Jul 21 08:17:20.271730 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: [pid: 80245|app: 0|req: 331/655] 10.222.0.47 () {56 vars in 1108 bytes} [Thu Jul 21 08:17:20 2022] GET /identity/v3/domains/Default => generated 87 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 08:17:20.272017 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12887eb8-1ab9-4d03-81ba-2d7d1ad2cf92 None admin] Authenticating user token {{(pid=80245) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:17:20.282591 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG dbcounter [-] [80244] Writing DB stats keystone:SELECT=100 {{(pid=80244) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:17:20.311165 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49b6ebce-383e-4a12-9464-29845bc39463 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:17:20.312339 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49b6ebce-383e-4a12-9464-29845bc39463 None admin] REQUEST_METHOD: `GET` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:17:20.312844 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49b6ebce-383e-4a12-9464-29845bc39463 None admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:17:20.313407 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49b6ebce-383e-4a12-9464-29845bc39463 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:17:20.322574 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49b6ebce-383e-4a12-9464-29845bc39463 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:17:20.324505 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49b6ebce-383e-4a12-9464-29845bc39463 None admin] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:17:20.340383 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application [None req-49b6ebce-383e-4a12-9464-29845bc39463 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Jul 21 08:17:20.340383 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 08:17:20.340383 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 08:17:20.340383 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 08:17:20.340383 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 08:17:20.340383 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 08:17:20.340383 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 08:17:20.340383 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 08:17:20.340383 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 08:17:20.340383 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 08:17:20.340383 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 08:17:20.340383 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 08:17:20.340383 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 21 08:17:20.340383 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 21 08:17:20.340383 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 21 08:17:20.340383 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 21 08:17:20.341347 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 08:17:20.341347 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 08:17:20.341347 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 21 08:17:20.341347 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 21 08:17:20.341347 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 21 08:17:20.341347 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 21 08:17:20.341347 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 08:17:20.341347 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 08:17:20.341347 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 08:17:20.341347 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 08:17:20.341347 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 08:17:20.341347 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application with Lock( Jul 21 08:17:20.341347 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 08:17:20.341347 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self._enter() Jul 21 08:17:20.341347 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 08:17:20.341347 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 08:17:20.341347 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 08:17:20.341347 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self.creator() Jul 21 08:17:20.342287 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 08:17:20.342287 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application created_value = creator( Jul 21 08:17:20.342287 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 21 08:17:20.342287 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 21 08:17:20.342287 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 21 08:17:20.342287 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 21 08:17:20.342287 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 21 08:17:20.342287 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 21 08:17:20.342287 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Jul 21 08:17:20.342287 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application Jul 21 08:17:20.349688 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 325/656] 10.222.0.47 () {56 vars in 1099 bytes} [Thu Jul 21 08:17:20 2022] GET /identity/v3/users/cinder => generated 84 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 08:17:20.350802 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49b6ebce-383e-4a12-9464-29845bc39463 None admin] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:17:20.372042 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG dbcounter [-] [80245] Writing DB stats keystone:SELECT=100 {{(pid=80245) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:17:20.378081 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12887eb8-1ab9-4d03-81ba-2d7d1ad2cf92 None admin] Validating token access rules against request {{(pid=80245) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:17:20.415400 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fde9db32-14a9-4c6b-b555-5ce4aab500b9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80245) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:17:20.416483 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fde9db32-14a9-4c6b-b555-5ce4aab500b9 None admin] REQUEST_METHOD: `GET` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:17:20.416767 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fde9db32-14a9-4c6b-b555-5ce4aab500b9 None admin] SCRIPT_NAME: `/identity` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:17:20.417078 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fde9db32-14a9-4c6b-b555-5ce4aab500b9 None admin] PATH_INFO: `/v3/domains` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:17:20.417752 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fde9db32-14a9-4c6b-b555-5ce4aab500b9 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80245) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 08:17:20.418067 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fde9db32-14a9-4c6b-b555-5ce4aab500b9 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:17:20.419492 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fde9db32-14a9-4c6b-b555-5ce4aab500b9 None admin] RBAC: Authorization granted {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:17:20.436400 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: [pid: 80245|app: 0|req: 332/657] 10.222.0.47 () {56 vars in 1109 bytes} [Thu Jul 21 08:17:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 08:17:20.436400 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fde9db32-14a9-4c6b-b555-5ce4aab500b9 None admin] Authenticating user token {{(pid=80245) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:17:20.468870 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49b6ebce-383e-4a12-9464-29845bc39463 None admin] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:17:20.518655 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ae21b1c-ed4e-4e15-8aa2-5a3d1d0be53f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:17:20.520488 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ae21b1c-ed4e-4e15-8aa2-5a3d1d0be53f None admin] REQUEST_METHOD: `GET` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:17:20.520681 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ae21b1c-ed4e-4e15-8aa2-5a3d1d0be53f None admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:17:20.520922 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ae21b1c-ed4e-4e15-8aa2-5a3d1d0be53f None admin] PATH_INFO: `/v3/domains` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:17:20.522025 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ae21b1c-ed4e-4e15-8aa2-5a3d1d0be53f None admin] RBAC: Adding query filter params (name=Default) {{(pid=80244) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 08:17:20.522339 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ae21b1c-ed4e-4e15-8aa2-5a3d1d0be53f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:17:20.524350 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ae21b1c-ed4e-4e15-8aa2-5a3d1d0be53f None admin] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:17:20.544839 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 326/658] 10.222.0.47 () {56 vars in 1109 bytes} [Thu Jul 21 08:17:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 08:17:20.546224 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f106728e-4648-42bb-88d5-fb734df603cd None None] REQUEST_METHOD: `GET` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:17:20.546518 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f106728e-4648-42bb-88d5-fb734df603cd None None] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:17:20.546806 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f106728e-4648-42bb-88d5-fb734df603cd None None] PATH_INFO: `/` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:17:20.547113 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fde9db32-14a9-4c6b-b555-5ce4aab500b9 None admin] Validating token access rules against request {{(pid=80245) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:17:20.548392 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 327/659] 10.222.0.47 () {54 vars in 926 bytes} [Thu Jul 21 08:17:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 08:17:20.548873 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f106728e-4648-42bb-88d5-fb734df603cd None None] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:17:20.581156 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG dbcounter [-] [80244] Writing DB stats keystone:SELECT=100 {{(pid=80244) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:17:20.592712 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a85e1be-4729-4816-847d-9b9017a58073 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80245) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:17:20.593400 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a85e1be-4729-4816-847d-9b9017a58073 None admin] REQUEST_METHOD: `GET` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:17:20.593682 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a85e1be-4729-4816-847d-9b9017a58073 None admin] SCRIPT_NAME: `/identity` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:17:20.593958 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a85e1be-4729-4816-847d-9b9017a58073 None admin] PATH_INFO: `/v3/users` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:17:20.594794 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a85e1be-4729-4816-847d-9b9017a58073 None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=80245) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 08:17:20.595112 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a85e1be-4729-4816-847d-9b9017a58073 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:17:20.596576 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a85e1be-4729-4816-847d-9b9017a58073 None admin] RBAC: Authorization granted {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:17:20.619565 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: [pid: 80245|app: 0|req: 333/660] 10.222.0.47 () {56 vars in 1137 bytes} [Thu Jul 21 08:17:20 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 08:17:20.619565 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a85e1be-4729-4816-847d-9b9017a58073 None admin] Authenticating user token {{(pid=80245) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:17:20.634445 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG dbcounter [-] [80245] Writing DB stats keystone:SELECT=100 {{(pid=80245) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:17:20.663979 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f106728e-4648-42bb-88d5-fb734df603cd None None] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:17:20.701758 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91744e75-ef2c-4c2d-82e4-11f0f4845e38 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:17:20.702558 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91744e75-ef2c-4c2d-82e4-11f0f4845e38 None admin] REQUEST_METHOD: `GET` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:17:20.702847 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91744e75-ef2c-4c2d-82e4-11f0f4845e38 None admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:17:20.703122 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91744e75-ef2c-4c2d-82e4-11f0f4845e38 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:17:20.710051 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91744e75-ef2c-4c2d-82e4-11f0f4845e38 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:17:20.711611 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91744e75-ef2c-4c2d-82e4-11f0f4845e38 None admin] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:17:20.726098 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application [None req-91744e75-ef2c-4c2d-82e4-11f0f4845e38 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 21 08:17:20.726098 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 08:17:20.726098 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 08:17:20.726098 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 08:17:20.726098 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 08:17:20.726098 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 08:17:20.726098 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 08:17:20.726098 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 08:17:20.726098 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 08:17:20.726098 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 08:17:20.726098 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 08:17:20.726098 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 08:17:20.726098 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 21 08:17:20.726098 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 21 08:17:20.726098 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 21 08:17:20.726975 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 21 08:17:20.726975 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 08:17:20.726975 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 08:17:20.726975 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 08:17:20.726975 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 08:17:20.726975 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 08:17:20.726975 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 08:17:20.726975 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 08:17:20.726975 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application with Lock( Jul 21 08:17:20.726975 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 08:17:20.726975 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self._enter() Jul 21 08:17:20.726975 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 08:17:20.726975 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 08:17:20.726975 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 08:17:20.726975 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self.creator() Jul 21 08:17:20.726975 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 08:17:20.726975 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application created_value = creator( Jul 21 08:17:20.727883 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 21 08:17:20.727883 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 21 08:17:20.727883 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 21 08:17:20.727883 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 21 08:17:20.727883 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 21 08:17:20.727883 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 21 08:17:20.727883 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 21 08:17:20.727883 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application Jul 21 08:17:20.727883 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 328/661] 10.222.0.47 () {56 vars in 1111 bytes} [Thu Jul 21 08:17:20 2022] GET /identity/v3/projects/service => generated 88 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 08:17:20.729002 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cc1d55f-1474-44ef-b74f-fe457347fde1 None None] REQUEST_METHOD: `POST` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:17:20.729351 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cc1d55f-1474-44ef-b74f-fe457347fde1 None None] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:17:20.729705 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cc1d55f-1474-44ef-b74f-fe457347fde1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:17:20.735184 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a85e1be-4729-4816-847d-9b9017a58073 None admin] Validating token access rules against request {{(pid=80245) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:17:20.746614 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.auth.core [None req-0cc1d55f-1474-44ef-b74f-fe457347fde1 None None] MFA Rules not processed for user `2dce40b8d74c41ec94cc36e6ff05d412`. Rule list: `[]` (Enabled: `True`). {{(pid=80244) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 08:17:20.747812 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.fernet_utils [None req-0cc1d55f-1474-44ef-b74f-fe457347fde1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80244) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 08:17:20.772906 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ba11587-191c-4cd6-8caf-495915bcd342 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80245) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:17:20.773707 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ba11587-191c-4cd6-8caf-495915bcd342 None admin] REQUEST_METHOD: `GET` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:17:20.773989 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ba11587-191c-4cd6-8caf-495915bcd342 None admin] SCRIPT_NAME: `/identity` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:17:20.774268 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ba11587-191c-4cd6-8caf-495915bcd342 None admin] PATH_INFO: `/v3/users/neutron` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:17:20.781645 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ba11587-191c-4cd6-8caf-495915bcd342 None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:17:20.783066 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ba11587-191c-4cd6-8caf-495915bcd342 None admin] RBAC: Authorization granted {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:17:20.798579 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application [None req-0ba11587-191c-4cd6-8caf-495915bcd342 None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Jul 21 08:17:20.798579 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 08:17:20.798579 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 08:17:20.798579 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 08:17:20.798579 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 08:17:20.798579 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 08:17:20.798579 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 08:17:20.798579 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 08:17:20.798579 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 08:17:20.798579 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 08:17:20.798579 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 08:17:20.798579 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 08:17:20.798579 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 21 08:17:20.798579 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 21 08:17:20.798579 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 21 08:17:20.798579 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 21 08:17:20.799590 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 08:17:20.799590 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 08:17:20.799590 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 21 08:17:20.799590 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 21 08:17:20.799590 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 21 08:17:20.799590 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 21 08:17:20.799590 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 08:17:20.799590 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 08:17:20.799590 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 08:17:20.799590 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 08:17:20.799590 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 08:17:20.799590 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application with Lock( Jul 21 08:17:20.799590 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 08:17:20.799590 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application return self._enter() Jul 21 08:17:20.799590 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 08:17:20.799590 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 08:17:20.799590 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 08:17:20.799590 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application return self.creator() Jul 21 08:17:20.800607 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 08:17:20.800607 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application created_value = creator( Jul 21 08:17:20.800607 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 21 08:17:20.800607 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 21 08:17:20.800607 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 21 08:17:20.800607 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 21 08:17:20.800607 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 21 08:17:20.800607 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 21 08:17:20.800607 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Jul 21 08:17:20.800607 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application Jul 21 08:17:20.800607 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: [pid: 80245|app: 0|req: 334/662] 10.222.0.47 () {56 vars in 1102 bytes} [Thu Jul 21 08:17:20 2022] GET /identity/v3/users/neutron => generated 85 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 08:17:20.800607 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ba11587-191c-4cd6-8caf-495915bcd342 None admin] Authenticating user token {{(pid=80245) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:17:20.831576 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 329/663] 10.222.0.47 () {58 vars in 1023 bytes} [Thu Jul 21 08:17:20 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 21 08:17:20.831576 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cc1d55f-1474-44ef-b74f-fe457347fde1 None None] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:17:20.870243 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG dbcounter [-] [80244] Writing DB stats keystone:SELECT=100 {{(pid=80244) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:17:20.900321 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG dbcounter [-] [80245] Writing DB stats keystone:SELECT=100 {{(pid=80245) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:17:20.929073 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ba11587-191c-4cd6-8caf-495915bcd342 None admin] Validating token access rules against request {{(pid=80245) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:17:20.968278 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cc1d55f-1474-44ef-b74f-fe457347fde1 None None] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:17:20.975889 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b30fbc6-30ab-4b1e-9cd5-ba2c7fc80483 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80245) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:17:20.977380 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b30fbc6-30ab-4b1e-9cd5-ba2c7fc80483 None admin] REQUEST_METHOD: `GET` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:17:20.977893 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b30fbc6-30ab-4b1e-9cd5-ba2c7fc80483 None admin] SCRIPT_NAME: `/identity` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:17:20.978391 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b30fbc6-30ab-4b1e-9cd5-ba2c7fc80483 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:17:21.012517 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-226b5226-ee9b-4f5e-8e2e-c8ccdc29c54b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:17:21.012882 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-226b5226-ee9b-4f5e-8e2e-c8ccdc29c54b None admin] REQUEST_METHOD: `GET` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:17:21.013108 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-226b5226-ee9b-4f5e-8e2e-c8ccdc29c54b None admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:17:21.013390 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-226b5226-ee9b-4f5e-8e2e-c8ccdc29c54b None admin] PATH_INFO: `/v3/projects` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:17:21.014195 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-226b5226-ee9b-4f5e-8e2e-c8ccdc29c54b None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80244) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 08:17:21.014507 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-226b5226-ee9b-4f5e-8e2e-c8ccdc29c54b None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:17:21.016003 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-226b5226-ee9b-4f5e-8e2e-c8ccdc29c54b None admin] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:17:21.029201 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 330/664] 10.222.0.47 () {56 vars in 1148 bytes} [Thu Jul 21 08:17:20 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 08:17:21.030062 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-226b5226-ee9b-4f5e-8e2e-c8ccdc29c54b None admin] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:17:21.074169 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b30fbc6-30ab-4b1e-9cd5-ba2c7fc80483 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:17:21.076210 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b30fbc6-30ab-4b1e-9cd5-ba2c7fc80483 None admin] RBAC: Authorization granted {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:17:21.133664 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG dbcounter [-] [80244] Writing DB stats keystone:SELECT=100 {{(pid=80244) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:17:21.170763 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-226b5226-ee9b-4f5e-8e2e-c8ccdc29c54b None admin] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:17:21.180244 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG dbcounter [-] [80245] Writing DB stats keystone:SELECT=100 {{(pid=80245) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:17:21.220474 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: [pid: 80245|app: 0|req: 335/665] 10.222.0.47 () {58 vars in 1282 bytes} [Thu Jul 21 08:17:20 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 420 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 21 08:17:21.221262 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96a1a016-e045-4550-ae26-efe6c7472a6d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:17:21.221646 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b30fbc6-30ab-4b1e-9cd5-ba2c7fc80483 None admin] Authenticating user token {{(pid=80245) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:17:21.224265 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96a1a016-e045-4550-ae26-efe6c7472a6d None admin] REQUEST_METHOD: `GET` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:17:21.225881 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96a1a016-e045-4550-ae26-efe6c7472a6d None admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:17:21.226162 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96a1a016-e045-4550-ae26-efe6c7472a6d None admin] PATH_INFO: `/v3/users` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:17:21.227022 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96a1a016-e045-4550-ae26-efe6c7472a6d None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=80244) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 08:17:21.227354 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96a1a016-e045-4550-ae26-efe6c7472a6d None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:17:21.232078 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96a1a016-e045-4550-ae26-efe6c7472a6d None admin] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:17:21.253132 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 331/666] 10.222.0.47 () {56 vars in 1139 bytes} [Thu Jul 21 08:17:21 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 223 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 08:17:21.253990 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96a1a016-e045-4550-ae26-efe6c7472a6d None admin] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:17:21.340068 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b30fbc6-30ab-4b1e-9cd5-ba2c7fc80483 None admin] Validating token access rules against request {{(pid=80245) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:17:21.362874 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96a1a016-e045-4550-ae26-efe6c7472a6d None admin] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:17:21.374715 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4caeca33-73be-46db-95b2-a0d26b8c5259 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80245) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:17:21.375554 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4caeca33-73be-46db-95b2-a0d26b8c5259 None admin] REQUEST_METHOD: `GET` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:17:21.375838 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4caeca33-73be-46db-95b2-a0d26b8c5259 None admin] SCRIPT_NAME: `/identity` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:17:21.376118 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4caeca33-73be-46db-95b2-a0d26b8c5259 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:17:21.387947 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4caeca33-73be-46db-95b2-a0d26b8c5259 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 21 08:17:21.395822 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application [None req-4caeca33-73be-46db-95b2-a0d26b8c5259 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 21 08:17:21.395822 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 08:17:21.395822 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 08:17:21.395822 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 08:17:21.395822 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 08:17:21.395822 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 08:17:21.395822 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 08:17:21.395822 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 08:17:21.395822 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 08:17:21.395822 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 08:17:21.395822 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 08:17:21.395822 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 08:17:21.395822 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 21 08:17:21.395822 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 21 08:17:21.395822 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 21 08:17:21.395822 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 21 08:17:21.397015 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 21 08:17:21.397015 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 21 08:17:21.397015 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 21 08:17:21.397015 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 21 08:17:21.397015 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 08:17:21.397015 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 08:17:21.397015 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 08:17:21.397015 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 08:17:21.397015 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 08:17:21.397015 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 08:17:21.397015 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 08:17:21.397015 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application with Lock( Jul 21 08:17:21.397015 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 08:17:21.397015 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application return self._enter() Jul 21 08:17:21.397015 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 08:17:21.397015 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 08:17:21.398135 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 08:17:21.398135 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application return self.creator() Jul 21 08:17:21.398135 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 08:17:21.398135 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application created_value = creator( Jul 21 08:17:21.398135 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 21 08:17:21.398135 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 21 08:17:21.398135 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 21 08:17:21.398135 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 21 08:17:21.398135 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 21 08:17:21.398135 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 21 08:17:21.398135 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 21 08:17:21.398135 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application Jul 21 08:17:21.398135 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: [pid: 80245|app: 0|req: 336/667] 10.222.0.47 () {56 vars in 1102 bytes} [Thu Jul 21 08:17:21 2022] GET /identity/v3/roles/service => generated 85 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 08:17:21.399006 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4caeca33-73be-46db-95b2-a0d26b8c5259 None admin] Authenticating user token {{(pid=80245) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:17:21.408442 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e2198a8-3aaa-467c-a7b6-5f9e7435eb6b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:17:21.409165 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e2198a8-3aaa-467c-a7b6-5f9e7435eb6b None admin] REQUEST_METHOD: `GET` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:17:21.409446 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e2198a8-3aaa-467c-a7b6-5f9e7435eb6b None admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:17:21.409717 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e2198a8-3aaa-467c-a7b6-5f9e7435eb6b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:17:21.410710 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e2198a8-3aaa-467c-a7b6-5f9e7435eb6b None admin] RBAC: Adding query filter params (role.id=50d1ec7f947f42acb65c57e07f2269d6, scope.project.id=4faef13aa50e4e36b6c265fb1398fa14, user.id=9b0524aed8c74abd8949b860d6b93936) {{(pid=80244) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 08:17:21.411037 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e2198a8-3aaa-467c-a7b6-5f9e7435eb6b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:17:21.412709 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e2198a8-3aaa-467c-a7b6-5f9e7435eb6b None admin] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:17:21.427359 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 332/668] 10.222.0.47 () {56 vars in 1374 bytes} [Thu Jul 21 08:17:21 2022] GET /identity/v3/role_assignments?user.id=9b0524aed8c74abd8949b860d6b93936&scope.project.id=4faef13aa50e4e36b6c265fb1398fa14&role.id=50d1ec7f947f42acb65c57e07f2269d6 => generated 608 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 08:17:21.427359 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e2198a8-3aaa-467c-a7b6-5f9e7435eb6b None admin] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:17:21.430593 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG dbcounter [-] [80244] Writing DB stats keystone:SELECT=100 {{(pid=80244) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:17:21.444219 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG dbcounter [-] [80245] Writing DB stats keystone:SELECT=100 {{(pid=80245) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:17:21.533296 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4caeca33-73be-46db-95b2-a0d26b8c5259 None admin] Validating token access rules against request {{(pid=80245) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:17:21.556202 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e2198a8-3aaa-467c-a7b6-5f9e7435eb6b None admin] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:17:21.574644 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3131b5bf-0b26-4dd9-b598-5cafd0fc1de0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80245) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:17:21.575511 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3131b5bf-0b26-4dd9-b598-5cafd0fc1de0 None admin] REQUEST_METHOD: `GET` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:17:21.575795 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3131b5bf-0b26-4dd9-b598-5cafd0fc1de0 None admin] SCRIPT_NAME: `/identity` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:17:21.576069 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3131b5bf-0b26-4dd9-b598-5cafd0fc1de0 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:17:21.588608 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3131b5bf-0b26-4dd9-b598-5cafd0fc1de0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:17:21.590073 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3131b5bf-0b26-4dd9-b598-5cafd0fc1de0 None admin] RBAC: Authorization granted {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:17:21.596570 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32f38851-3e8d-4c98-b0a9-36cacbae1d87 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:17:21.597240 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32f38851-3e8d-4c98-b0a9-36cacbae1d87 None admin] REQUEST_METHOD: `GET` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:17:21.597846 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32f38851-3e8d-4c98-b0a9-36cacbae1d87 None admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:17:21.597972 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32f38851-3e8d-4c98-b0a9-36cacbae1d87 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:17:21.608764 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application [None req-3131b5bf-0b26-4dd9-b598-5cafd0fc1de0 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 08:17:21.608764 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 08:17:21.608764 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 08:17:21.608764 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 08:17:21.608764 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 08:17:21.608764 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 08:17:21.608764 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 08:17:21.608764 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 08:17:21.608764 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 08:17:21.608764 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 08:17:21.608764 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 08:17:21.608764 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 08:17:21.608764 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 21 08:17:21.608764 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 21 08:17:21.608764 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 21 08:17:21.609872 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 21 08:17:21.609872 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 08:17:21.609872 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 08:17:21.609872 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 08:17:21.609872 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 08:17:21.609872 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 08:17:21.609872 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 08:17:21.609872 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 08:17:21.609872 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application with Lock( Jul 21 08:17:21.609872 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 08:17:21.609872 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application return self._enter() Jul 21 08:17:21.609872 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 08:17:21.609872 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 08:17:21.609872 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 08:17:21.609872 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application return self.creator() Jul 21 08:17:21.609872 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 08:17:21.609872 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application created_value = creator( Jul 21 08:17:21.610904 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 21 08:17:21.610904 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 21 08:17:21.610904 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 08:17:21.610904 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application Jul 21 08:17:21.610904 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: [pid: 80245|app: 0|req: 337/669] 10.222.0.47 () {56 vars in 1108 bytes} [Thu Jul 21 08:17:21 2022] GET /identity/v3/domains/Default => generated 87 bytes in 212 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 08:17:21.610904 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3131b5bf-0b26-4dd9-b598-5cafd0fc1de0 None admin] Authenticating user token {{(pid=80245) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:17:21.683898 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32f38851-3e8d-4c98-b0a9-36cacbae1d87 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:17:21.685535 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32f38851-3e8d-4c98-b0a9-36cacbae1d87 None admin] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:17:21.687580 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG dbcounter [-] [80244] Writing DB stats keystone:SELECT=100 {{(pid=80244) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:17:21.713369 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG dbcounter [-] [80245] Writing DB stats keystone:SELECT=100 {{(pid=80245) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:17:21.730527 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3131b5bf-0b26-4dd9-b598-5cafd0fc1de0 None admin] Validating token access rules against request {{(pid=80245) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:17:21.770258 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d18ada6-c719-43bc-869f-536b5281b4b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80245) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:17:21.770974 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d18ada6-c719-43bc-869f-536b5281b4b8 None admin] REQUEST_METHOD: `GET` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:17:21.771404 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d18ada6-c719-43bc-869f-536b5281b4b8 None admin] SCRIPT_NAME: `/identity` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:17:21.771787 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d18ada6-c719-43bc-869f-536b5281b4b8 None admin] PATH_INFO: `/v3/roles` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:17:21.772611 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d18ada6-c719-43bc-869f-536b5281b4b8 None admin] RBAC: Adding query filter params (name=service) {{(pid=80245) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 08:17:21.772922 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d18ada6-c719-43bc-869f-536b5281b4b8 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:17:21.774455 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d18ada6-c719-43bc-869f-536b5281b4b8 None admin] RBAC: Authorization granted {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:17:21.783529 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: [pid: 80245|app: 0|req: 338/670] 10.222.0.47 () {56 vars in 1103 bytes} [Thu Jul 21 08:17:21 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 08:17:21.789390 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-061970e6-ab2f-46be-a170-bd5fe44106dc None None] REQUEST_METHOD: `GET` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:17:21.789685 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-061970e6-ab2f-46be-a170-bd5fe44106dc None None] SCRIPT_NAME: `/identity` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:17:21.790109 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-061970e6-ab2f-46be-a170-bd5fe44106dc None None] PATH_INFO: `/` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:17:21.791598 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: [pid: 80245|app: 0|req: 339/671] 10.222.0.47 () {54 vars in 926 bytes} [Thu Jul 21 08:17:21 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 08:17:21.792129 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-061970e6-ab2f-46be-a170-bd5fe44106dc None None] Authenticating user token {{(pid=80245) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:17:21.804594 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 333/672] 10.222.0.47 () {58 vars in 1282 bytes} [Thu Jul 21 08:17:21 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 380 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 21 08:17:21.804594 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32f38851-3e8d-4c98-b0a9-36cacbae1d87 None admin] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:17:21.905779 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-061970e6-ab2f-46be-a170-bd5fe44106dc None None] Validating token access rules against request {{(pid=80245) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:17:21.913711 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32f38851-3e8d-4c98-b0a9-36cacbae1d87 None admin] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:17:21.915420 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG dbcounter [-] [80244] Writing DB stats keystone:SELECT=100 {{(pid=80244) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:17:21.944911 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c45d47a0-b2da-4650-a121-53af5fd25b2d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80245) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:17:21.945489 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c45d47a0-b2da-4650-a121-53af5fd25b2d None admin] REQUEST_METHOD: `GET` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:17:21.945770 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c45d47a0-b2da-4650-a121-53af5fd25b2d None admin] SCRIPT_NAME: `/identity` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:17:21.946058 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c45d47a0-b2da-4650-a121-53af5fd25b2d None admin] PATH_INFO: `/v3/domains` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:17:21.946915 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c45d47a0-b2da-4650-a121-53af5fd25b2d None admin] RBAC: Adding query filter params (name=Default) {{(pid=80245) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 08:17:21.947023 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c45d47a0-b2da-4650-a121-53af5fd25b2d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:17:21.948443 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c45d47a0-b2da-4650-a121-53af5fd25b2d None admin] RBAC: Authorization granted {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:17:21.956050 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47c15a5e-7325-4f7a-ad9d-1276b2965a3e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:17:21.956908 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47c15a5e-7325-4f7a-ad9d-1276b2965a3e None admin] REQUEST_METHOD: `GET` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:17:21.957146 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47c15a5e-7325-4f7a-ad9d-1276b2965a3e None admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:17:21.957420 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47c15a5e-7325-4f7a-ad9d-1276b2965a3e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:17:21.963615 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: [pid: 80245|app: 0|req: 340/673] 10.222.0.47 () {56 vars in 1109 bytes} [Thu Jul 21 08:17:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 08:17:21.965448 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60b96b34-b72e-4e95-a327-a65cacdd3903 None None] REQUEST_METHOD: `POST` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:17:21.965742 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60b96b34-b72e-4e95-a327-a65cacdd3903 None None] SCRIPT_NAME: `/identity` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:17:21.966019 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60b96b34-b72e-4e95-a327-a65cacdd3903 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:17:21.969426 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47c15a5e-7325-4f7a-ad9d-1276b2965a3e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:17:21.970887 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47c15a5e-7325-4f7a-ad9d-1276b2965a3e None admin] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:17:21.993340 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application [None req-47c15a5e-7325-4f7a-ad9d-1276b2965a3e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 08:17:21.993340 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 08:17:21.993340 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 08:17:21.993340 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 08:17:21.993340 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 08:17:21.993340 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 08:17:21.993340 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 08:17:21.993340 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 08:17:21.993340 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 08:17:21.993340 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 08:17:21.993340 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 08:17:21.993340 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 08:17:21.993340 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 21 08:17:21.993340 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 21 08:17:21.993340 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 21 08:17:21.994407 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 21 08:17:21.994407 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 08:17:21.994407 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 08:17:21.994407 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 08:17:21.994407 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 08:17:21.994407 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 08:17:21.994407 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 08:17:21.994407 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 08:17:21.994407 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application with Lock( Jul 21 08:17:21.994407 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 08:17:21.994407 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self._enter() Jul 21 08:17:21.994407 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 08:17:21.994407 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 08:17:21.994407 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 08:17:21.994407 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self.creator() Jul 21 08:17:21.994407 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 08:17:21.994407 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application created_value = creator( Jul 21 08:17:21.995301 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 21 08:17:21.995301 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 21 08:17:21.995301 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 08:17:21.995301 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application Jul 21 08:17:21.997679 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.auth.core [None req-60b96b34-b72e-4e95-a327-a65cacdd3903 None None] MFA Rules not processed for user `2dce40b8d74c41ec94cc36e6ff05d412`. Rule list: `[]` (Enabled: `True`). {{(pid=80245) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 08:17:21.999380 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 334/674] 10.222.0.47 () {56 vars in 1108 bytes} [Thu Jul 21 08:17:21 2022] GET /identity/v3/domains/Default => generated 87 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 08:17:21.999555 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.fernet_utils [None req-60b96b34-b72e-4e95-a327-a65cacdd3903 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80245) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 08:17:21.999857 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47c15a5e-7325-4f7a-ad9d-1276b2965a3e None admin] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:17:22.002107 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG dbcounter [-] [80245] Writing DB stats keystone:SELECT=100 {{(pid=80245) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:17:22.077866 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: [pid: 80245|app: 0|req: 341/675] 10.222.0.47 () {58 vars in 1023 bytes} [Thu Jul 21 08:17:21 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 21 08:17:22.078726 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60b96b34-b72e-4e95-a327-a65cacdd3903 None None] Authenticating user token {{(pid=80245) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:17:22.108919 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47c15a5e-7325-4f7a-ad9d-1276b2965a3e None admin] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:17:22.145710 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6950122-1a06-4c6b-aac5-93bd12449b9c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:17:22.146498 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6950122-1a06-4c6b-aac5-93bd12449b9c None admin] REQUEST_METHOD: `GET` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:17:22.146778 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6950122-1a06-4c6b-aac5-93bd12449b9c None admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:17:22.147053 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6950122-1a06-4c6b-aac5-93bd12449b9c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:17:22.157591 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6950122-1a06-4c6b-aac5-93bd12449b9c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:17:22.159044 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6950122-1a06-4c6b-aac5-93bd12449b9c None admin] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:17:22.177175 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application [None req-d6950122-1a06-4c6b-aac5-93bd12449b9c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 08:17:22.177175 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 08:17:22.177175 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 08:17:22.177175 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 08:17:22.177175 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 08:17:22.177175 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 08:17:22.177175 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 08:17:22.177175 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 08:17:22.177175 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 08:17:22.177175 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 08:17:22.177175 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 08:17:22.177175 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 08:17:22.177175 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 21 08:17:22.177175 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 21 08:17:22.177175 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 21 08:17:22.178113 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 21 08:17:22.178113 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 08:17:22.178113 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 08:17:22.178113 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 08:17:22.178113 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 08:17:22.178113 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 08:17:22.178113 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 08:17:22.178113 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 08:17:22.178113 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application with Lock( Jul 21 08:17:22.178113 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 08:17:22.178113 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self._enter() Jul 21 08:17:22.178113 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 08:17:22.178113 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 08:17:22.178113 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 08:17:22.178113 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self.creator() Jul 21 08:17:22.178113 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 08:17:22.178113 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application created_value = creator( Jul 21 08:17:22.178986 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 21 08:17:22.178986 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 21 08:17:22.178986 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 08:17:22.178986 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application Jul 21 08:17:22.178986 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 335/676] 10.222.0.47 () {56 vars in 1108 bytes} [Thu Jul 21 08:17:21 2022] GET /identity/v3/domains/Default => generated 87 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 08:17:22.179278 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6950122-1a06-4c6b-aac5-93bd12449b9c None admin] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:17:22.188490 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG dbcounter [-] [80244] Writing DB stats keystone:SELECT=100 {{(pid=80244) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:17:22.202926 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60b96b34-b72e-4e95-a327-a65cacdd3903 None None] Validating token access rules against request {{(pid=80245) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:17:22.242240 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efca97b0-7361-43da-8764-11249a2b2ecb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80245) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:17:22.243301 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efca97b0-7361-43da-8764-11249a2b2ecb None admin] REQUEST_METHOD: `GET` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:17:22.243606 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efca97b0-7361-43da-8764-11249a2b2ecb None admin] SCRIPT_NAME: `/identity` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:17:22.243875 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efca97b0-7361-43da-8764-11249a2b2ecb None admin] PATH_INFO: `/v3/projects/service` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:17:22.251558 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efca97b0-7361-43da-8764-11249a2b2ecb None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:17:22.253011 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efca97b0-7361-43da-8764-11249a2b2ecb None admin] RBAC: Authorization granted {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:17:22.254892 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG dbcounter [-] [80245] Writing DB stats keystone:SELECT=100 {{(pid=80245) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:17:22.271965 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application [None req-efca97b0-7361-43da-8764-11249a2b2ecb None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 21 08:17:22.271965 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 08:17:22.271965 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 08:17:22.271965 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 08:17:22.271965 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 08:17:22.271965 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 08:17:22.271965 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 08:17:22.271965 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 08:17:22.271965 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 08:17:22.271965 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 08:17:22.271965 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 08:17:22.271965 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 08:17:22.271965 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 21 08:17:22.271965 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 21 08:17:22.271965 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 21 08:17:22.272906 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 21 08:17:22.272906 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 08:17:22.272906 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 08:17:22.272906 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 08:17:22.272906 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 08:17:22.272906 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 08:17:22.272906 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 08:17:22.272906 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 08:17:22.272906 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application with Lock( Jul 21 08:17:22.272906 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 08:17:22.272906 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application return self._enter() Jul 21 08:17:22.272906 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 08:17:22.272906 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 08:17:22.272906 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 08:17:22.272906 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application return self.creator() Jul 21 08:17:22.272906 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 08:17:22.272906 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application created_value = creator( Jul 21 08:17:22.273768 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 21 08:17:22.273768 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 21 08:17:22.273768 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 21 08:17:22.273768 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 21 08:17:22.273768 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 21 08:17:22.273768 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 21 08:17:22.273768 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 21 08:17:22.273768 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application Jul 21 08:17:22.273768 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: [pid: 80245|app: 0|req: 342/677] 10.222.0.47 () {56 vars in 1111 bytes} [Thu Jul 21 08:17:22 2022] GET /identity/v3/projects/service => generated 88 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 08:17:22.274235 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efca97b0-7361-43da-8764-11249a2b2ecb None admin] Authenticating user token {{(pid=80245) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:17:22.292198 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6950122-1a06-4c6b-aac5-93bd12449b9c None admin] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:17:22.327927 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c126e06-deae-43b1-b58c-734e197a28e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:17:22.328596 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c126e06-deae-43b1-b58c-734e197a28e0 None admin] REQUEST_METHOD: `GET` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:17:22.328880 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c126e06-deae-43b1-b58c-734e197a28e0 None admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:17:22.329156 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c126e06-deae-43b1-b58c-734e197a28e0 None admin] PATH_INFO: `/v3/domains` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:17:22.329884 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c126e06-deae-43b1-b58c-734e197a28e0 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80244) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 08:17:22.330190 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c126e06-deae-43b1-b58c-734e197a28e0 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:17:22.331606 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c126e06-deae-43b1-b58c-734e197a28e0 None admin] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:17:22.344821 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 336/678] 10.222.0.47 () {56 vars in 1109 bytes} [Thu Jul 21 08:17:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 08:17:22.345743 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c126e06-deae-43b1-b58c-734e197a28e0 None admin] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:17:22.390742 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efca97b0-7361-43da-8764-11249a2b2ecb None admin] Validating token access rules against request {{(pid=80245) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:17:22.415384 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG dbcounter [-] [80244] Writing DB stats keystone:SELECT=100 {{(pid=80244) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:17:22.430461 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c273b71-3f43-46f7-9821-d36614f6bd1f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80245) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:17:22.431691 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c273b71-3f43-46f7-9821-d36614f6bd1f None admin] REQUEST_METHOD: `GET` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:17:22.432325 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c273b71-3f43-46f7-9821-d36614f6bd1f None admin] SCRIPT_NAME: `/identity` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:17:22.432666 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c273b71-3f43-46f7-9821-d36614f6bd1f None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:17:22.443367 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c273b71-3f43-46f7-9821-d36614f6bd1f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 21 08:17:22.449800 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c126e06-deae-43b1-b58c-734e197a28e0 None admin] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:17:22.451567 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application [None req-3c273b71-3f43-46f7-9821-d36614f6bd1f None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 21 08:17:22.451567 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 08:17:22.451567 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 08:17:22.451567 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 08:17:22.451567 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 08:17:22.451567 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 08:17:22.451567 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 08:17:22.451567 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 08:17:22.451567 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 08:17:22.451567 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 08:17:22.451567 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 08:17:22.451567 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 08:17:22.451567 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 21 08:17:22.451567 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 21 08:17:22.451567 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 21 08:17:22.451567 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 21 08:17:22.452641 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 21 08:17:22.452641 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 21 08:17:22.452641 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 21 08:17:22.452641 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 21 08:17:22.452641 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 08:17:22.452641 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 08:17:22.452641 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 08:17:22.452641 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 08:17:22.452641 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 08:17:22.452641 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 08:17:22.452641 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 08:17:22.452641 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application with Lock( Jul 21 08:17:22.452641 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 08:17:22.452641 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application return self._enter() Jul 21 08:17:22.452641 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 08:17:22.452641 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 08:17:22.453617 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 08:17:22.453617 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application return self.creator() Jul 21 08:17:22.453617 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 08:17:22.453617 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application created_value = creator( Jul 21 08:17:22.453617 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 21 08:17:22.453617 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 21 08:17:22.453617 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 21 08:17:22.453617 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 21 08:17:22.453617 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 21 08:17:22.453617 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 21 08:17:22.453617 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 21 08:17:22.453617 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application Jul 21 08:17:22.453617 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: [pid: 80245|app: 0|req: 343/679] 10.222.0.47 () {56 vars in 1096 bytes} [Thu Jul 21 08:17:22 2022] GET /identity/v3/roles/admin => generated 83 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 08:17:22.453617 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c273b71-3f43-46f7-9821-d36614f6bd1f None admin] Authenticating user token {{(pid=80245) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:17:22.496788 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02dc3253-d1b4-4686-a3e1-603a043475ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:17:22.497561 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02dc3253-d1b4-4686-a3e1-603a043475ba None admin] REQUEST_METHOD: `GET` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:17:22.497853 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02dc3253-d1b4-4686-a3e1-603a043475ba None admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:17:22.498278 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02dc3253-d1b4-4686-a3e1-603a043475ba None admin] PATH_INFO: `/v3/domains` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:17:22.499574 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02dc3253-d1b4-4686-a3e1-603a043475ba None admin] RBAC: Adding query filter params (name=Default) {{(pid=80244) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 08:17:22.499574 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02dc3253-d1b4-4686-a3e1-603a043475ba None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:17:22.500732 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02dc3253-d1b4-4686-a3e1-603a043475ba None admin] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:17:22.513173 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 337/680] 10.222.0.47 () {56 vars in 1109 bytes} [Thu Jul 21 08:17:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 08:17:22.518622 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02dc3253-d1b4-4686-a3e1-603a043475ba None admin] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:17:22.526240 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG dbcounter [-] [80245] Writing DB stats keystone:SELECT=100 {{(pid=80245) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:17:22.573746 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c273b71-3f43-46f7-9821-d36614f6bd1f None admin] Validating token access rules against request {{(pid=80245) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:17:22.609380 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bd4dbcf-5326-4068-b417-ab430d0e758f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80245) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:17:22.610148 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bd4dbcf-5326-4068-b417-ab430d0e758f None admin] REQUEST_METHOD: `GET` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:17:22.610358 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bd4dbcf-5326-4068-b417-ab430d0e758f None admin] SCRIPT_NAME: `/identity` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:17:22.610602 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bd4dbcf-5326-4068-b417-ab430d0e758f None admin] PATH_INFO: `/v3/projects` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:17:22.611419 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bd4dbcf-5326-4068-b417-ab430d0e758f None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80245) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 08:17:22.611731 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bd4dbcf-5326-4068-b417-ab430d0e758f None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:17:22.613161 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bd4dbcf-5326-4068-b417-ab430d0e758f None admin] RBAC: Authorization granted {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:17:22.623284 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02dc3253-d1b4-4686-a3e1-603a043475ba None admin] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:17:22.625756 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: [pid: 80245|app: 0|req: 344/681] 10.222.0.47 () {56 vars in 1148 bytes} [Thu Jul 21 08:17:22 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 08:17:22.626777 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bd4dbcf-5326-4068-b417-ab430d0e758f None admin] Authenticating user token {{(pid=80245) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:17:22.659119 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG dbcounter [-] [80244] Writing DB stats keystone:SELECT=100 {{(pid=80244) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:17:22.670235 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77dadbf1-1d93-4ee8-8572-de88c13f28a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:17:22.671046 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77dadbf1-1d93-4ee8-8572-de88c13f28a6 None admin] REQUEST_METHOD: `GET` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:17:22.671351 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77dadbf1-1d93-4ee8-8572-de88c13f28a6 None admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:17:22.671627 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77dadbf1-1d93-4ee8-8572-de88c13f28a6 None admin] PATH_INFO: `/v3/users/swift` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:17:22.680434 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77dadbf1-1d93-4ee8-8572-de88c13f28a6 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:17:22.681922 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77dadbf1-1d93-4ee8-8572-de88c13f28a6 None admin] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:17:22.696522 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application [None req-77dadbf1-1d93-4ee8-8572-de88c13f28a6 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Jul 21 08:17:22.696522 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 08:17:22.696522 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 08:17:22.696522 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 08:17:22.696522 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 08:17:22.696522 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 08:17:22.696522 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 08:17:22.696522 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 08:17:22.696522 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 08:17:22.696522 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 08:17:22.696522 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 08:17:22.696522 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 08:17:22.696522 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 21 08:17:22.696522 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 21 08:17:22.696522 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 21 08:17:22.696522 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 21 08:17:22.697635 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 08:17:22.697635 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 08:17:22.697635 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 21 08:17:22.697635 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 21 08:17:22.697635 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 21 08:17:22.697635 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 21 08:17:22.697635 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 08:17:22.697635 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 08:17:22.697635 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 08:17:22.697635 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 08:17:22.697635 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 08:17:22.697635 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application with Lock( Jul 21 08:17:22.697635 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 08:17:22.697635 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self._enter() Jul 21 08:17:22.697635 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 08:17:22.697635 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 08:17:22.697635 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 08:17:22.697635 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self.creator() Jul 21 08:17:22.698822 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 08:17:22.698822 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application created_value = creator( Jul 21 08:17:22.698822 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 21 08:17:22.698822 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 21 08:17:22.698822 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 21 08:17:22.698822 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 21 08:17:22.698822 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 21 08:17:22.698822 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 21 08:17:22.698822 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Jul 21 08:17:22.698822 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application Jul 21 08:17:22.702532 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 338/682] 10.222.0.47 () {56 vars in 1096 bytes} [Thu Jul 21 08:17:22 2022] GET /identity/v3/users/swift => generated 83 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 08:17:22.703425 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77dadbf1-1d93-4ee8-8572-de88c13f28a6 None admin] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:17:22.737776 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bd4dbcf-5326-4068-b417-ab430d0e758f None admin] Validating token access rules against request {{(pid=80245) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:17:22.758037 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG dbcounter [-] [80245] Writing DB stats keystone:SELECT=100 {{(pid=80245) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:17:22.777389 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a3b4e36-6907-49f2-a4d1-d4e6dc104084 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80245) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:17:22.778076 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a3b4e36-6907-49f2-a4d1-d4e6dc104084 None admin] REQUEST_METHOD: `GET` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:17:22.778362 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a3b4e36-6907-49f2-a4d1-d4e6dc104084 None admin] SCRIPT_NAME: `/identity` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:17:22.778641 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a3b4e36-6907-49f2-a4d1-d4e6dc104084 None admin] PATH_INFO: `/v3/roles` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:17:22.779423 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a3b4e36-6907-49f2-a4d1-d4e6dc104084 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80245) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 08:17:22.779735 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a3b4e36-6907-49f2-a4d1-d4e6dc104084 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:17:22.781121 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a3b4e36-6907-49f2-a4d1-d4e6dc104084 None admin] RBAC: Authorization granted {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:17:22.789637 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: [pid: 80245|app: 0|req: 345/683] 10.222.0.47 () {56 vars in 1099 bytes} [Thu Jul 21 08:17:22 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 08:17:22.790441 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a3b4e36-6907-49f2-a4d1-d4e6dc104084 None admin] Authenticating user token {{(pid=80245) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:17:22.822445 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77dadbf1-1d93-4ee8-8572-de88c13f28a6 None admin] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:17:22.859365 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88011408-2bc6-48b3-94b6-496226caa5bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:17:22.860225 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88011408-2bc6-48b3-94b6-496226caa5bc None admin] REQUEST_METHOD: `GET` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:17:22.860443 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88011408-2bc6-48b3-94b6-496226caa5bc None admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:17:22.860721 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88011408-2bc6-48b3-94b6-496226caa5bc None admin] PATH_INFO: `/v3/projects/service` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:17:22.868565 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88011408-2bc6-48b3-94b6-496226caa5bc None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:17:22.870161 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88011408-2bc6-48b3-94b6-496226caa5bc None admin] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:17:22.885965 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application [None req-88011408-2bc6-48b3-94b6-496226caa5bc None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 21 08:17:22.885965 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 08:17:22.885965 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 08:17:22.885965 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 08:17:22.885965 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 08:17:22.885965 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 08:17:22.885965 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 08:17:22.885965 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 08:17:22.885965 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 08:17:22.885965 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 08:17:22.885965 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 08:17:22.885965 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 08:17:22.885965 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 21 08:17:22.885965 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 21 08:17:22.885965 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 21 08:17:22.886836 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 21 08:17:22.886836 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 08:17:22.886836 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 08:17:22.886836 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 08:17:22.886836 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 08:17:22.886836 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 08:17:22.886836 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 08:17:22.886836 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 08:17:22.886836 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application with Lock( Jul 21 08:17:22.886836 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 08:17:22.886836 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self._enter() Jul 21 08:17:22.886836 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 08:17:22.886836 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 08:17:22.886836 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 08:17:22.886836 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self.creator() Jul 21 08:17:22.886836 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 08:17:22.886836 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application created_value = creator( Jul 21 08:17:22.888133 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 21 08:17:22.888133 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 21 08:17:22.888133 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 21 08:17:22.888133 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 21 08:17:22.888133 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 21 08:17:22.888133 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 21 08:17:22.888133 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 21 08:17:22.888133 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application Jul 21 08:17:22.888133 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 339/684] 10.222.0.47 () {56 vars in 1111 bytes} [Thu Jul 21 08:17:22 2022] GET /identity/v3/projects/service => generated 88 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 08:17:22.888609 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88011408-2bc6-48b3-94b6-496226caa5bc None admin] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:17:22.899669 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a3b4e36-6907-49f2-a4d1-d4e6dc104084 None admin] Validating token access rules against request {{(pid=80245) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:17:22.933543 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-787d6eab-2456-417b-a46f-393760f35548 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80245) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:17:22.934206 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-787d6eab-2456-417b-a46f-393760f35548 None admin] REQUEST_METHOD: `GET` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:17:22.934489 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-787d6eab-2456-417b-a46f-393760f35548 None admin] SCRIPT_NAME: `/identity` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:17:22.934763 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-787d6eab-2456-417b-a46f-393760f35548 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:17:22.935608 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-787d6eab-2456-417b-a46f-393760f35548 None admin] RBAC: Adding query filter params (role.id=50d1ec7f947f42acb65c57e07f2269d6, scope.project.id=4faef13aa50e4e36b6c265fb1398fa14, user.id=b4f40410b7674687ab5362ac982248ef) {{(pid=80245) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 08:17:22.935917 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-787d6eab-2456-417b-a46f-393760f35548 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:17:22.937334 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-787d6eab-2456-417b-a46f-393760f35548 None admin] RBAC: Authorization granted {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:17:22.938768 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG dbcounter [-] [80244] Writing DB stats keystone:SELECT=100 {{(pid=80244) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:17:22.955358 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: [pid: 80245|app: 0|req: 346/685] 10.222.0.47 () {56 vars in 1374 bytes} [Thu Jul 21 08:17:22 2022] GET /identity/v3/role_assignments?user.id=b4f40410b7674687ab5362ac982248ef&scope.project.id=4faef13aa50e4e36b6c265fb1398fa14&role.id=50d1ec7f947f42acb65c57e07f2269d6 => generated 608 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 08:17:22.955358 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-787d6eab-2456-417b-a46f-393760f35548 None admin] Authenticating user token {{(pid=80245) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:17:23.001923 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88011408-2bc6-48b3-94b6-496226caa5bc None admin] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:17:23.010410 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG dbcounter [-] [80245] Writing DB stats keystone:SELECT=100 {{(pid=80245) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:17:23.043749 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26e6036e-7666-4525-bae1-1718bd50a6ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:17:23.044444 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26e6036e-7666-4525-bae1-1718bd50a6ad None admin] REQUEST_METHOD: `GET` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:17:23.044730 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26e6036e-7666-4525-bae1-1718bd50a6ad None admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:17:23.045006 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26e6036e-7666-4525-bae1-1718bd50a6ad None admin] PATH_INFO: `/v3/users` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:17:23.045986 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26e6036e-7666-4525-bae1-1718bd50a6ad None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80244) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 08:17:23.046288 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26e6036e-7666-4525-bae1-1718bd50a6ad None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:17:23.047810 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26e6036e-7666-4525-bae1-1718bd50a6ad None admin] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:17:23.074006 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 340/686] 10.222.0.47 () {56 vars in 1135 bytes} [Thu Jul 21 08:17:22 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 08:17:23.074899 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26e6036e-7666-4525-bae1-1718bd50a6ad None admin] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:17:23.082745 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-787d6eab-2456-417b-a46f-393760f35548 None admin] Validating token access rules against request {{(pid=80245) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:17:23.120330 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23a35ca6-17d0-4796-8036-650cc21103b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80245) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:17:23.121170 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23a35ca6-17d0-4796-8036-650cc21103b1 None admin] REQUEST_METHOD: `GET` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:17:23.121453 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23a35ca6-17d0-4796-8036-650cc21103b1 None admin] SCRIPT_NAME: `/identity` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:17:23.121719 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23a35ca6-17d0-4796-8036-650cc21103b1 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:17:23.134002 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23a35ca6-17d0-4796-8036-650cc21103b1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:17:23.135491 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23a35ca6-17d0-4796-8036-650cc21103b1 None admin] RBAC: Authorization granted {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:17:23.153546 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application [None req-23a35ca6-17d0-4796-8036-650cc21103b1 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 08:17:23.153546 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 08:17:23.153546 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 08:17:23.153546 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 08:17:23.153546 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 08:17:23.153546 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 08:17:23.153546 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 08:17:23.153546 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 08:17:23.153546 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 08:17:23.153546 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 08:17:23.153546 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 08:17:23.153546 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 08:17:23.153546 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 21 08:17:23.153546 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 21 08:17:23.153546 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 21 08:17:23.154474 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 21 08:17:23.154474 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 08:17:23.154474 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 08:17:23.154474 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 08:17:23.154474 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 08:17:23.154474 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 08:17:23.154474 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 08:17:23.154474 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 08:17:23.154474 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application with Lock( Jul 21 08:17:23.154474 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 08:17:23.154474 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application return self._enter() Jul 21 08:17:23.154474 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 08:17:23.154474 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 08:17:23.154474 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 08:17:23.154474 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application return self.creator() Jul 21 08:17:23.154474 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 08:17:23.154474 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application created_value = creator( Jul 21 08:17:23.155913 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 21 08:17:23.155913 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 21 08:17:23.155913 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 08:17:23.155913 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application Jul 21 08:17:23.155913 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: [pid: 80245|app: 0|req: 347/687] 10.222.0.47 () {56 vars in 1108 bytes} [Thu Jul 21 08:17:22 2022] GET /identity/v3/domains/Default => generated 87 bytes in 209 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 08:17:23.155913 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23a35ca6-17d0-4796-8036-650cc21103b1 None admin] Authenticating user token {{(pid=80245) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:17:23.209725 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26e6036e-7666-4525-bae1-1718bd50a6ad None admin] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:17:23.222552 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG dbcounter [-] [80244] Writing DB stats keystone:SELECT=100 {{(pid=80244) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:17:23.266443 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fa12eb8-9118-4c49-a024-98ef68a1326b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:17:23.267214 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fa12eb8-9118-4c49-a024-98ef68a1326b None admin] REQUEST_METHOD: `GET` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:17:23.268273 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fa12eb8-9118-4c49-a024-98ef68a1326b None admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:17:23.268563 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fa12eb8-9118-4c49-a024-98ef68a1326b None admin] PATH_INFO: `/v3/projects` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:17:23.269365 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fa12eb8-9118-4c49-a024-98ef68a1326b None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80244) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 08:17:23.270291 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fa12eb8-9118-4c49-a024-98ef68a1326b None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:17:23.272385 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fa12eb8-9118-4c49-a024-98ef68a1326b None admin] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:17:23.276552 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG dbcounter [-] [80245] Writing DB stats keystone:SELECT=100 {{(pid=80245) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:17:23.283177 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23a35ca6-17d0-4796-8036-650cc21103b1 None admin] Validating token access rules against request {{(pid=80245) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:17:23.295498 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 341/688] 10.222.0.47 () {56 vars in 1148 bytes} [Thu Jul 21 08:17:23 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 08:17:23.296393 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fa12eb8-9118-4c49-a024-98ef68a1326b None admin] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:17:23.323346 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-605e8aa3-98bb-4b42-92ae-b4cfb7445177 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80245) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:17:23.324017 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-605e8aa3-98bb-4b42-92ae-b4cfb7445177 None admin] REQUEST_METHOD: `GET` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:17:23.324295 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-605e8aa3-98bb-4b42-92ae-b4cfb7445177 None admin] SCRIPT_NAME: `/identity` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:17:23.324566 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-605e8aa3-98bb-4b42-92ae-b4cfb7445177 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:17:23.398920 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-605e8aa3-98bb-4b42-92ae-b4cfb7445177 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:17:23.400532 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-605e8aa3-98bb-4b42-92ae-b4cfb7445177 None admin] RBAC: Authorization granted {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:17:23.438464 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fa12eb8-9118-4c49-a024-98ef68a1326b None admin] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:17:23.484192 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2991a5fe-1df4-421a-85dc-9e7d304b2645 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:17:23.485058 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2991a5fe-1df4-421a-85dc-9e7d304b2645 None admin] REQUEST_METHOD: `GET` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:17:23.485343 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2991a5fe-1df4-421a-85dc-9e7d304b2645 None admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:17:23.485614 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2991a5fe-1df4-421a-85dc-9e7d304b2645 None admin] PATH_INFO: `/v3/domains` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:17:23.486349 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2991a5fe-1df4-421a-85dc-9e7d304b2645 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80244) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 08:17:23.486653 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2991a5fe-1df4-421a-85dc-9e7d304b2645 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:17:23.488063 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2991a5fe-1df4-421a-85dc-9e7d304b2645 None admin] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:17:23.505905 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 342/689] 10.222.0.47 () {56 vars in 1109 bytes} [Thu Jul 21 08:17:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 08:17:23.505905 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2991a5fe-1df4-421a-85dc-9e7d304b2645 None admin] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:17:23.526699 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG dbcounter [-] [80245] Writing DB stats keystone:SELECT=100 {{(pid=80245) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:17:23.534931 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: [pid: 80245|app: 0|req: 348/690] 10.222.0.47 () {58 vars in 1282 bytes} [Thu Jul 21 08:17:23 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 379 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 21 08:17:23.535960 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-605e8aa3-98bb-4b42-92ae-b4cfb7445177 None admin] Authenticating user token {{(pid=80245) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:17:23.543293 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG dbcounter [-] [80244] Writing DB stats keystone:SELECT=100 {{(pid=80244) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:17:23.624479 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2991a5fe-1df4-421a-85dc-9e7d304b2645 None admin] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:17:23.646363 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-605e8aa3-98bb-4b42-92ae-b4cfb7445177 None admin] Validating token access rules against request {{(pid=80245) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:17:23.660641 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20c0625b-fa27-45b3-9eaf-2a076378acb4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:17:23.661292 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20c0625b-fa27-45b3-9eaf-2a076378acb4 None admin] REQUEST_METHOD: `GET` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:17:23.661580 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20c0625b-fa27-45b3-9eaf-2a076378acb4 None admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:17:23.661853 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20c0625b-fa27-45b3-9eaf-2a076378acb4 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:17:23.662682 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20c0625b-fa27-45b3-9eaf-2a076378acb4 None admin] RBAC: Adding query filter params (role.id=50d1ec7f947f42acb65c57e07f2269d6, scope.project.id=4faef13aa50e4e36b6c265fb1398fa14, user.id=20ca3c3bd3774691b3374cfbfec6e494) {{(pid=80244) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 08:17:23.662993 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20c0625b-fa27-45b3-9eaf-2a076378acb4 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:17:23.664470 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20c0625b-fa27-45b3-9eaf-2a076378acb4 None admin] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:17:23.670934 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 343/691] 10.222.0.47 () {56 vars in 1374 bytes} [Thu Jul 21 08:17:23 2022] GET /identity/v3/role_assignments?user.id=20ca3c3bd3774691b3374cfbfec6e494&scope.project.id=4faef13aa50e4e36b6c265fb1398fa14&role.id=50d1ec7f947f42acb65c57e07f2269d6 => generated 608 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 08:17:23.671767 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20c0625b-fa27-45b3-9eaf-2a076378acb4 None admin] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:17:23.693016 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcb10bff-ea15-4d49-aa6d-e736998a957b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80245) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:17:23.694166 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcb10bff-ea15-4d49-aa6d-e736998a957b None admin] REQUEST_METHOD: `GET` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:17:23.694672 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcb10bff-ea15-4d49-aa6d-e736998a957b None admin] SCRIPT_NAME: `/identity` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:17:23.695143 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcb10bff-ea15-4d49-aa6d-e736998a957b None admin] PATH_INFO: `/v3/users/nova` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:17:23.704759 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcb10bff-ea15-4d49-aa6d-e736998a957b None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:17:23.706698 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcb10bff-ea15-4d49-aa6d-e736998a957b None admin] RBAC: Authorization granted {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:17:23.725697 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application [None req-fcb10bff-ea15-4d49-aa6d-e736998a957b None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jul 21 08:17:23.725697 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 08:17:23.725697 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 08:17:23.725697 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 08:17:23.725697 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 08:17:23.725697 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 08:17:23.725697 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 08:17:23.725697 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 08:17:23.725697 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 08:17:23.725697 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 08:17:23.725697 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 08:17:23.725697 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 08:17:23.725697 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 21 08:17:23.725697 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 21 08:17:23.725697 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 21 08:17:23.725697 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 21 08:17:23.726803 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 08:17:23.726803 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 08:17:23.726803 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 21 08:17:23.726803 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 21 08:17:23.726803 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 21 08:17:23.726803 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 21 08:17:23.726803 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 08:17:23.726803 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 08:17:23.726803 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 08:17:23.726803 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 08:17:23.726803 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 08:17:23.726803 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application with Lock( Jul 21 08:17:23.726803 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 08:17:23.726803 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application return self._enter() Jul 21 08:17:23.726803 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 08:17:23.726803 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 08:17:23.726803 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 08:17:23.726803 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application return self.creator() Jul 21 08:17:23.727814 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 08:17:23.727814 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application created_value = creator( Jul 21 08:17:23.727814 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 21 08:17:23.727814 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 21 08:17:23.727814 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 21 08:17:23.727814 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 21 08:17:23.727814 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 21 08:17:23.727814 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 21 08:17:23.727814 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jul 21 08:17:23.727814 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application Jul 21 08:17:23.730200 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: [pid: 80245|app: 0|req: 349/692] 10.222.0.47 () {56 vars in 1093 bytes} [Thu Jul 21 08:17:23 2022] GET /identity/v3/users/nova => generated 82 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 08:17:23.731773 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6262a819-f015-4acc-b905-3411afc47499 None None] REQUEST_METHOD: `GET` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:17:23.732230 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6262a819-f015-4acc-b905-3411afc47499 None None] SCRIPT_NAME: `/identity` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:17:23.732659 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6262a819-f015-4acc-b905-3411afc47499 None None] PATH_INFO: `/` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:17:23.738192 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: [pid: 80245|app: 0|req: 350/693] 10.222.0.47 () {54 vars in 926 bytes} [Thu Jul 21 08:17:23 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 08:17:23.738192 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6262a819-f015-4acc-b905-3411afc47499 None None] Authenticating user token {{(pid=80245) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:17:23.795363 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG dbcounter [-] [80244] Writing DB stats keystone:SELECT=100 {{(pid=80244) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:17:23.802046 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG dbcounter [-] [80245] Writing DB stats keystone:SELECT=100 {{(pid=80245) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:17:23.813367 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20c0625b-fa27-45b3-9eaf-2a076378acb4 None admin] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:17:23.850485 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6262a819-f015-4acc-b905-3411afc47499 None None] Validating token access rules against request {{(pid=80245) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:17:23.856481 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d44d39db-db30-45c9-86c1-f7d8e75158b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:17:23.857831 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d44d39db-db30-45c9-86c1-f7d8e75158b2 None admin] REQUEST_METHOD: `GET` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:17:23.858357 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d44d39db-db30-45c9-86c1-f7d8e75158b2 None admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:17:23.858828 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d44d39db-db30-45c9-86c1-f7d8e75158b2 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:17:23.870424 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d44d39db-db30-45c9-86c1-f7d8e75158b2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:17:23.872180 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d44d39db-db30-45c9-86c1-f7d8e75158b2 None admin] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:17:23.889532 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2001fbf0-5a5f-4688-8c5c-62abd3fdf72f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80245) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:17:23.890214 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2001fbf0-5a5f-4688-8c5c-62abd3fdf72f None admin] REQUEST_METHOD: `GET` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:17:23.890503 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2001fbf0-5a5f-4688-8c5c-62abd3fdf72f None admin] SCRIPT_NAME: `/identity` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:17:23.890778 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2001fbf0-5a5f-4688-8c5c-62abd3fdf72f None admin] PATH_INFO: `/v3/users` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:17:23.891661 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2001fbf0-5a5f-4688-8c5c-62abd3fdf72f None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80245) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 08:17:23.891970 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2001fbf0-5a5f-4688-8c5c-62abd3fdf72f None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:17:23.893427 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2001fbf0-5a5f-4688-8c5c-62abd3fdf72f None admin] RBAC: Authorization granted {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:17:23.894041 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application [None req-d44d39db-db30-45c9-86c1-f7d8e75158b2 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 08:17:23.894041 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 08:17:23.894041 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 08:17:23.894041 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 08:17:23.894041 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 08:17:23.894041 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 08:17:23.894041 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 08:17:23.894041 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 08:17:23.894041 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 08:17:23.894041 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 08:17:23.894041 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 08:17:23.894041 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 08:17:23.894041 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 21 08:17:23.894041 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 21 08:17:23.894041 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 21 08:17:23.895483 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 21 08:17:23.895483 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 08:17:23.895483 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 08:17:23.895483 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 08:17:23.895483 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 08:17:23.895483 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 08:17:23.895483 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 08:17:23.895483 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 08:17:23.895483 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application with Lock( Jul 21 08:17:23.895483 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 08:17:23.895483 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self._enter() Jul 21 08:17:23.895483 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 08:17:23.895483 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 08:17:23.895483 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 08:17:23.895483 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self.creator() Jul 21 08:17:23.895483 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 08:17:23.895483 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application created_value = creator( Jul 21 08:17:23.896765 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 21 08:17:23.896765 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 21 08:17:23.896765 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 08:17:23.896765 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application Jul 21 08:17:23.896765 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 344/694] 10.222.0.47 () {56 vars in 1108 bytes} [Thu Jul 21 08:17:23 2022] GET /identity/v3/domains/Default => generated 87 bytes in 225 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 08:17:23.900478 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b5b9b62-72e0-406e-abc7-fd2fc0dbe2c4 None None] REQUEST_METHOD: `POST` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:17:23.900478 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b5b9b62-72e0-406e-abc7-fd2fc0dbe2c4 None None] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:17:23.900478 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b5b9b62-72e0-406e-abc7-fd2fc0dbe2c4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:17:23.915217 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: [pid: 80245|app: 0|req: 351/695] 10.222.0.47 () {56 vars in 1133 bytes} [Thu Jul 21 08:17:23 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 08:17:23.916503 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2001fbf0-5a5f-4688-8c5c-62abd3fdf72f None admin] Authenticating user token {{(pid=80245) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:17:23.919232 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.auth.core [None req-2b5b9b62-72e0-406e-abc7-fd2fc0dbe2c4 None None] MFA Rules not processed for user `2dce40b8d74c41ec94cc36e6ff05d412`. Rule list: `[]` (Enabled: `True`). {{(pid=80244) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 08:17:23.921600 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.fernet_utils [None req-2b5b9b62-72e0-406e-abc7-fd2fc0dbe2c4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80244) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 08:17:24.015692 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 345/696] 10.222.0.47 () {58 vars in 1023 bytes} [Thu Jul 21 08:17:23 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 21 08:17:24.016906 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b5b9b62-72e0-406e-abc7-fd2fc0dbe2c4 None None] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:17:24.031823 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2001fbf0-5a5f-4688-8c5c-62abd3fdf72f None admin] Validating token access rules against request {{(pid=80245) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:17:24.047856 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG dbcounter [-] [80245] Writing DB stats keystone:SELECT=100 {{(pid=80245) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:17:24.070077 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb45af69-d157-4f23-886c-6f6fdbac7e25 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80245) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:17:24.070753 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb45af69-d157-4f23-886c-6f6fdbac7e25 None admin] REQUEST_METHOD: `GET` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:17:24.071031 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb45af69-d157-4f23-886c-6f6fdbac7e25 None admin] SCRIPT_NAME: `/identity` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:17:24.071303 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb45af69-d157-4f23-886c-6f6fdbac7e25 None admin] PATH_INFO: `/v3/domains` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:17:24.072151 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb45af69-d157-4f23-886c-6f6fdbac7e25 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80245) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 08:17:24.072455 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb45af69-d157-4f23-886c-6f6fdbac7e25 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:17:24.073812 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb45af69-d157-4f23-886c-6f6fdbac7e25 None admin] RBAC: Authorization granted {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:17:24.087044 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: [pid: 80245|app: 0|req: 352/697] 10.222.0.47 () {56 vars in 1109 bytes} [Thu Jul 21 08:17:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 08:17:24.088015 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb45af69-d157-4f23-886c-6f6fdbac7e25 None admin] Authenticating user token {{(pid=80245) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:17:24.114286 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG dbcounter [-] [80244] Writing DB stats keystone:SELECT=100 {{(pid=80244) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:17:24.147911 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b5b9b62-72e0-406e-abc7-fd2fc0dbe2c4 None None] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:17:24.184758 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99e16bdc-7b24-458d-a9e1-9f95f7d10317 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:17:24.185467 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99e16bdc-7b24-458d-a9e1-9f95f7d10317 None admin] REQUEST_METHOD: `GET` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:17:24.185757 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99e16bdc-7b24-458d-a9e1-9f95f7d10317 None admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:17:24.186029 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99e16bdc-7b24-458d-a9e1-9f95f7d10317 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:17:24.221105 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb45af69-d157-4f23-886c-6f6fdbac7e25 None admin] Validating token access rules against request {{(pid=80245) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:17:24.264666 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99e16bdc-7b24-458d-a9e1-9f95f7d10317 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:17:24.266334 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99e16bdc-7b24-458d-a9e1-9f95f7d10317 None admin] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:17:24.267126 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0d67153-c14e-47b6-960b-8ac2102f9969 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80245) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:17:24.268346 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0d67153-c14e-47b6-960b-8ac2102f9969 None admin] REQUEST_METHOD: `GET` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:17:24.268829 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0d67153-c14e-47b6-960b-8ac2102f9969 None admin] SCRIPT_NAME: `/identity` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:17:24.269382 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0d67153-c14e-47b6-960b-8ac2102f9969 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:17:24.284257 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0d67153-c14e-47b6-960b-8ac2102f9969 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:17:24.286312 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0d67153-c14e-47b6-960b-8ac2102f9969 None admin] RBAC: Authorization granted {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:17:24.305163 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application [None req-e0d67153-c14e-47b6-960b-8ac2102f9969 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 08:17:24.305163 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 08:17:24.305163 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 08:17:24.305163 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 08:17:24.305163 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 08:17:24.305163 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 08:17:24.305163 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 08:17:24.305163 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 08:17:24.305163 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 08:17:24.305163 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 08:17:24.305163 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 08:17:24.305163 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 08:17:24.305163 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 21 08:17:24.305163 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 21 08:17:24.305163 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 21 08:17:24.306044 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 21 08:17:24.306044 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 08:17:24.306044 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 08:17:24.306044 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 08:17:24.306044 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 08:17:24.306044 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 08:17:24.306044 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 08:17:24.306044 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 08:17:24.306044 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application with Lock( Jul 21 08:17:24.306044 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 08:17:24.306044 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application return self._enter() Jul 21 08:17:24.306044 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 08:17:24.306044 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 08:17:24.306044 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 08:17:24.306044 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application return self.creator() Jul 21 08:17:24.306044 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 08:17:24.306044 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application created_value = creator( Jul 21 08:17:24.306916 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 21 08:17:24.306916 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 21 08:17:24.306916 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 08:17:24.306916 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application Jul 21 08:17:24.309225 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: [pid: 80245|app: 0|req: 353/698] 10.222.0.47 () {56 vars in 1108 bytes} [Thu Jul 21 08:17:24 2022] GET /identity/v3/domains/Default => generated 87 bytes in 221 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 08:17:24.310305 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0d67153-c14e-47b6-960b-8ac2102f9969 None admin] Authenticating user token {{(pid=80245) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:17:24.339977 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG dbcounter [-] [80244] Writing DB stats keystone:SELECT=100 {{(pid=80244) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:17:24.342223 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG dbcounter [-] [80245] Writing DB stats keystone:SELECT=100 {{(pid=80245) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:17:24.379548 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 346/699] 10.222.0.47 () {58 vars in 1282 bytes} [Thu Jul 21 08:17:24 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 360 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 21 08:17:24.379548 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99e16bdc-7b24-458d-a9e1-9f95f7d10317 None admin] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:17:24.425720 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0d67153-c14e-47b6-960b-8ac2102f9969 None admin] Validating token access rules against request {{(pid=80245) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:17:24.462909 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58253532-380a-417c-b800-370cf8e2f0ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80245) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:17:24.463845 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58253532-380a-417c-b800-370cf8e2f0ce None admin] REQUEST_METHOD: `GET` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:17:24.464229 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58253532-380a-417c-b800-370cf8e2f0ce None admin] SCRIPT_NAME: `/identity` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:17:24.464606 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58253532-380a-417c-b800-370cf8e2f0ce None admin] PATH_INFO: `/v3/projects/service` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:17:24.473297 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58253532-380a-417c-b800-370cf8e2f0ce None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:17:24.473824 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58253532-380a-417c-b800-370cf8e2f0ce None admin] RBAC: Authorization granted {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:17:24.493474 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application [None req-58253532-380a-417c-b800-370cf8e2f0ce None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 21 08:17:24.493474 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 08:17:24.493474 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 08:17:24.493474 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 08:17:24.493474 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 08:17:24.493474 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 08:17:24.493474 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 08:17:24.493474 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 08:17:24.493474 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 08:17:24.493474 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 08:17:24.493474 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 08:17:24.493474 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 08:17:24.493474 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 21 08:17:24.493474 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 21 08:17:24.493474 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 21 08:17:24.494323 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 21 08:17:24.494323 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 08:17:24.494323 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 08:17:24.494323 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 08:17:24.494323 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 08:17:24.494323 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 08:17:24.494323 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 08:17:24.494323 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 08:17:24.494323 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application with Lock( Jul 21 08:17:24.494323 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 08:17:24.494323 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application return self._enter() Jul 21 08:17:24.494323 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 08:17:24.494323 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 08:17:24.494323 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 08:17:24.494323 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application return self.creator() Jul 21 08:17:24.494323 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 08:17:24.494323 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application created_value = creator( Jul 21 08:17:24.495213 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 21 08:17:24.495213 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 21 08:17:24.495213 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 21 08:17:24.495213 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 21 08:17:24.495213 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 21 08:17:24.495213 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 21 08:17:24.495213 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 21 08:17:24.495213 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application Jul 21 08:17:24.495663 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99e16bdc-7b24-458d-a9e1-9f95f7d10317 None admin] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:17:24.498235 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: [pid: 80245|app: 0|req: 354/700] 10.222.0.47 () {56 vars in 1111 bytes} [Thu Jul 21 08:17:24 2022] GET /identity/v3/projects/service => generated 88 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 08:17:24.499244 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58253532-380a-417c-b800-370cf8e2f0ce None admin] Authenticating user token {{(pid=80245) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:17:24.533397 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20fa5cd1-6bce-4870-8cff-2142e63eefe0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:17:24.534100 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20fa5cd1-6bce-4870-8cff-2142e63eefe0 None admin] REQUEST_METHOD: `GET` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:17:24.534376 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20fa5cd1-6bce-4870-8cff-2142e63eefe0 None admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:17:24.534653 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20fa5cd1-6bce-4870-8cff-2142e63eefe0 None admin] PATH_INFO: `/v3/domains` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:17:24.535437 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20fa5cd1-6bce-4870-8cff-2142e63eefe0 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80244) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 08:17:24.535742 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20fa5cd1-6bce-4870-8cff-2142e63eefe0 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:17:24.537182 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20fa5cd1-6bce-4870-8cff-2142e63eefe0 None admin] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:17:24.549846 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 347/701] 10.222.0.47 () {56 vars in 1109 bytes} [Thu Jul 21 08:17:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 08:17:24.550690 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20fa5cd1-6bce-4870-8cff-2142e63eefe0 None admin] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:17:24.584913 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG dbcounter [-] [80244] Writing DB stats keystone:SELECT=100 {{(pid=80244) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:17:24.596598 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG dbcounter [-] [80245] Writing DB stats keystone:SELECT=100 {{(pid=80245) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:17:24.612948 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58253532-380a-417c-b800-370cf8e2f0ce None admin] Validating token access rules against request {{(pid=80245) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:17:24.651025 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3099ba0c-d141-48e0-8a24-510bfaa0d881 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80245) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:17:24.652796 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3099ba0c-d141-48e0-8a24-510bfaa0d881 None admin] REQUEST_METHOD: `GET` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:17:24.653281 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3099ba0c-d141-48e0-8a24-510bfaa0d881 None admin] SCRIPT_NAME: `/identity` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:17:24.653755 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3099ba0c-d141-48e0-8a24-510bfaa0d881 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:17:24.661825 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20fa5cd1-6bce-4870-8cff-2142e63eefe0 None admin] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:17:24.665586 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3099ba0c-d141-48e0-8a24-510bfaa0d881 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:17:24.667289 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3099ba0c-d141-48e0-8a24-510bfaa0d881 None admin] RBAC: Authorization granted {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:17:24.686844 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application [None req-3099ba0c-d141-48e0-8a24-510bfaa0d881 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 08:17:24.686844 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 08:17:24.686844 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 08:17:24.686844 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 08:17:24.686844 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 08:17:24.686844 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 08:17:24.686844 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 08:17:24.686844 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 08:17:24.686844 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 08:17:24.686844 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 08:17:24.686844 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 08:17:24.686844 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 08:17:24.686844 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 21 08:17:24.686844 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 21 08:17:24.686844 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 21 08:17:24.687782 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 21 08:17:24.687782 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 08:17:24.687782 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 08:17:24.687782 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 08:17:24.687782 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 08:17:24.687782 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 08:17:24.687782 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 08:17:24.687782 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 08:17:24.687782 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application with Lock( Jul 21 08:17:24.687782 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 08:17:24.687782 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application return self._enter() Jul 21 08:17:24.687782 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 08:17:24.687782 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 08:17:24.687782 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 08:17:24.687782 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application return self.creator() Jul 21 08:17:24.687782 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 08:17:24.687782 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application created_value = creator( Jul 21 08:17:24.688641 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 21 08:17:24.688641 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 21 08:17:24.688641 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 08:17:24.688641 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application Jul 21 08:17:24.695669 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: [pid: 80245|app: 0|req: 355/702] 10.222.0.47 () {56 vars in 1108 bytes} [Thu Jul 21 08:17:24 2022] GET /identity/v3/domains/Default => generated 87 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 08:17:24.696783 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3099ba0c-d141-48e0-8a24-510bfaa0d881 None admin] Authenticating user token {{(pid=80245) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:17:24.698124 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a9b4e32-0114-4903-b71b-1c2176a8d74e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:17:24.698938 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a9b4e32-0114-4903-b71b-1c2176a8d74e None admin] REQUEST_METHOD: `GET` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:17:24.699316 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a9b4e32-0114-4903-b71b-1c2176a8d74e None admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:17:24.699710 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a9b4e32-0114-4903-b71b-1c2176a8d74e None admin] PATH_INFO: `/v3/projects` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:17:24.700749 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a9b4e32-0114-4903-b71b-1c2176a8d74e None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80244) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 08:17:24.701209 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a9b4e32-0114-4903-b71b-1c2176a8d74e None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:17:24.702832 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a9b4e32-0114-4903-b71b-1c2176a8d74e None admin] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:17:24.716498 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 348/703] 10.222.0.47 () {56 vars in 1148 bytes} [Thu Jul 21 08:17:24 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 08:17:24.717554 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a9b4e32-0114-4903-b71b-1c2176a8d74e None admin] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:17:24.806385 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3099ba0c-d141-48e0-8a24-510bfaa0d881 None admin] Validating token access rules against request {{(pid=80245) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:17:24.815539 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG dbcounter [-] [80244] Writing DB stats keystone:SELECT=100 {{(pid=80244) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:17:24.826429 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a9b4e32-0114-4903-b71b-1c2176a8d74e None admin] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:17:24.845249 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a305393c-d5a1-432e-b10e-33c9fc77447e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80245) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:17:24.846203 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a305393c-d5a1-432e-b10e-33c9fc77447e None admin] REQUEST_METHOD: `POST` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:17:24.846646 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a305393c-d5a1-432e-b10e-33c9fc77447e None admin] SCRIPT_NAME: `/identity` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:17:24.847068 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a305393c-d5a1-432e-b10e-33c9fc77447e None admin] PATH_INFO: `/v3/users` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:17:24.848480 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a305393c-d5a1-432e-b10e-33c9fc77447e None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:17:24.850192 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a305393c-d5a1-432e-b10e-33c9fc77447e None admin] RBAC: Authorization granted {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:17:24.859939 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG dbcounter [-] [80245] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80245) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:17:24.866361 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5f3fed3-9833-4db8-b511-9bc2c4f4e9d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:17:24.867027 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5f3fed3-9833-4db8-b511-9bc2c4f4e9d6 None admin] REQUEST_METHOD: `GET` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:17:24.867452 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5f3fed3-9833-4db8-b511-9bc2c4f4e9d6 None admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:17:24.867849 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5f3fed3-9833-4db8-b511-9bc2c4f4e9d6 None admin] PATH_INFO: `/v3/domains` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:17:24.869815 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5f3fed3-9833-4db8-b511-9bc2c4f4e9d6 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80244) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 08:17:24.870132 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5f3fed3-9833-4db8-b511-9bc2c4f4e9d6 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:17:24.871598 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5f3fed3-9833-4db8-b511-9bc2c4f4e9d6 None admin] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:17:24.879502 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: [pid: 80245|app: 0|req: 356/704] 10.222.0.47 () {60 vars in 1132 bytes} [Thu Jul 21 08:17:24 2022] POST /identity/v3/users => generated 254 bytes in 183 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 08:17:24.881215 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a305393c-d5a1-432e-b10e-33c9fc77447e None admin] Authenticating user token {{(pid=80245) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:17:24.888945 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 349/705] 10.222.0.47 () {56 vars in 1109 bytes} [Thu Jul 21 08:17:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 08:17:24.890012 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5f3fed3-9833-4db8-b511-9bc2c4f4e9d6 None admin] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:17:25.007520 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5f3fed3-9833-4db8-b511-9bc2c4f4e9d6 None admin] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:17:25.024150 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a305393c-d5a1-432e-b10e-33c9fc77447e None admin] Validating token access rules against request {{(pid=80245) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:17:25.044152 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cade77c5-8ebe-4a21-b69c-118cfb6cb4bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:17:25.045028 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cade77c5-8ebe-4a21-b69c-118cfb6cb4bf None admin] REQUEST_METHOD: `GET` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:17:25.045229 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cade77c5-8ebe-4a21-b69c-118cfb6cb4bf None admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:17:25.045501 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cade77c5-8ebe-4a21-b69c-118cfb6cb4bf None admin] PATH_INFO: `/v3/projects/service` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:17:25.053595 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cade77c5-8ebe-4a21-b69c-118cfb6cb4bf None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:17:25.054866 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cade77c5-8ebe-4a21-b69c-118cfb6cb4bf None admin] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:17:25.070151 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4f70b22-b73f-4041-b1be-8469e3b89bbd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80245) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:17:25.070366 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application [None req-cade77c5-8ebe-4a21-b69c-118cfb6cb4bf None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 21 08:17:25.070366 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 08:17:25.070366 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 08:17:25.070366 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 08:17:25.070366 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 08:17:25.070366 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 08:17:25.070366 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 08:17:25.070366 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 08:17:25.070366 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 08:17:25.070366 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 08:17:25.070366 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 08:17:25.070366 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 08:17:25.070366 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 21 08:17:25.070366 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 21 08:17:25.070366 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 21 08:17:25.071187 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 21 08:17:25.071187 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 08:17:25.071187 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 08:17:25.071187 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 08:17:25.071187 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 08:17:25.071187 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 08:17:25.071187 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 08:17:25.071187 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 08:17:25.071187 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application with Lock( Jul 21 08:17:25.071187 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 08:17:25.071187 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self._enter() Jul 21 08:17:25.071187 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 08:17:25.071187 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 08:17:25.071187 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 08:17:25.071187 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self.creator() Jul 21 08:17:25.071187 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 08:17:25.071187 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application created_value = creator( Jul 21 08:17:25.072074 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 21 08:17:25.072074 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 21 08:17:25.072074 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 21 08:17:25.072074 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 21 08:17:25.072074 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 21 08:17:25.072074 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 21 08:17:25.072074 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 21 08:17:25.072074 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application Jul 21 08:17:25.072074 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 350/706] 10.222.0.47 () {56 vars in 1111 bytes} [Thu Jul 21 08:17:24 2022] GET /identity/v3/projects/service => generated 88 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 08:17:25.073712 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4f70b22-b73f-4041-b1be-8469e3b89bbd None admin] REQUEST_METHOD: `GET` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:17:25.074191 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4f70b22-b73f-4041-b1be-8469e3b89bbd None admin] SCRIPT_NAME: `/identity` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:17:25.074665 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4f70b22-b73f-4041-b1be-8469e3b89bbd None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:17:25.075708 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4f70b22-b73f-4041-b1be-8469e3b89bbd None admin] RBAC: Adding query filter params (role.id=50d1ec7f947f42acb65c57e07f2269d6, scope.project.id=4faef13aa50e4e36b6c265fb1398fa14, user.id=302351411fc84248a6b166b25d21609f) {{(pid=80245) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 08:17:25.076225 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4f70b22-b73f-4041-b1be-8469e3b89bbd None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:17:25.077297 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cade77c5-8ebe-4a21-b69c-118cfb6cb4bf None admin] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:17:25.078196 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4f70b22-b73f-4041-b1be-8469e3b89bbd None admin] RBAC: Authorization granted {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:17:25.086523 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: [pid: 80245|app: 0|req: 357/707] 10.222.0.47 () {56 vars in 1374 bytes} [Thu Jul 21 08:17:24 2022] GET /identity/v3/role_assignments?user.id=302351411fc84248a6b166b25d21609f&scope.project.id=4faef13aa50e4e36b6c265fb1398fa14&role.id=50d1ec7f947f42acb65c57e07f2269d6 => generated 608 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 08:17:25.091189 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG dbcounter [-] [80244] Writing DB stats keystone:SELECT=100 {{(pid=80244) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:17:25.205776 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cade77c5-8ebe-4a21-b69c-118cfb6cb4bf None admin] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:17:25.251497 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91341dde-eef6-482a-852d-4028ee788163 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:17:25.252188 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91341dde-eef6-482a-852d-4028ee788163 None admin] REQUEST_METHOD: `GET` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:17:25.252474 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91341dde-eef6-482a-852d-4028ee788163 None admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:17:25.252754 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91341dde-eef6-482a-852d-4028ee788163 None admin] PATH_INFO: `/v3/projects` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:17:25.253567 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91341dde-eef6-482a-852d-4028ee788163 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80244) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 08:17:25.253912 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91341dde-eef6-482a-852d-4028ee788163 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:17:25.255963 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91341dde-eef6-482a-852d-4028ee788163 None admin] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:17:25.270212 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 351/708] 10.222.0.47 () {56 vars in 1148 bytes} [Thu Jul 21 08:17:25 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 21 08:17:25.276333 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4f70b22-b73f-4041-b1be-8469e3b89bbd None admin] Authenticating user token {{(pid=80245) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:17:25.317229 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48cef7e1-2303-4642-91c4-56af29e59c28 None None] REQUEST_METHOD: `GET` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:17:25.317545 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48cef7e1-2303-4642-91c4-56af29e59c28 None None] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:17:25.317786 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48cef7e1-2303-4642-91c4-56af29e59c28 None None] PATH_INFO: `/` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:17:25.319310 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 352/709] 10.222.0.47 () {54 vars in 926 bytes} [Thu Jul 21 08:17:25 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 08:17:25.327984 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06643eac-fd9d-46e2-9e31-8eee47a4980c None None] REQUEST_METHOD: `POST` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:17:25.328263 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06643eac-fd9d-46e2-9e31-8eee47a4980c None None] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:17:25.328542 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06643eac-fd9d-46e2-9e31-8eee47a4980c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:17:25.343632 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG dbcounter [-] [80245] Writing DB stats keystone:INSERT=1,keystone:SELECT=99 {{(pid=80245) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:17:25.350220 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.auth.core [None req-06643eac-fd9d-46e2-9e31-8eee47a4980c None None] MFA Rules not processed for user `2dce40b8d74c41ec94cc36e6ff05d412`. Rule list: `[]` (Enabled: `True`). {{(pid=80244) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 08:17:25.351636 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.fernet_utils [None req-06643eac-fd9d-46e2-9e31-8eee47a4980c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80244) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 08:17:25.391044 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4f70b22-b73f-4041-b1be-8469e3b89bbd None admin] Validating token access rules against request {{(pid=80245) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:17:25.428721 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-569350ae-dae2-4364-b6bb-4e5ee89d8a27 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80245) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:17:25.429390 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-569350ae-dae2-4364-b6bb-4e5ee89d8a27 None admin] REQUEST_METHOD: `GET` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:17:25.429524 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG dbcounter [-] [80244] Writing DB stats keystone:SELECT=100 {{(pid=80244) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:17:25.429679 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-569350ae-dae2-4364-b6bb-4e5ee89d8a27 None admin] SCRIPT_NAME: `/identity` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:17:25.429972 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-569350ae-dae2-4364-b6bb-4e5ee89d8a27 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:17:25.430802 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-569350ae-dae2-4364-b6bb-4e5ee89d8a27 None admin] RBAC: Adding query filter params (role.id=e29870f6e9e2411a9db0d5bf45f32ca5, scope.project.id=4faef13aa50e4e36b6c265fb1398fa14, user.id=ac2b5b99a36442148fe743269054e991) {{(pid=80245) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 08:17:25.431116 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-569350ae-dae2-4364-b6bb-4e5ee89d8a27 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:17:25.433828 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-569350ae-dae2-4364-b6bb-4e5ee89d8a27 None admin] RBAC: Authorization granted {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:17:25.435692 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 353/710] 10.222.0.47 () {58 vars in 1023 bytes} [Thu Jul 21 08:17:25 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 21 08:17:25.443546 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: [pid: 80245|app: 0|req: 358/711] 10.222.0.47 () {56 vars in 1374 bytes} [Thu Jul 21 08:17:25 2022] GET /identity/v3/role_assignments?user.id=ac2b5b99a36442148fe743269054e991&scope.project.id=4faef13aa50e4e36b6c265fb1398fa14&role.id=e29870f6e9e2411a9db0d5bf45f32ca5 => generated 259 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 08:17:25.460573 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06643eac-fd9d-46e2-9e31-8eee47a4980c None None] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:17:25.600592 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06643eac-fd9d-46e2-9e31-8eee47a4980c None None] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:17:25.638706 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e7c6130-bcfa-48d8-bd20-445536b7cc1c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:17:25.639551 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e7c6130-bcfa-48d8-bd20-445536b7cc1c None admin] REQUEST_METHOD: `GET` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:17:25.639832 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e7c6130-bcfa-48d8-bd20-445536b7cc1c None admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:17:25.640104 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e7c6130-bcfa-48d8-bd20-445536b7cc1c None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:17:25.640765 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e7c6130-bcfa-48d8-bd20-445536b7cc1c None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:17:25.642150 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e7c6130-bcfa-48d8-bd20-445536b7cc1c None admin] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:17:25.656411 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application [None req-7e7c6130-bcfa-48d8-bd20-445536b7cc1c None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Jul 21 08:17:25.656411 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 08:17:25.656411 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 21 08:17:25.656411 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 21 08:17:25.656411 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 21 08:17:25.656411 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 21 08:17:25.656411 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 21 08:17:25.656411 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 21 08:17:25.656411 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Jul 21 08:17:25.656411 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application Jul 21 08:17:25.656411 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 21 08:17:25.656411 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application Jul 21 08:17:25.656411 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 08:17:25.656411 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 08:17:25.656411 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 08:17:25.656411 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 08:17:25.657412 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 08:17:25.657412 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 08:17:25.657412 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 08:17:25.657412 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 08:17:25.657412 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 08:17:25.657412 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 08:17:25.657412 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 08:17:25.657412 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 21 08:17:25.657412 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 21 08:17:25.657412 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 21 08:17:25.657412 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 21 08:17:25.657412 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 08:17:25.657412 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 08:17:25.657412 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 08:17:25.657412 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 08:17:25.657412 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 08:17:25.658240 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 08:17:25.658240 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 08:17:25.658240 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application with Lock( Jul 21 08:17:25.658240 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 08:17:25.658240 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self._enter() Jul 21 08:17:25.658240 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 08:17:25.658240 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 08:17:25.658240 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 08:17:25.658240 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self.creator() Jul 21 08:17:25.658240 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 08:17:25.658240 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application created_value = creator( Jul 21 08:17:25.658240 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 21 08:17:25.658240 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 21 08:17:25.658240 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Jul 21 08:17:25.658240 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application Jul 21 08:17:25.658240 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 354/712] 10.222.0.47 () {56 vars in 1129 bytes} [Thu Jul 21 08:17:25 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 08:17:25.668428 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-569350ae-dae2-4364-b6bb-4e5ee89d8a27 None admin] Authenticating user token {{(pid=80245) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:17:25.777789 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-569350ae-dae2-4364-b6bb-4e5ee89d8a27 None admin] Validating token access rules against request {{(pid=80245) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:17:25.806732 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG dbcounter [-] [80245] Writing DB stats keystone:SELECT=100 {{(pid=80245) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:17:25.819262 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf8009c4-c966-4981-9522-e4038d163d2f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80245) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:17:25.820138 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf8009c4-c966-4981-9522-e4038d163d2f None admin] REQUEST_METHOD: `GET` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:17:25.820523 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf8009c4-c966-4981-9522-e4038d163d2f None admin] SCRIPT_NAME: `/identity` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:17:25.820898 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf8009c4-c966-4981-9522-e4038d163d2f None admin] PATH_INFO: `/v3/services` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:17:25.821712 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf8009c4-c966-4981-9522-e4038d163d2f None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=80245) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 08:17:25.822139 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf8009c4-c966-4981-9522-e4038d163d2f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:17:25.823634 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf8009c4-c966-4981-9522-e4038d163d2f None admin] RBAC: Authorization granted {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:17:25.828963 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: [pid: 80245|app: 0|req: 359/713] 10.222.0.47 () {56 vars in 1124 bytes} [Thu Jul 21 08:17:25 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 08:17:25.834387 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e7c6130-bcfa-48d8-bd20-445536b7cc1c None admin] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:17:25.911196 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG dbcounter [-] [80244] Writing DB stats keystone:SELECT=100 {{(pid=80244) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:17:25.956446 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e7c6130-bcfa-48d8-bd20-445536b7cc1c None admin] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:17:25.968368 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b248b434-4d1f-45fe-a6c9-546d2aae362e None None] REQUEST_METHOD: `GET` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:17:25.968650 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b248b434-4d1f-45fe-a6c9-546d2aae362e None None] SCRIPT_NAME: `/identity` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:17:25.968926 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b248b434-4d1f-45fe-a6c9-546d2aae362e None None] PATH_INFO: `/` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:17:25.970343 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: [pid: 80245|app: 0|req: 360/714] 10.222.0.47 () {54 vars in 926 bytes} [Thu Jul 21 08:17:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 21 08:17:25.978736 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88d0128c-08bd-4bd0-b09e-1d129c74f225 None None] REQUEST_METHOD: `POST` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:17:25.979021 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88d0128c-08bd-4bd0-b09e-1d129c74f225 None None] SCRIPT_NAME: `/identity` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:17:25.979296 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88d0128c-08bd-4bd0-b09e-1d129c74f225 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:17:26.002601 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.auth.core [None req-88d0128c-08bd-4bd0-b09e-1d129c74f225 None None] MFA Rules not processed for user `2dce40b8d74c41ec94cc36e6ff05d412`. Rule list: `[]` (Enabled: `True`). {{(pid=80245) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 08:17:26.004143 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.fernet_utils [None req-88d0128c-08bd-4bd0-b09e-1d129c74f225 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80245) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 08:17:26.042006 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1eb3736-f7c9-4419-9e0a-382ab2f1649d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:17:26.042606 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1eb3736-f7c9-4419-9e0a-382ab2f1649d None admin] REQUEST_METHOD: `GET` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:17:26.042912 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1eb3736-f7c9-4419-9e0a-382ab2f1649d None admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:17:26.043190 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1eb3736-f7c9-4419-9e0a-382ab2f1649d None admin] PATH_INFO: `/v3/services` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:17:26.043933 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1eb3736-f7c9-4419-9e0a-382ab2f1649d None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=80244) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 08:17:26.044241 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1eb3736-f7c9-4419-9e0a-382ab2f1649d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:17:26.045641 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1eb3736-f7c9-4419-9e0a-382ab2f1649d None admin] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:17:26.062496 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 355/715] 10.222.0.47 () {56 vars in 1124 bytes} [Thu Jul 21 08:17:25 2022] GET /identity/v3/services?type=block-storage => generated 130 bytes in 228 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 08:17:26.099090 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: [pid: 80245|app: 0|req: 361/716] 10.222.0.47 () {58 vars in 1023 bytes} [Thu Jul 21 08:17:25 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 21 08:17:26.151771 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1eb3736-f7c9-4419-9e0a-382ab2f1649d None admin] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:17:26.292897 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1eb3736-f7c9-4419-9e0a-382ab2f1649d None admin] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:17:26.323853 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG dbcounter [-] [80244] Writing DB stats keystone:SELECT=100 {{(pid=80244) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:17:26.333522 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0da977a8-c8e5-4117-a617-d2f57a8819d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:17:26.334318 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0da977a8-c8e5-4117-a617-d2f57a8819d8 None admin] REQUEST_METHOD: `GET` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:17:26.334599 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0da977a8-c8e5-4117-a617-d2f57a8819d8 None admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:17:26.334871 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0da977a8-c8e5-4117-a617-d2f57a8819d8 None admin] PATH_INFO: `/v3/services/network` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:17:26.335553 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0da977a8-c8e5-4117-a617-d2f57a8819d8 None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:17:26.336933 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0da977a8-c8e5-4117-a617-d2f57a8819d8 None admin] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:17:26.349085 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application [None req-0da977a8-c8e5-4117-a617-d2f57a8819d8 None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Jul 21 08:17:26.349085 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 08:17:26.349085 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 21 08:17:26.349085 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 21 08:17:26.349085 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 21 08:17:26.349085 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 21 08:17:26.349085 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 21 08:17:26.349085 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 21 08:17:26.349085 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Jul 21 08:17:26.349085 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application Jul 21 08:17:26.349085 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 21 08:17:26.349085 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application Jul 21 08:17:26.349085 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 08:17:26.349085 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 08:17:26.349085 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 08:17:26.349085 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 08:17:26.349085 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 08:17:26.350123 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 08:17:26.350123 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 08:17:26.350123 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 08:17:26.350123 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 08:17:26.350123 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 08:17:26.350123 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 08:17:26.350123 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 21 08:17:26.350123 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 21 08:17:26.350123 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 21 08:17:26.350123 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 21 08:17:26.350123 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 08:17:26.350123 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 08:17:26.350123 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 08:17:26.350123 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 08:17:26.350123 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 08:17:26.350123 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 08:17:26.351680 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 08:17:26.351680 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application with Lock( Jul 21 08:17:26.351680 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 08:17:26.351680 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self._enter() Jul 21 08:17:26.351680 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 08:17:26.351680 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 08:17:26.351680 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 08:17:26.351680 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self.creator() Jul 21 08:17:26.351680 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 08:17:26.351680 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application created_value = creator( Jul 21 08:17:26.351680 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 21 08:17:26.351680 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 21 08:17:26.351680 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Jul 21 08:17:26.351680 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application Jul 21 08:17:26.351680 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 356/717] 10.222.0.47 () {56 vars in 1111 bytes} [Thu Jul 21 08:17:26 2022] GET /identity/v3/services/network => generated 88 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 08:17:26.356441 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88d0128c-08bd-4bd0-b09e-1d129c74f225 None None] Authenticating user token {{(pid=80245) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:17:26.535291 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88d0128c-08bd-4bd0-b09e-1d129c74f225 None None] Validating token access rules against request {{(pid=80245) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:17:26.561229 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG dbcounter [-] [80245] Writing DB stats keystone:SELECT=100 {{(pid=80245) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:17:26.593434 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39359d35-0fc4-4132-922e-8edce329d3d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80245) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:17:26.594123 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39359d35-0fc4-4132-922e-8edce329d3d6 None admin] REQUEST_METHOD: `GET` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:17:26.594401 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39359d35-0fc4-4132-922e-8edce329d3d6 None admin] SCRIPT_NAME: `/identity` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:17:26.594671 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39359d35-0fc4-4132-922e-8edce329d3d6 None admin] PATH_INFO: `/v3/services` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:17:26.595408 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39359d35-0fc4-4132-922e-8edce329d3d6 None admin] RBAC: Adding query filter params (name=network) {{(pid=80245) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 08:17:26.595714 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39359d35-0fc4-4132-922e-8edce329d3d6 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:17:26.597076 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39359d35-0fc4-4132-922e-8edce329d3d6 None admin] RBAC: Authorization granted {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:17:26.603131 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: [pid: 80245|app: 0|req: 362/718] 10.222.0.47 () {56 vars in 1112 bytes} [Thu Jul 21 08:17:26 2022] GET /identity/v3/services?name=network => generated 124 bytes in 247 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 08:17:26.609112 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0da977a8-c8e5-4117-a617-d2f57a8819d8 None admin] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:17:26.777865 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0da977a8-c8e5-4117-a617-d2f57a8819d8 None admin] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:17:26.822260 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ad16f0c-c96d-4bfc-b49e-0b62b8564c4f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:17:26.822847 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ad16f0c-c96d-4bfc-b49e-0b62b8564c4f None admin] REQUEST_METHOD: `GET` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:17:26.823126 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ad16f0c-c96d-4bfc-b49e-0b62b8564c4f None admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:17:26.823426 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ad16f0c-c96d-4bfc-b49e-0b62b8564c4f None admin] PATH_INFO: `/v3/services` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:17:26.824140 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ad16f0c-c96d-4bfc-b49e-0b62b8564c4f None admin] RBAC: Adding query filter params (type=network) {{(pid=80244) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 08:17:26.824441 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ad16f0c-c96d-4bfc-b49e-0b62b8564c4f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:17:26.825802 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ad16f0c-c96d-4bfc-b49e-0b62b8564c4f None admin] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:17:26.830765 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 357/719] 10.222.0.47 () {56 vars in 1112 bytes} [Thu Jul 21 08:17:26 2022] GET /identity/v3/services?type=network => generated 124 bytes in 222 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 08:17:27.628864 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baefbf76-00ce-4582-b081-d144325dcaa6 None None] REQUEST_METHOD: `GET` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:17:27.629358 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baefbf76-00ce-4582-b081-d144325dcaa6 None None] SCRIPT_NAME: `/identity` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:17:27.629459 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baefbf76-00ce-4582-b081-d144325dcaa6 None None] PATH_INFO: `/` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:17:27.631076 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: [pid: 80245|app: 0|req: 363/720] 10.222.0.47 () {54 vars in 926 bytes} [Thu Jul 21 08:17:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 08:17:27.636925 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce97511d-5de4-4bce-8bc9-cd3c95075415 None None] REQUEST_METHOD: `POST` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:17:27.637356 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce97511d-5de4-4bce-8bc9-cd3c95075415 None None] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:17:27.637531 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce97511d-5de4-4bce-8bc9-cd3c95075415 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:17:27.661187 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.auth.core [None req-ce97511d-5de4-4bce-8bc9-cd3c95075415 None None] MFA Rules not processed for user `2dce40b8d74c41ec94cc36e6ff05d412`. Rule list: `[]` (Enabled: `True`). {{(pid=80244) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 08:17:27.662419 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.fernet_utils [None req-ce97511d-5de4-4bce-8bc9-cd3c95075415 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80244) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 08:17:27.743810 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG dbcounter [-] [80244] Writing DB stats keystone:SELECT=100 {{(pid=80244) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:17:27.763272 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 358/721] 10.222.0.47 () {58 vars in 1023 bytes} [Thu Jul 21 08:17:27 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 21 08:17:27.799578 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baefbf76-00ce-4582-b081-d144325dcaa6 None None] Authenticating user token {{(pid=80245) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:17:27.941113 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31742036-9825-48ca-8c99-d7ce00b8faa1 None None] REQUEST_METHOD: `GET` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:17:27.942296 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31742036-9825-48ca-8c99-d7ce00b8faa1 None None] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:17:27.942784 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31742036-9825-48ca-8c99-d7ce00b8faa1 None None] PATH_INFO: `/` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:17:27.944677 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 359/722] 10.222.0.47 () {54 vars in 926 bytes} [Thu Jul 21 08:17:27 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 08:17:27.948030 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baefbf76-00ce-4582-b081-d144325dcaa6 None None] Validating token access rules against request {{(pid=80245) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:17:27.948854 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1f7657a-92e5-4fc1-8507-b979c3bc4567 None None] REQUEST_METHOD: `GET` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:17:27.949147 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1f7657a-92e5-4fc1-8507-b979c3bc4567 None None] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:17:27.949425 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1f7657a-92e5-4fc1-8507-b979c3bc4567 None None] PATH_INFO: `/` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:17:27.950719 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 360/723] 10.222.0.47 () {54 vars in 926 bytes} [Thu Jul 21 08:17:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 21 08:17:27.951787 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43bd0a07-2332-49da-a209-432f0cc01fb9 None None] REQUEST_METHOD: `POST` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:17:27.952067 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43bd0a07-2332-49da-a209-432f0cc01fb9 None None] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:17:27.952339 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43bd0a07-2332-49da-a209-432f0cc01fb9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:17:27.973011 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.auth.core [None req-43bd0a07-2332-49da-a209-432f0cc01fb9 None None] MFA Rules not processed for user `2dce40b8d74c41ec94cc36e6ff05d412`. Rule list: `[]` (Enabled: `True`). {{(pid=80244) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 08:17:27.974214 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.fernet_utils [None req-43bd0a07-2332-49da-a209-432f0cc01fb9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80244) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 08:17:27.989570 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38d43c08-23d6-4df0-988e-2422e32709d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80245) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:17:27.990845 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38d43c08-23d6-4df0-988e-2422e32709d8 None admin] REQUEST_METHOD: `GET` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:17:27.991126 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38d43c08-23d6-4df0-988e-2422e32709d8 None admin] SCRIPT_NAME: `/identity` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:17:27.991786 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38d43c08-23d6-4df0-988e-2422e32709d8 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:17:28.005164 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38d43c08-23d6-4df0-988e-2422e32709d8 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 21 08:17:28.013436 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application [None req-38d43c08-23d6-4df0-988e-2422e32709d8 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 21 08:17:28.013436 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 08:17:28.013436 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 08:17:28.013436 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 08:17:28.013436 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 08:17:28.013436 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 08:17:28.013436 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 08:17:28.013436 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 08:17:28.013436 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 08:17:28.013436 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 08:17:28.013436 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 08:17:28.013436 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 08:17:28.013436 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 21 08:17:28.013436 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 21 08:17:28.013436 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 21 08:17:28.013436 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 21 08:17:28.014329 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 21 08:17:28.014329 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 21 08:17:28.014329 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 21 08:17:28.014329 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 21 08:17:28.014329 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 08:17:28.014329 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 08:17:28.014329 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 08:17:28.014329 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 08:17:28.014329 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 08:17:28.014329 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 08:17:28.014329 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 08:17:28.014329 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application with Lock( Jul 21 08:17:28.014329 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 08:17:28.014329 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application return self._enter() Jul 21 08:17:28.014329 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 08:17:28.014329 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 08:17:28.015155 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 08:17:28.015155 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application return self.creator() Jul 21 08:17:28.015155 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 08:17:28.015155 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application created_value = creator( Jul 21 08:17:28.015155 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 21 08:17:28.015155 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 21 08:17:28.015155 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 21 08:17:28.015155 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 21 08:17:28.015155 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 21 08:17:28.015155 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 21 08:17:28.015155 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 21 08:17:28.015155 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application Jul 21 08:17:28.020122 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: [pid: 80245|app: 0|req: 364/724] 10.222.0.47 () {56 vars in 1102 bytes} [Thu Jul 21 08:17:27 2022] GET /identity/v3/roles/service => generated 85 bytes in 216 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 08:17:28.021461 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d90f3f93-4da1-47e9-a9df-61114f400f6f None None] REQUEST_METHOD: `POST` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:17:28.021752 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d90f3f93-4da1-47e9-a9df-61114f400f6f None None] SCRIPT_NAME: `/identity` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:17:28.022035 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d90f3f93-4da1-47e9-a9df-61114f400f6f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:17:28.039037 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.auth.core [None req-d90f3f93-4da1-47e9-a9df-61114f400f6f None None] MFA Rules not processed for user `2dce40b8d74c41ec94cc36e6ff05d412`. Rule list: `[]` (Enabled: `True`). {{(pid=80245) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 08:17:28.040260 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.fernet_utils [None req-d90f3f93-4da1-47e9-a9df-61114f400f6f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80245) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 08:17:28.049778 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 361/725] 10.222.0.47 () {58 vars in 1023 bytes} [Thu Jul 21 08:17:27 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 21 08:17:28.050730 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43bd0a07-2332-49da-a209-432f0cc01fb9 None None] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:17:28.086443 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG dbcounter [-] [80245] Writing DB stats keystone:SELECT=100 {{(pid=80245) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:17:28.122820 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: [pid: 80245|app: 0|req: 365/726] 10.222.0.47 () {58 vars in 1023 bytes} [Thu Jul 21 08:17:28 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 21 08:17:28.123816 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d90f3f93-4da1-47e9-a9df-61114f400f6f None None] Authenticating user token {{(pid=80245) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:17:28.190869 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43bd0a07-2332-49da-a209-432f0cc01fb9 None None] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:17:28.209746 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG dbcounter [-] [80244] Writing DB stats keystone:SELECT=100 {{(pid=80244) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:17:28.241209 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b021120-fcfa-42d1-8b2c-f54301468be3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:17:28.242616 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b021120-fcfa-42d1-8b2c-f54301468be3 None admin] REQUEST_METHOD: `GET` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:17:28.243267 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b021120-fcfa-42d1-8b2c-f54301468be3 None admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:17:28.243950 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b021120-fcfa-42d1-8b2c-f54301468be3 None admin] PATH_INFO: `/v3/roles` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:17:28.245416 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b021120-fcfa-42d1-8b2c-f54301468be3 None admin] RBAC: Adding query filter params (name=service) {{(pid=80244) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 08:17:28.246108 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b021120-fcfa-42d1-8b2c-f54301468be3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:17:28.249142 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b021120-fcfa-42d1-8b2c-f54301468be3 None admin] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:17:28.264388 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 362/727] 10.222.0.47 () {56 vars in 1103 bytes} [Thu Jul 21 08:17:28 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 08:17:28.267504 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b021120-fcfa-42d1-8b2c-f54301468be3 None admin] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:17:28.291487 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d90f3f93-4da1-47e9-a9df-61114f400f6f None None] Validating token access rules against request {{(pid=80245) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:17:28.330173 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e62b804-7345-4c0d-8ca5-bcdbcfaf6870 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80245) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:17:28.331276 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e62b804-7345-4c0d-8ca5-bcdbcfaf6870 None admin] REQUEST_METHOD: `GET` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:17:28.331795 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e62b804-7345-4c0d-8ca5-bcdbcfaf6870 None admin] SCRIPT_NAME: `/identity` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:17:28.332350 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e62b804-7345-4c0d-8ca5-bcdbcfaf6870 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:17:28.344722 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e62b804-7345-4c0d-8ca5-bcdbcfaf6870 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 21 08:17:28.353383 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application [None req-4e62b804-7345-4c0d-8ca5-bcdbcfaf6870 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 21 08:17:28.353383 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 08:17:28.353383 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 08:17:28.353383 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 08:17:28.353383 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 08:17:28.353383 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 08:17:28.353383 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 08:17:28.353383 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 08:17:28.353383 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 08:17:28.353383 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 08:17:28.353383 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 08:17:28.353383 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 08:17:28.353383 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 21 08:17:28.353383 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 21 08:17:28.353383 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 21 08:17:28.353383 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 21 08:17:28.354500 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 21 08:17:28.354500 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 21 08:17:28.354500 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 21 08:17:28.354500 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 21 08:17:28.354500 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 08:17:28.354500 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 08:17:28.354500 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 08:17:28.354500 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 08:17:28.354500 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 08:17:28.354500 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 08:17:28.354500 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 08:17:28.354500 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application with Lock( Jul 21 08:17:28.354500 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 08:17:28.354500 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application return self._enter() Jul 21 08:17:28.354500 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 08:17:28.354500 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 08:17:28.355515 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 08:17:28.355515 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application return self.creator() Jul 21 08:17:28.355515 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 08:17:28.355515 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application created_value = creator( Jul 21 08:17:28.355515 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 21 08:17:28.355515 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 21 08:17:28.355515 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 21 08:17:28.355515 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 21 08:17:28.355515 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 21 08:17:28.355515 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 21 08:17:28.355515 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 21 08:17:28.355515 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application Jul 21 08:17:28.357802 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: [pid: 80245|app: 0|req: 366/728] 10.222.0.47 () {56 vars in 1096 bytes} [Thu Jul 21 08:17:28 2022] GET /identity/v3/roles/admin => generated 83 bytes in 234 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 08:17:28.358721 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e62b804-7345-4c0d-8ca5-bcdbcfaf6870 None admin] Authenticating user token {{(pid=80245) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:17:28.396394 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG dbcounter [-] [80245] Writing DB stats keystone:SELECT=100 {{(pid=80245) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:17:28.406738 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b021120-fcfa-42d1-8b2c-f54301468be3 None admin] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:17:28.457721 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b23b5f89-82a4-467b-a041-d12e1e61cf73 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:17:28.458781 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b23b5f89-82a4-467b-a041-d12e1e61cf73 None admin] REQUEST_METHOD: `GET` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:17:28.459073 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b23b5f89-82a4-467b-a041-d12e1e61cf73 None admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:17:28.459372 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b23b5f89-82a4-467b-a041-d12e1e61cf73 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:17:28.476088 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b23b5f89-82a4-467b-a041-d12e1e61cf73 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 21 08:17:28.476475 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e62b804-7345-4c0d-8ca5-bcdbcfaf6870 None admin] Validating token access rules against request {{(pid=80245) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:17:28.486873 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application [None req-b23b5f89-82a4-467b-a041-d12e1e61cf73 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 21 08:17:28.486873 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 08:17:28.486873 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 08:17:28.486873 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 08:17:28.486873 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 08:17:28.486873 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 08:17:28.486873 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 08:17:28.486873 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 08:17:28.486873 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 08:17:28.486873 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 08:17:28.486873 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 08:17:28.486873 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 08:17:28.486873 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 21 08:17:28.486873 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 21 08:17:28.486873 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 21 08:17:28.486873 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 21 08:17:28.487864 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 21 08:17:28.487864 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 21 08:17:28.487864 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 21 08:17:28.487864 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 21 08:17:28.487864 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 08:17:28.487864 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 08:17:28.487864 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 08:17:28.487864 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 08:17:28.487864 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 08:17:28.487864 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 08:17:28.487864 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 08:17:28.487864 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application with Lock( Jul 21 08:17:28.487864 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 08:17:28.487864 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self._enter() Jul 21 08:17:28.487864 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 08:17:28.487864 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 08:17:28.489371 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 08:17:28.489371 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self.creator() Jul 21 08:17:28.489371 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 08:17:28.489371 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application created_value = creator( Jul 21 08:17:28.489371 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 21 08:17:28.489371 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 21 08:17:28.489371 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 21 08:17:28.489371 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 21 08:17:28.489371 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 21 08:17:28.489371 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 21 08:17:28.489371 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 21 08:17:28.489371 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application Jul 21 08:17:28.494274 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 363/729] 10.222.0.47 () {56 vars in 1096 bytes} [Thu Jul 21 08:17:28 2022] GET /identity/v3/roles/admin => generated 83 bytes in 223 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 08:17:28.495601 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b23b5f89-82a4-467b-a041-d12e1e61cf73 None admin] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:17:28.526652 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce1e3899-2dec-43cd-bca2-3543f4dca53e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80245) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:17:28.527466 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce1e3899-2dec-43cd-bca2-3543f4dca53e None admin] REQUEST_METHOD: `GET` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:17:28.527754 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce1e3899-2dec-43cd-bca2-3543f4dca53e None admin] SCRIPT_NAME: `/identity` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:17:28.528034 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce1e3899-2dec-43cd-bca2-3543f4dca53e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:17:28.544876 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce1e3899-2dec-43cd-bca2-3543f4dca53e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:17:28.546337 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce1e3899-2dec-43cd-bca2-3543f4dca53e None admin] RBAC: Authorization granted {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:17:28.561718 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG dbcounter [-] [80244] Writing DB stats keystone:SELECT=100 {{(pid=80244) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:17:28.567671 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application [None req-ce1e3899-2dec-43cd-bca2-3543f4dca53e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 08:17:28.567671 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 08:17:28.567671 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 08:17:28.567671 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 08:17:28.567671 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 08:17:28.567671 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 08:17:28.567671 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 08:17:28.567671 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 08:17:28.567671 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 08:17:28.567671 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 08:17:28.567671 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 08:17:28.567671 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 08:17:28.567671 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 21 08:17:28.567671 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 21 08:17:28.567671 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 21 08:17:28.568584 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 21 08:17:28.568584 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 08:17:28.568584 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 08:17:28.568584 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 08:17:28.568584 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 08:17:28.568584 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 08:17:28.568584 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 08:17:28.568584 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 08:17:28.568584 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application with Lock( Jul 21 08:17:28.568584 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 08:17:28.568584 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application return self._enter() Jul 21 08:17:28.568584 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 08:17:28.568584 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 08:17:28.568584 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 08:17:28.568584 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application return self.creator() Jul 21 08:17:28.568584 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 08:17:28.568584 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application created_value = creator( Jul 21 08:17:28.569442 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 21 08:17:28.569442 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 21 08:17:28.569442 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 08:17:28.569442 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application Jul 21 08:17:28.573701 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: [pid: 80245|app: 0|req: 367/730] 10.222.0.47 () {56 vars in 1108 bytes} [Thu Jul 21 08:17:28 2022] GET /identity/v3/domains/Default => generated 87 bytes in 211 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 08:17:28.574592 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce1e3899-2dec-43cd-bca2-3543f4dca53e None admin] Authenticating user token {{(pid=80245) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:17:28.645853 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b23b5f89-82a4-467b-a041-d12e1e61cf73 None admin] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:17:28.668161 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG dbcounter [-] [80245] Writing DB stats keystone:SELECT=100 {{(pid=80245) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:17:28.687838 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8599c8a9-16c7-4e58-9750-93435e3ae7c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:17:28.688520 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8599c8a9-16c7-4e58-9750-93435e3ae7c2 None admin] REQUEST_METHOD: `GET` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:17:28.688805 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8599c8a9-16c7-4e58-9750-93435e3ae7c2 None admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:17:28.689082 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8599c8a9-16c7-4e58-9750-93435e3ae7c2 None admin] PATH_INFO: `/v3/roles` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:17:28.689549 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce1e3899-2dec-43cd-bca2-3543f4dca53e None admin] Validating token access rules against request {{(pid=80245) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:17:28.689863 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8599c8a9-16c7-4e58-9750-93435e3ae7c2 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80244) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 08:17:28.690239 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8599c8a9-16c7-4e58-9750-93435e3ae7c2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:17:28.692151 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8599c8a9-16c7-4e58-9750-93435e3ae7c2 None admin] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:17:28.703682 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 364/731] 10.222.0.47 () {56 vars in 1099 bytes} [Thu Jul 21 08:17:28 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 08:17:28.703682 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8599c8a9-16c7-4e58-9750-93435e3ae7c2 None admin] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:17:28.731436 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42917b67-e654-4c24-a194-364d6e5639d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80245) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:17:28.732392 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42917b67-e654-4c24-a194-364d6e5639d4 None admin] REQUEST_METHOD: `GET` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:17:28.732684 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42917b67-e654-4c24-a194-364d6e5639d4 None admin] SCRIPT_NAME: `/identity` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:17:28.732982 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42917b67-e654-4c24-a194-364d6e5639d4 None admin] PATH_INFO: `/v3/roles` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:17:28.734164 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42917b67-e654-4c24-a194-364d6e5639d4 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80245) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 08:17:28.734486 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42917b67-e654-4c24-a194-364d6e5639d4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:17:28.736149 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42917b67-e654-4c24-a194-364d6e5639d4 None admin] RBAC: Authorization granted {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:17:28.744533 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: [pid: 80245|app: 0|req: 368/732] 10.222.0.47 () {56 vars in 1099 bytes} [Thu Jul 21 08:17:28 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 08:17:28.745795 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c1ed977-fb33-4f08-9d8e-c5fcfc6975fc None None] REQUEST_METHOD: `GET` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:17:28.746086 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c1ed977-fb33-4f08-9d8e-c5fcfc6975fc None None] SCRIPT_NAME: `/identity` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:17:28.746366 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c1ed977-fb33-4f08-9d8e-c5fcfc6975fc None None] PATH_INFO: `/` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:17:28.747763 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: [pid: 80245|app: 0|req: 369/733] 10.222.0.47 () {54 vars in 926 bytes} [Thu Jul 21 08:17:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 08:17:28.752215 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c1ed977-fb33-4f08-9d8e-c5fcfc6975fc None None] Authenticating user token {{(pid=80245) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:17:28.808008 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG dbcounter [-] [80244] Writing DB stats keystone:SELECT=100 {{(pid=80244) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:17:28.811208 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8599c8a9-16c7-4e58-9750-93435e3ae7c2 None admin] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:17:28.849968 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-207e8a26-d623-4497-80e9-512408490705 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:17:28.850989 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-207e8a26-d623-4497-80e9-512408490705 None admin] REQUEST_METHOD: `GET` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:17:28.851281 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-207e8a26-d623-4497-80e9-512408490705 None admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:17:28.851726 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-207e8a26-d623-4497-80e9-512408490705 None admin] PATH_INFO: `/v3/domains` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:17:28.852460 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-207e8a26-d623-4497-80e9-512408490705 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80244) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 08:17:28.852776 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-207e8a26-d623-4497-80e9-512408490705 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:17:28.854173 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-207e8a26-d623-4497-80e9-512408490705 None admin] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:17:28.860367 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c1ed977-fb33-4f08-9d8e-c5fcfc6975fc None None] Validating token access rules against request {{(pid=80245) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:17:28.872224 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 365/734] 10.222.0.47 () {56 vars in 1109 bytes} [Thu Jul 21 08:17:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 08:17:28.872224 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-207e8a26-d623-4497-80e9-512408490705 None admin] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:17:28.898988 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-140d8ffd-7291-4244-9a11-cfb320c6624f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80245) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:17:28.899813 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-140d8ffd-7291-4244-9a11-cfb320c6624f None admin] REQUEST_METHOD: `GET` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:17:28.900091 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-140d8ffd-7291-4244-9a11-cfb320c6624f None admin] SCRIPT_NAME: `/identity` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:17:28.900364 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-140d8ffd-7291-4244-9a11-cfb320c6624f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:17:28.913205 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-140d8ffd-7291-4244-9a11-cfb320c6624f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:17:28.914658 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-140d8ffd-7291-4244-9a11-cfb320c6624f None admin] RBAC: Authorization granted {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:17:28.918078 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG dbcounter [-] [80245] Writing DB stats keystone:SELECT=100 {{(pid=80245) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:17:28.935012 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application [None req-140d8ffd-7291-4244-9a11-cfb320c6624f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 08:17:28.935012 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 08:17:28.935012 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 08:17:28.935012 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 08:17:28.935012 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 08:17:28.935012 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 08:17:28.935012 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 08:17:28.935012 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 08:17:28.935012 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 08:17:28.935012 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 08:17:28.935012 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 08:17:28.935012 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 08:17:28.935012 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 21 08:17:28.935012 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 21 08:17:28.935012 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 21 08:17:28.935960 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 21 08:17:28.935960 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 08:17:28.935960 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 08:17:28.935960 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 08:17:28.935960 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 08:17:28.935960 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 08:17:28.935960 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 08:17:28.935960 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 08:17:28.935960 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application with Lock( Jul 21 08:17:28.935960 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 08:17:28.935960 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application return self._enter() Jul 21 08:17:28.935960 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 08:17:28.935960 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 08:17:28.935960 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 08:17:28.935960 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application return self.creator() Jul 21 08:17:28.935960 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 08:17:28.935960 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application created_value = creator( Jul 21 08:17:28.936861 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 21 08:17:28.936861 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 21 08:17:28.936861 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 08:17:28.936861 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application Jul 21 08:17:28.936861 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: [pid: 80245|app: 0|req: 370/735] 10.222.0.47 () {56 vars in 1108 bytes} [Thu Jul 21 08:17:28 2022] GET /identity/v3/domains/Default => generated 87 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 08:17:28.937923 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5d82148-3290-4051-8663-bba143ee9975 None None] REQUEST_METHOD: `POST` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:17:28.937923 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5d82148-3290-4051-8663-bba143ee9975 None None] SCRIPT_NAME: `/identity` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:17:28.938098 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5d82148-3290-4051-8663-bba143ee9975 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:17:28.955753 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.auth.core [None req-a5d82148-3290-4051-8663-bba143ee9975 None None] MFA Rules not processed for user `2dce40b8d74c41ec94cc36e6ff05d412`. Rule list: `[]` (Enabled: `True`). {{(pid=80245) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 08:17:28.957488 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.fernet_utils [None req-a5d82148-3290-4051-8663-bba143ee9975 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80245) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 08:17:28.987033 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-207e8a26-d623-4497-80e9-512408490705 None admin] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:17:29.023424 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-585f2cd5-4ac7-43c2-8dc9-b33589e48290 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:17:29.024220 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-585f2cd5-4ac7-43c2-8dc9-b33589e48290 None admin] REQUEST_METHOD: `GET` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:17:29.024510 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-585f2cd5-4ac7-43c2-8dc9-b33589e48290 None admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:17:29.024782 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-585f2cd5-4ac7-43c2-8dc9-b33589e48290 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:17:29.031991 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: [pid: 80245|app: 0|req: 371/736] 10.222.0.47 () {58 vars in 1023 bytes} [Thu Jul 21 08:17:28 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 21 08:17:29.033030 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5d82148-3290-4051-8663-bba143ee9975 None None] Authenticating user token {{(pid=80245) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:17:29.042554 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-585f2cd5-4ac7-43c2-8dc9-b33589e48290 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:17:29.044165 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-585f2cd5-4ac7-43c2-8dc9-b33589e48290 None admin] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:17:29.062269 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application [None req-585f2cd5-4ac7-43c2-8dc9-b33589e48290 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 08:17:29.062269 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 08:17:29.062269 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 08:17:29.062269 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 08:17:29.062269 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 08:17:29.062269 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 08:17:29.062269 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 08:17:29.062269 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 08:17:29.062269 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 08:17:29.062269 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 08:17:29.062269 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 08:17:29.062269 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 08:17:29.062269 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 21 08:17:29.062269 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 21 08:17:29.062269 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 21 08:17:29.063289 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 21 08:17:29.063289 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 08:17:29.063289 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 08:17:29.063289 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 08:17:29.063289 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 08:17:29.063289 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 08:17:29.063289 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 08:17:29.063289 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 08:17:29.063289 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application with Lock( Jul 21 08:17:29.063289 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 08:17:29.063289 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self._enter() Jul 21 08:17:29.063289 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 08:17:29.063289 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 08:17:29.063289 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 08:17:29.063289 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self.creator() Jul 21 08:17:29.063289 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 08:17:29.063289 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application created_value = creator( Jul 21 08:17:29.064356 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 21 08:17:29.064356 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 21 08:17:29.064356 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 08:17:29.064356 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application Jul 21 08:17:29.068233 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 366/737] 10.222.0.47 () {56 vars in 1108 bytes} [Thu Jul 21 08:17:28 2022] GET /identity/v3/domains/Default => generated 87 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 08:17:29.069144 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-585f2cd5-4ac7-43c2-8dc9-b33589e48290 None admin] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:17:29.088102 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG dbcounter [-] [80244] Writing DB stats keystone:SELECT=100 {{(pid=80244) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:17:29.145787 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5d82148-3290-4051-8663-bba143ee9975 None None] Validating token access rules against request {{(pid=80245) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:17:29.173650 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG dbcounter [-] [80245] Writing DB stats keystone:SELECT=100 {{(pid=80245) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:17:29.182637 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f26c3dd9-477a-4e3e-838e-5a729de68886 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80245) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:17:29.183482 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f26c3dd9-477a-4e3e-838e-5a729de68886 None admin] REQUEST_METHOD: `GET` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:17:29.183765 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f26c3dd9-477a-4e3e-838e-5a729de68886 None admin] SCRIPT_NAME: `/identity` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:17:29.184033 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f26c3dd9-477a-4e3e-838e-5a729de68886 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:17:29.184267 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-585f2cd5-4ac7-43c2-8dc9-b33589e48290 None admin] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:17:29.191245 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f26c3dd9-477a-4e3e-838e-5a729de68886 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:17:29.192766 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f26c3dd9-477a-4e3e-838e-5a729de68886 None admin] RBAC: Authorization granted {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:17:29.208175 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application [None req-f26c3dd9-477a-4e3e-838e-5a729de68886 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 21 08:17:29.208175 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 08:17:29.208175 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 08:17:29.208175 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 08:17:29.208175 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 08:17:29.208175 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 08:17:29.208175 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 08:17:29.208175 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 08:17:29.208175 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 08:17:29.208175 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 08:17:29.208175 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 08:17:29.208175 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 08:17:29.208175 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 21 08:17:29.208175 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 21 08:17:29.208175 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 21 08:17:29.208175 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 21 08:17:29.209141 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 08:17:29.209141 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 08:17:29.209141 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 21 08:17:29.209141 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 21 08:17:29.209141 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 21 08:17:29.209141 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 21 08:17:29.209141 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 08:17:29.209141 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 08:17:29.209141 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 08:17:29.209141 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 08:17:29.209141 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 08:17:29.209141 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application with Lock( Jul 21 08:17:29.209141 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 08:17:29.209141 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application return self._enter() Jul 21 08:17:29.209141 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 08:17:29.209141 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 08:17:29.209141 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 08:17:29.209141 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application return self.creator() Jul 21 08:17:29.210052 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 08:17:29.210052 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application created_value = creator( Jul 21 08:17:29.210052 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 21 08:17:29.210052 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 21 08:17:29.210052 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 21 08:17:29.210052 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 21 08:17:29.210052 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 21 08:17:29.210052 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 21 08:17:29.210052 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 21 08:17:29.210052 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application Jul 21 08:17:29.210052 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: [pid: 80245|app: 0|req: 372/738] 10.222.0.47 () {56 vars in 1117 bytes} [Thu Jul 21 08:17:29 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 08:17:29.211240 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f26c3dd9-477a-4e3e-838e-5a729de68886 None admin] Authenticating user token {{(pid=80245) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:17:29.228045 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-960449f9-4fe4-42c5-a345-8fc1f63bdeb6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:17:29.228715 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-960449f9-4fe4-42c5-a345-8fc1f63bdeb6 None admin] REQUEST_METHOD: `GET` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:17:29.229045 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-960449f9-4fe4-42c5-a345-8fc1f63bdeb6 None admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:17:29.229279 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-960449f9-4fe4-42c5-a345-8fc1f63bdeb6 None admin] PATH_INFO: `/v3/domains` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:17:29.230144 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-960449f9-4fe4-42c5-a345-8fc1f63bdeb6 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80244) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 08:17:29.230454 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-960449f9-4fe4-42c5-a345-8fc1f63bdeb6 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:17:29.231880 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-960449f9-4fe4-42c5-a345-8fc1f63bdeb6 None admin] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:17:29.248821 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 367/739] 10.222.0.47 () {56 vars in 1109 bytes} [Thu Jul 21 08:17:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 08:17:29.248821 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-960449f9-4fe4-42c5-a345-8fc1f63bdeb6 None admin] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:17:29.317666 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f26c3dd9-477a-4e3e-838e-5a729de68886 None admin] Validating token access rules against request {{(pid=80245) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:17:29.321710 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG dbcounter [-] [80244] Writing DB stats keystone:SELECT=100 {{(pid=80244) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:17:29.353006 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69d6cce8-e2e8-4cf9-bd18-50daaf8c6bbd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80245) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:17:29.353672 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69d6cce8-e2e8-4cf9-bd18-50daaf8c6bbd None admin] REQUEST_METHOD: `POST` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:17:29.353950 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69d6cce8-e2e8-4cf9-bd18-50daaf8c6bbd None admin] SCRIPT_NAME: `/identity` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:17:29.354222 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69d6cce8-e2e8-4cf9-bd18-50daaf8c6bbd None admin] PATH_INFO: `/v3/services` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:17:29.354444 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-960449f9-4fe4-42c5-a345-8fc1f63bdeb6 None admin] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:17:29.354938 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69d6cce8-e2e8-4cf9-bd18-50daaf8c6bbd None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:17:29.356345 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69d6cce8-e2e8-4cf9-bd18-50daaf8c6bbd None admin] RBAC: Authorization granted {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:17:29.383387 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: [pid: 80245|app: 0|req: 373/740] 10.222.0.47 () {60 vars in 1141 bytes} [Thu Jul 21 08:17:29 2022] POST /identity/v3/services => generated 251 bytes in 162 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 08:17:29.383387 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69d6cce8-e2e8-4cf9-bd18-50daaf8c6bbd None admin] Authenticating user token {{(pid=80245) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:17:29.414636 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fab3a3ae-1e63-4a97-99fb-703f2ed90913 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:17:29.416069 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fab3a3ae-1e63-4a97-99fb-703f2ed90913 None admin] REQUEST_METHOD: `GET` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:17:29.416362 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fab3a3ae-1e63-4a97-99fb-703f2ed90913 None admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:17:29.416962 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fab3a3ae-1e63-4a97-99fb-703f2ed90913 None admin] PATH_INFO: `/v3/domains` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:17:29.417731 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fab3a3ae-1e63-4a97-99fb-703f2ed90913 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80244) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 08:17:29.418042 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fab3a3ae-1e63-4a97-99fb-703f2ed90913 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:17:29.420332 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fab3a3ae-1e63-4a97-99fb-703f2ed90913 None admin] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:17:29.438153 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 368/741] 10.222.0.47 () {56 vars in 1109 bytes} [Thu Jul 21 08:17:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 08:17:29.439672 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fab3a3ae-1e63-4a97-99fb-703f2ed90913 None admin] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:17:29.445578 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG dbcounter [-] [80245] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80245) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:17:29.514713 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69d6cce8-e2e8-4cf9-bd18-50daaf8c6bbd None admin] Validating token access rules against request {{(pid=80245) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:17:29.561684 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c601f0d7-b486-4f6f-9213-25e40e74b2d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80245) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:17:29.562403 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c601f0d7-b486-4f6f-9213-25e40e74b2d4 None admin] REQUEST_METHOD: `GET` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:17:29.562720 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c601f0d7-b486-4f6f-9213-25e40e74b2d4 None admin] SCRIPT_NAME: `/identity` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:17:29.562960 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c601f0d7-b486-4f6f-9213-25e40e74b2d4 None admin] PATH_INFO: `/v3/users` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:17:29.563836 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c601f0d7-b486-4f6f-9213-25e40e74b2d4 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80245) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 08:17:29.564145 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c601f0d7-b486-4f6f-9213-25e40e74b2d4 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:17:29.565640 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c601f0d7-b486-4f6f-9213-25e40e74b2d4 None admin] RBAC: Authorization granted {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:17:29.572601 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fab3a3ae-1e63-4a97-99fb-703f2ed90913 None admin] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:17:29.587781 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: [pid: 80245|app: 0|req: 374/742] 10.222.0.47 () {56 vars in 1149 bytes} [Thu Jul 21 08:17:29 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 08:17:29.588719 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c601f0d7-b486-4f6f-9213-25e40e74b2d4 None admin] Authenticating user token {{(pid=80245) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:17:29.607051 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG dbcounter [-] [80244] Writing DB stats keystone:SELECT=100 {{(pid=80244) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:17:29.617765 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aa570f3-70d4-4a85-bc01-23dcdb73dffb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:17:29.618677 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aa570f3-70d4-4a85-bc01-23dcdb73dffb None admin] REQUEST_METHOD: `GET` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:17:29.619066 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aa570f3-70d4-4a85-bc01-23dcdb73dffb None admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:17:29.619367 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aa570f3-70d4-4a85-bc01-23dcdb73dffb None admin] PATH_INFO: `/v3/users/swift` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:17:29.627265 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7aa570f3-70d4-4a85-bc01-23dcdb73dffb None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:17:29.629466 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7aa570f3-70d4-4a85-bc01-23dcdb73dffb None admin] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:17:29.645028 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application [None req-7aa570f3-70d4-4a85-bc01-23dcdb73dffb None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Jul 21 08:17:29.645028 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 08:17:29.645028 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 08:17:29.645028 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 08:17:29.645028 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 08:17:29.645028 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 08:17:29.645028 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 08:17:29.645028 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 08:17:29.645028 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 08:17:29.645028 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 08:17:29.645028 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 08:17:29.645028 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 08:17:29.645028 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 21 08:17:29.645028 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 21 08:17:29.645028 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 21 08:17:29.645028 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 21 08:17:29.646479 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 08:17:29.646479 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 08:17:29.646479 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 21 08:17:29.646479 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 21 08:17:29.646479 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 21 08:17:29.646479 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 21 08:17:29.646479 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 08:17:29.646479 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 08:17:29.646479 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 08:17:29.646479 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 08:17:29.646479 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 08:17:29.646479 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application with Lock( Jul 21 08:17:29.646479 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 08:17:29.646479 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self._enter() Jul 21 08:17:29.646479 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 08:17:29.646479 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 08:17:29.646479 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 08:17:29.646479 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self.creator() Jul 21 08:17:29.648073 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 08:17:29.648073 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application created_value = creator( Jul 21 08:17:29.648073 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 21 08:17:29.648073 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 21 08:17:29.648073 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 21 08:17:29.648073 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 21 08:17:29.648073 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 21 08:17:29.648073 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 21 08:17:29.648073 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Jul 21 08:17:29.648073 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application Jul 21 08:17:29.648073 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 369/743] 10.222.0.47 () {56 vars in 1096 bytes} [Thu Jul 21 08:17:29 2022] GET /identity/v3/users/swift => generated 83 bytes in 208 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 08:17:29.648073 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef9c2015-bf33-48ce-ba48-d029f2dcfca1 None None] REQUEST_METHOD: `GET` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:17:29.648073 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef9c2015-bf33-48ce-ba48-d029f2dcfca1 None None] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:17:29.648782 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef9c2015-bf33-48ce-ba48-d029f2dcfca1 None None] PATH_INFO: `/` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:17:29.649526 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 370/744] 10.222.0.47 () {54 vars in 926 bytes} [Thu Jul 21 08:17:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 08:17:29.650009 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef9c2015-bf33-48ce-ba48-d029f2dcfca1 None None] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:17:29.711311 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c601f0d7-b486-4f6f-9213-25e40e74b2d4 None admin] Validating token access rules against request {{(pid=80245) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:17:29.715018 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG dbcounter [-] [80245] Writing DB stats keystone:SELECT=100 {{(pid=80245) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:17:29.751962 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8de146c3-c303-47d2-bb5d-b428352d8d2e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80245) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:17:29.752903 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8de146c3-c303-47d2-bb5d-b428352d8d2e None admin] REQUEST_METHOD: `GET` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:17:29.753188 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8de146c3-c303-47d2-bb5d-b428352d8d2e None admin] SCRIPT_NAME: `/identity` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:17:29.753523 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8de146c3-c303-47d2-bb5d-b428352d8d2e None admin] PATH_INFO: `/v3/users/nova` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:17:29.763275 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef9c2015-bf33-48ce-ba48-d029f2dcfca1 None None] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:17:29.764014 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8de146c3-c303-47d2-bb5d-b428352d8d2e None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:17:29.766734 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8de146c3-c303-47d2-bb5d-b428352d8d2e None admin] RBAC: Authorization granted {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:17:29.784012 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application [None req-8de146c3-c303-47d2-bb5d-b428352d8d2e None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jul 21 08:17:29.784012 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 08:17:29.784012 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 08:17:29.784012 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 08:17:29.784012 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 08:17:29.784012 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 08:17:29.784012 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 08:17:29.784012 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 08:17:29.784012 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 08:17:29.784012 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 08:17:29.784012 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 08:17:29.784012 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 08:17:29.784012 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 21 08:17:29.784012 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 21 08:17:29.784012 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 21 08:17:29.784012 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 21 08:17:29.785182 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 08:17:29.785182 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 08:17:29.785182 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 21 08:17:29.785182 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 21 08:17:29.785182 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 21 08:17:29.785182 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 21 08:17:29.785182 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 08:17:29.785182 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 08:17:29.785182 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 08:17:29.785182 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 08:17:29.785182 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 08:17:29.785182 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application with Lock( Jul 21 08:17:29.785182 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 08:17:29.785182 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application return self._enter() Jul 21 08:17:29.785182 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 08:17:29.785182 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 08:17:29.785182 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 08:17:29.785182 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application return self.creator() Jul 21 08:17:29.786372 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 08:17:29.786372 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application created_value = creator( Jul 21 08:17:29.786372 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 21 08:17:29.786372 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 21 08:17:29.786372 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 21 08:17:29.786372 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 21 08:17:29.786372 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 21 08:17:29.786372 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 21 08:17:29.786372 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jul 21 08:17:29.786372 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application Jul 21 08:17:29.790217 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: [pid: 80245|app: 0|req: 375/745] 10.222.0.47 () {56 vars in 1093 bytes} [Thu Jul 21 08:17:29 2022] GET /identity/v3/users/nova => generated 82 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 08:17:29.791002 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8de146c3-c303-47d2-bb5d-b428352d8d2e None admin] Authenticating user token {{(pid=80245) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:17:29.807896 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48b52afb-bbfe-4649-9d1c-4094267f174d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:17:29.808557 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48b52afb-bbfe-4649-9d1c-4094267f174d None admin] REQUEST_METHOD: `GET` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:17:29.808846 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48b52afb-bbfe-4649-9d1c-4094267f174d None admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:17:29.809131 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48b52afb-bbfe-4649-9d1c-4094267f174d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:17:29.868880 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG dbcounter [-] [80244] Writing DB stats keystone:SELECT=100 {{(pid=80244) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:17:29.883651 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48b52afb-bbfe-4649-9d1c-4094267f174d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:17:29.885111 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48b52afb-bbfe-4649-9d1c-4094267f174d None admin] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:17:29.906277 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8de146c3-c303-47d2-bb5d-b428352d8d2e None admin] Validating token access rules against request {{(pid=80245) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:17:29.946318 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52733d51-592c-4baf-a969-30f53fae087e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80245) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:17:29.947023 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52733d51-592c-4baf-a969-30f53fae087e None admin] REQUEST_METHOD: `GET` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:17:29.947367 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52733d51-592c-4baf-a969-30f53fae087e None admin] SCRIPT_NAME: `/identity` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:17:29.947670 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52733d51-592c-4baf-a969-30f53fae087e None admin] PATH_INFO: `/v3/users` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:17:29.948452 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52733d51-592c-4baf-a969-30f53fae087e None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80245) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 08:17:29.948766 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52733d51-592c-4baf-a969-30f53fae087e None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:17:29.950209 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52733d51-592c-4baf-a969-30f53fae087e None admin] RBAC: Authorization granted {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:17:29.975936 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: [pid: 80245|app: 0|req: 376/746] 10.222.0.47 () {56 vars in 1135 bytes} [Thu Jul 21 08:17:29 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 08:17:29.975936 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d68baa5e-3819-4060-998b-5917f36fce18 None None] REQUEST_METHOD: `POST` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:17:29.975936 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d68baa5e-3819-4060-998b-5917f36fce18 None None] SCRIPT_NAME: `/identity` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:17:29.975936 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d68baa5e-3819-4060-998b-5917f36fce18 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:17:29.990409 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.auth.core [None req-d68baa5e-3819-4060-998b-5917f36fce18 None None] MFA Rules not processed for user `2dce40b8d74c41ec94cc36e6ff05d412`. Rule list: `[]` (Enabled: `True`). {{(pid=80245) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 08:17:29.991652 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.fernet_utils [None req-d68baa5e-3819-4060-998b-5917f36fce18 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80245) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 08:17:30.001612 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 371/747] 10.222.0.47 () {58 vars in 1282 bytes} [Thu Jul 21 08:17:29 2022] GET /identity/v3/auth/tokens => generated 903 bytes in 347 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 21 08:17:30.001612 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48b52afb-bbfe-4649-9d1c-4094267f174d None admin] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:17:30.015996 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG dbcounter [-] [80245] Writing DB stats keystone:SELECT=100 {{(pid=80245) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:17:30.080877 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: [pid: 80245|app: 0|req: 377/748] 10.222.0.47 () {58 vars in 1023 bytes} [Thu Jul 21 08:17:29 2022] POST /identity/v3/auth/tokens => generated 903 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 21 08:17:30.082187 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d68baa5e-3819-4060-998b-5917f36fce18 None None] Authenticating user token {{(pid=80245) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:17:30.096579 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG dbcounter [-] [80244] Writing DB stats keystone:SELECT=100 {{(pid=80244) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:17:30.113715 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48b52afb-bbfe-4649-9d1c-4094267f174d None admin] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:17:30.166372 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40a5c6ee-a780-43f4-bfbf-94e64e94db3a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:17:30.167120 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40a5c6ee-a780-43f4-bfbf-94e64e94db3a None admin] REQUEST_METHOD: `GET` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:17:30.167430 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40a5c6ee-a780-43f4-bfbf-94e64e94db3a None admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:17:30.167707 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40a5c6ee-a780-43f4-bfbf-94e64e94db3a None admin] PATH_INFO: `/v3/users` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:17:30.168548 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40a5c6ee-a780-43f4-bfbf-94e64e94db3a None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80244) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 08:17:30.168854 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40a5c6ee-a780-43f4-bfbf-94e64e94db3a None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:17:30.170297 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40a5c6ee-a780-43f4-bfbf-94e64e94db3a None admin] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:17:30.190323 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 372/749] 10.222.0.47 () {56 vars in 1133 bytes} [Thu Jul 21 08:17:29 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 08:17:30.191339 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40a5c6ee-a780-43f4-bfbf-94e64e94db3a None admin] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:17:30.213613 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d68baa5e-3819-4060-998b-5917f36fce18 None None] Validating token access rules against request {{(pid=80245) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:17:30.260177 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2cd5134-53bd-44c4-af74-2e224ba2101b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80245) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:17:30.261097 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2cd5134-53bd-44c4-af74-2e224ba2101b None admin] REQUEST_METHOD: `GET` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:17:30.261793 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2cd5134-53bd-44c4-af74-2e224ba2101b None admin] SCRIPT_NAME: `/identity` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:17:30.262082 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2cd5134-53bd-44c4-af74-2e224ba2101b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:17:30.288958 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG dbcounter [-] [80245] Writing DB stats keystone:SELECT=100 {{(pid=80245) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:17:30.302722 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40a5c6ee-a780-43f4-bfbf-94e64e94db3a None admin] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:17:30.337396 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7577277-8737-4684-ada8-714a79ba7d75 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:17:30.337653 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2cd5134-53bd-44c4-af74-2e224ba2101b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:17:30.338186 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7577277-8737-4684-ada8-714a79ba7d75 None admin] REQUEST_METHOD: `GET` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:17:30.338464 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7577277-8737-4684-ada8-714a79ba7d75 None admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:17:30.338734 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7577277-8737-4684-ada8-714a79ba7d75 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:17:30.339163 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2cd5134-53bd-44c4-af74-2e224ba2101b None admin] RBAC: Authorization granted {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:17:30.350123 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7577277-8737-4684-ada8-714a79ba7d75 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:17:30.351584 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7577277-8737-4684-ada8-714a79ba7d75 None admin] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:17:30.359573 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG dbcounter [-] [80244] Writing DB stats keystone:SELECT=100 {{(pid=80244) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:17:30.372235 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application [None req-d7577277-8737-4684-ada8-714a79ba7d75 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 08:17:30.372235 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 08:17:30.372235 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 08:17:30.372235 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 08:17:30.372235 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 08:17:30.372235 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 08:17:30.372235 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 08:17:30.372235 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 08:17:30.372235 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 08:17:30.372235 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 08:17:30.372235 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 08:17:30.372235 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 08:17:30.372235 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 21 08:17:30.372235 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 21 08:17:30.372235 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 21 08:17:30.373135 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 21 08:17:30.373135 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 08:17:30.373135 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 08:17:30.373135 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 08:17:30.373135 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 08:17:30.373135 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 08:17:30.373135 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 08:17:30.373135 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 08:17:30.373135 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application with Lock( Jul 21 08:17:30.373135 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 08:17:30.373135 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self._enter() Jul 21 08:17:30.373135 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 08:17:30.373135 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 08:17:30.373135 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 08:17:30.373135 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self.creator() Jul 21 08:17:30.373135 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 08:17:30.373135 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application created_value = creator( Jul 21 08:17:30.374005 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 21 08:17:30.374005 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 21 08:17:30.374005 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 08:17:30.374005 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application Jul 21 08:17:30.381158 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 373/750] 10.222.0.47 () {56 vars in 1108 bytes} [Thu Jul 21 08:17:30 2022] GET /identity/v3/domains/Default => generated 87 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 08:17:30.381158 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7577277-8737-4684-ada8-714a79ba7d75 None admin] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:17:30.461827 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: [pid: 80245|app: 0|req: 378/751] 10.222.0.47 () {58 vars in 1282 bytes} [Thu Jul 21 08:17:30 2022] GET /identity/v3/auth/tokens => generated 903 bytes in 380 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 21 08:17:30.462932 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2cd5134-53bd-44c4-af74-2e224ba2101b None admin] Authenticating user token {{(pid=80245) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:17:30.494582 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7577277-8737-4684-ada8-714a79ba7d75 None admin] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:17:30.526711 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG dbcounter [-] [80245] Writing DB stats keystone:SELECT=100 {{(pid=80245) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:17:30.533474 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0368645-8dc3-45a1-980d-269188a963de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:17:30.534218 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0368645-8dc3-45a1-980d-269188a963de None admin] REQUEST_METHOD: `POST` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:17:30.534432 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0368645-8dc3-45a1-980d-269188a963de None admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:17:30.534705 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0368645-8dc3-45a1-980d-269188a963de None admin] PATH_INFO: `/v3/services` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:17:30.536134 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0368645-8dc3-45a1-980d-269188a963de None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:17:30.537508 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0368645-8dc3-45a1-980d-269188a963de None admin] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:17:30.558796 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 374/752] 10.222.0.47 () {60 vars in 1141 bytes} [Thu Jul 21 08:17:30 2022] POST /identity/v3/services => generated 240 bytes in 182 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 08:17:30.559670 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0368645-8dc3-45a1-980d-269188a963de None admin] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:17:30.618020 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2cd5134-53bd-44c4-af74-2e224ba2101b None admin] Validating token access rules against request {{(pid=80245) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:17:30.671963 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG dbcounter [-] [80244] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80244) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:17:30.681707 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa10d990-5f3e-45fb-bc29-ada7bbb11367 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80245) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:17:30.683749 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa10d990-5f3e-45fb-bc29-ada7bbb11367 None admin] REQUEST_METHOD: `GET` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:17:30.684269 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa10d990-5f3e-45fb-bc29-ada7bbb11367 None admin] SCRIPT_NAME: `/identity` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:17:30.684863 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa10d990-5f3e-45fb-bc29-ada7bbb11367 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:17:30.698176 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa10d990-5f3e-45fb-bc29-ada7bbb11367 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:17:30.700158 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa10d990-5f3e-45fb-bc29-ada7bbb11367 None admin] RBAC: Authorization granted {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:17:30.721056 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application [None req-aa10d990-5f3e-45fb-bc29-ada7bbb11367 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 08:17:30.721056 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 08:17:30.721056 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 08:17:30.721056 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 08:17:30.721056 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 08:17:30.721056 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 08:17:30.721056 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 08:17:30.721056 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 08:17:30.721056 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 08:17:30.721056 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 08:17:30.721056 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 08:17:30.721056 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 08:17:30.721056 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 21 08:17:30.721056 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 21 08:17:30.721056 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 21 08:17:30.723148 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 21 08:17:30.723148 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 08:17:30.723148 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 08:17:30.723148 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 08:17:30.723148 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 08:17:30.723148 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 08:17:30.723148 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 08:17:30.723148 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 08:17:30.723148 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application with Lock( Jul 21 08:17:30.723148 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 08:17:30.723148 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application return self._enter() Jul 21 08:17:30.723148 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 08:17:30.723148 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 08:17:30.723148 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 08:17:30.723148 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application return self.creator() Jul 21 08:17:30.723148 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 08:17:30.723148 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application created_value = creator( Jul 21 08:17:30.724285 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 21 08:17:30.724285 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 21 08:17:30.724285 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 08:17:30.724285 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application Jul 21 08:17:30.725696 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0368645-8dc3-45a1-980d-269188a963de None admin] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:17:30.726446 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: [pid: 80245|app: 0|req: 379/753] 10.222.0.47 () {56 vars in 1108 bytes} [Thu Jul 21 08:17:30 2022] GET /identity/v3/domains/Default => generated 87 bytes in 264 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 08:17:30.727429 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa10d990-5f3e-45fb-bc29-ada7bbb11367 None admin] Authenticating user token {{(pid=80245) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:17:30.770743 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa5666be-8f76-4166-9f43-638c8cbe74ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:17:30.772601 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa5666be-8f76-4166-9f43-638c8cbe74ca None admin] REQUEST_METHOD: `GET` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:17:30.772601 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa5666be-8f76-4166-9f43-638c8cbe74ca None admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:17:30.772601 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa5666be-8f76-4166-9f43-638c8cbe74ca None admin] PATH_INFO: `/v3/domains` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:17:30.772846 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa5666be-8f76-4166-9f43-638c8cbe74ca None admin] RBAC: Adding query filter params (name=Default) {{(pid=80244) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 08:17:30.773141 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa5666be-8f76-4166-9f43-638c8cbe74ca None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:17:30.774547 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa5666be-8f76-4166-9f43-638c8cbe74ca None admin] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:17:30.792496 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 375/754] 10.222.0.47 () {56 vars in 1109 bytes} [Thu Jul 21 08:17:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 229 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 08:17:30.793422 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa5666be-8f76-4166-9f43-638c8cbe74ca None admin] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:17:30.841983 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG dbcounter [-] [80245] Writing DB stats keystone:SELECT=100 {{(pid=80245) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:17:30.844974 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa10d990-5f3e-45fb-bc29-ada7bbb11367 None admin] Validating token access rules against request {{(pid=80245) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:17:30.895520 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cf82007-aa1a-4f1f-ab5e-a912ff504e80 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80245) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:17:30.896495 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cf82007-aa1a-4f1f-ab5e-a912ff504e80 None admin] REQUEST_METHOD: `GET` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:17:30.896849 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cf82007-aa1a-4f1f-ab5e-a912ff504e80 None admin] SCRIPT_NAME: `/identity` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:17:30.897192 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cf82007-aa1a-4f1f-ab5e-a912ff504e80 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:17:30.908970 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cf82007-aa1a-4f1f-ab5e-a912ff504e80 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:17:30.911097 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cf82007-aa1a-4f1f-ab5e-a912ff504e80 None admin] RBAC: Authorization granted {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:17:30.929304 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa5666be-8f76-4166-9f43-638c8cbe74ca None admin] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:17:30.929490 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application [None req-9cf82007-aa1a-4f1f-ab5e-a912ff504e80 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 08:17:30.929490 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 08:17:30.929490 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 08:17:30.929490 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 08:17:30.929490 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 08:17:30.929490 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 08:17:30.929490 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 08:17:30.929490 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 08:17:30.929490 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 08:17:30.929490 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 08:17:30.929490 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 08:17:30.929490 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 08:17:30.929490 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 21 08:17:30.929490 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 21 08:17:30.929490 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 21 08:17:30.930318 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 21 08:17:30.930318 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 08:17:30.930318 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 08:17:30.930318 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 08:17:30.930318 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 08:17:30.930318 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 08:17:30.930318 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 08:17:30.930318 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 08:17:30.930318 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application with Lock( Jul 21 08:17:30.930318 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 08:17:30.930318 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application return self._enter() Jul 21 08:17:30.930318 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 08:17:30.930318 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 08:17:30.930318 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 08:17:30.930318 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application return self.creator() Jul 21 08:17:30.930318 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 08:17:30.930318 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application created_value = creator( Jul 21 08:17:30.931225 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 21 08:17:30.931225 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 21 08:17:30.931225 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 08:17:30.931225 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application Jul 21 08:17:30.931225 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: [pid: 80245|app: 0|req: 380/755] 10.222.0.47 () {56 vars in 1108 bytes} [Thu Jul 21 08:17:30 2022] GET /identity/v3/domains/Default => generated 87 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 08:17:30.932195 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cf82007-aa1a-4f1f-ab5e-a912ff504e80 None admin] Authenticating user token {{(pid=80245) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:17:30.951956 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG dbcounter [-] [80244] Writing DB stats keystone:SELECT=100 {{(pid=80244) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:17:30.972647 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36760abf-2f3c-41c8-8617-39a7e8999a8b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:17:30.973898 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36760abf-2f3c-41c8-8617-39a7e8999a8b None admin] REQUEST_METHOD: `GET` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:17:30.974334 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36760abf-2f3c-41c8-8617-39a7e8999a8b None admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:17:30.974767 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36760abf-2f3c-41c8-8617-39a7e8999a8b None admin] PATH_INFO: `/v3/domains` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:17:30.975719 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36760abf-2f3c-41c8-8617-39a7e8999a8b None admin] RBAC: Adding query filter params (name=Default) {{(pid=80244) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 08:17:30.976177 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36760abf-2f3c-41c8-8617-39a7e8999a8b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:17:30.977745 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36760abf-2f3c-41c8-8617-39a7e8999a8b None admin] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:17:30.991103 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 376/756] 10.222.0.47 () {56 vars in 1109 bytes} [Thu Jul 21 08:17:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 08:17:30.992289 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36760abf-2f3c-41c8-8617-39a7e8999a8b None admin] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:17:31.042835 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cf82007-aa1a-4f1f-ab5e-a912ff504e80 None admin] Validating token access rules against request {{(pid=80245) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:17:31.079109 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dbfa19b-bac8-445d-bf63-33eb43f1c971 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80245) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:17:31.080106 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dbfa19b-bac8-445d-bf63-33eb43f1c971 None admin] REQUEST_METHOD: `GET` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:17:31.080483 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dbfa19b-bac8-445d-bf63-33eb43f1c971 None admin] SCRIPT_NAME: `/identity` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:17:31.080853 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dbfa19b-bac8-445d-bf63-33eb43f1c971 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:17:31.088674 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dbfa19b-bac8-445d-bf63-33eb43f1c971 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:17:31.090240 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dbfa19b-bac8-445d-bf63-33eb43f1c971 None admin] RBAC: Authorization granted {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:17:31.104104 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36760abf-2f3c-41c8-8617-39a7e8999a8b None admin] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:17:31.106523 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application [None req-7dbfa19b-bac8-445d-bf63-33eb43f1c971 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 21 08:17:31.106523 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 08:17:31.106523 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 08:17:31.106523 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 08:17:31.106523 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 08:17:31.106523 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 08:17:31.106523 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 08:17:31.106523 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 08:17:31.106523 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 08:17:31.106523 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 08:17:31.106523 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 08:17:31.106523 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 08:17:31.106523 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 21 08:17:31.106523 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 21 08:17:31.106523 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 21 08:17:31.107619 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 21 08:17:31.107619 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 08:17:31.107619 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 08:17:31.107619 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 08:17:31.107619 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 08:17:31.107619 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 08:17:31.107619 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 08:17:31.107619 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 08:17:31.107619 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application with Lock( Jul 21 08:17:31.107619 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 08:17:31.107619 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application return self._enter() Jul 21 08:17:31.107619 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 08:17:31.107619 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 08:17:31.107619 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 08:17:31.107619 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application return self.creator() Jul 21 08:17:31.107619 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 08:17:31.107619 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application created_value = creator( Jul 21 08:17:31.108540 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 21 08:17:31.108540 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 21 08:17:31.108540 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 21 08:17:31.108540 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 21 08:17:31.108540 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 21 08:17:31.108540 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 21 08:17:31.108540 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 21 08:17:31.108540 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application Jul 21 08:17:31.108540 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: [pid: 80245|app: 0|req: 381/757] 10.222.0.47 () {56 vars in 1111 bytes} [Thu Jul 21 08:17:30 2022] GET /identity/v3/projects/service => generated 88 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 08:17:31.110421 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dbfa19b-bac8-445d-bf63-33eb43f1c971 None admin] Authenticating user token {{(pid=80245) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:17:31.125868 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG dbcounter [-] [80245] Writing DB stats keystone:SELECT=100 {{(pid=80245) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:17:31.148027 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fe58c7e-fc4c-4cdf-b3c5-4723cd61c1c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:17:31.149097 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fe58c7e-fc4c-4cdf-b3c5-4723cd61c1c6 None admin] REQUEST_METHOD: `GET` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:17:31.149564 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fe58c7e-fc4c-4cdf-b3c5-4723cd61c1c6 None admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:17:31.150057 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fe58c7e-fc4c-4cdf-b3c5-4723cd61c1c6 None admin] PATH_INFO: `/v3/domains` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:17:31.150977 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fe58c7e-fc4c-4cdf-b3c5-4723cd61c1c6 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80244) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 08:17:31.151506 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fe58c7e-fc4c-4cdf-b3c5-4723cd61c1c6 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:17:31.153077 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fe58c7e-fc4c-4cdf-b3c5-4723cd61c1c6 None admin] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:17:31.167285 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 377/758] 10.222.0.47 () {56 vars in 1109 bytes} [Thu Jul 21 08:17:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 08:17:31.168451 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fe58c7e-fc4c-4cdf-b3c5-4723cd61c1c6 None admin] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:17:31.212155 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG dbcounter [-] [80244] Writing DB stats keystone:SELECT=100 {{(pid=80244) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:17:31.219971 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dbfa19b-bac8-445d-bf63-33eb43f1c971 None admin] Validating token access rules against request {{(pid=80245) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:17:31.257286 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-362de014-e9f7-4bf4-85ef-7965b2cab6d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80245) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:17:31.258567 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-362de014-e9f7-4bf4-85ef-7965b2cab6d6 None admin] REQUEST_METHOD: `GET` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:17:31.259016 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-362de014-e9f7-4bf4-85ef-7965b2cab6d6 None admin] SCRIPT_NAME: `/identity` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:17:31.259419 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-362de014-e9f7-4bf4-85ef-7965b2cab6d6 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:17:31.266885 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-362de014-e9f7-4bf4-85ef-7965b2cab6d6 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:17:31.268477 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-362de014-e9f7-4bf4-85ef-7965b2cab6d6 None admin] RBAC: Authorization granted {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:17:31.281268 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fe58c7e-fc4c-4cdf-b3c5-4723cd61c1c6 None admin] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:17:31.283296 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application [None req-362de014-e9f7-4bf4-85ef-7965b2cab6d6 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 21 08:17:31.283296 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 08:17:31.283296 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 08:17:31.283296 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 08:17:31.283296 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 08:17:31.283296 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 08:17:31.283296 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 08:17:31.283296 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 08:17:31.283296 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 08:17:31.283296 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 08:17:31.283296 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 08:17:31.283296 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 08:17:31.283296 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 21 08:17:31.283296 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 21 08:17:31.283296 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 21 08:17:31.284196 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 21 08:17:31.284196 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 08:17:31.284196 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 08:17:31.284196 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 08:17:31.284196 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 08:17:31.284196 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 08:17:31.284196 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 08:17:31.284196 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 08:17:31.284196 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application with Lock( Jul 21 08:17:31.284196 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 08:17:31.284196 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application return self._enter() Jul 21 08:17:31.284196 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 08:17:31.284196 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 08:17:31.284196 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 08:17:31.284196 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application return self.creator() Jul 21 08:17:31.284196 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 08:17:31.284196 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application created_value = creator( Jul 21 08:17:31.285634 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 21 08:17:31.285634 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 21 08:17:31.285634 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 21 08:17:31.285634 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 21 08:17:31.285634 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 21 08:17:31.285634 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 21 08:17:31.285634 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 21 08:17:31.285634 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application Jul 21 08:17:31.285634 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: [pid: 80245|app: 0|req: 382/759] 10.222.0.47 () {56 vars in 1111 bytes} [Thu Jul 21 08:17:31 2022] GET /identity/v3/projects/service => generated 88 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 08:17:31.286707 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-362de014-e9f7-4bf4-85ef-7965b2cab6d6 None admin] Authenticating user token {{(pid=80245) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:17:31.323092 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6c2debf-5951-4187-a8d1-3bf63427b863 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:17:31.324097 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6c2debf-5951-4187-a8d1-3bf63427b863 None admin] REQUEST_METHOD: `GET` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:17:31.324806 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6c2debf-5951-4187-a8d1-3bf63427b863 None admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:17:31.325269 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6c2debf-5951-4187-a8d1-3bf63427b863 None admin] PATH_INFO: `/v3/projects` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:17:31.326300 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6c2debf-5951-4187-a8d1-3bf63427b863 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80244) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 08:17:31.326804 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6c2debf-5951-4187-a8d1-3bf63427b863 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:17:31.328541 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6c2debf-5951-4187-a8d1-3bf63427b863 None admin] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:17:31.342467 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 378/760] 10.222.0.47 () {56 vars in 1148 bytes} [Thu Jul 21 08:17:31 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 08:17:31.343677 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6c2debf-5951-4187-a8d1-3bf63427b863 None admin] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:17:31.362958 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG dbcounter [-] [80245] Writing DB stats keystone:SELECT=100 {{(pid=80245) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:17:31.398546 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-362de014-e9f7-4bf4-85ef-7965b2cab6d6 None admin] Validating token access rules against request {{(pid=80245) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:17:31.434503 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c23eb4c-7d94-46b8-bed6-9c118494b787 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80245) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:17:31.435319 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c23eb4c-7d94-46b8-bed6-9c118494b787 None admin] REQUEST_METHOD: `GET` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:17:31.435731 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c23eb4c-7d94-46b8-bed6-9c118494b787 None admin] SCRIPT_NAME: `/identity` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:17:31.436261 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c23eb4c-7d94-46b8-bed6-9c118494b787 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:17:31.444238 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c23eb4c-7d94-46b8-bed6-9c118494b787 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:17:31.445705 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c23eb4c-7d94-46b8-bed6-9c118494b787 None admin] RBAC: Authorization granted {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:17:31.456754 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG dbcounter [-] [80244] Writing DB stats keystone:SELECT=100 {{(pid=80244) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:17:31.461531 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application [None req-4c23eb4c-7d94-46b8-bed6-9c118494b787 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 21 08:17:31.461531 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 08:17:31.461531 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 08:17:31.461531 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 08:17:31.461531 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 08:17:31.461531 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 08:17:31.461531 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 08:17:31.461531 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 08:17:31.461531 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 08:17:31.461531 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 08:17:31.461531 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 08:17:31.461531 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 08:17:31.461531 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 21 08:17:31.461531 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 21 08:17:31.461531 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 21 08:17:31.462421 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 21 08:17:31.462421 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 08:17:31.462421 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 08:17:31.462421 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 08:17:31.462421 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 08:17:31.462421 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 08:17:31.462421 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 08:17:31.462421 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 08:17:31.462421 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application with Lock( Jul 21 08:17:31.462421 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 08:17:31.462421 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application return self._enter() Jul 21 08:17:31.462421 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 08:17:31.462421 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 08:17:31.462421 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 08:17:31.462421 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application return self.creator() Jul 21 08:17:31.462421 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 08:17:31.462421 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application created_value = creator( Jul 21 08:17:31.463729 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 21 08:17:31.463729 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 21 08:17:31.463729 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 21 08:17:31.463729 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 21 08:17:31.463729 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 21 08:17:31.463729 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 21 08:17:31.463729 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 21 08:17:31.463729 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application Jul 21 08:17:31.463729 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: [pid: 80245|app: 0|req: 383/761] 10.222.0.47 () {56 vars in 1111 bytes} [Thu Jul 21 08:17:31 2022] GET /identity/v3/projects/service => generated 88 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 08:17:31.464264 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c23eb4c-7d94-46b8-bed6-9c118494b787 None admin] Authenticating user token {{(pid=80245) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:17:31.466251 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6c2debf-5951-4187-a8d1-3bf63427b863 None admin] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:17:31.508547 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88ce175d-9e91-4559-929e-de6779ff7991 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:17:31.509566 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88ce175d-9e91-4559-929e-de6779ff7991 None admin] REQUEST_METHOD: `GET` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:17:31.510068 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88ce175d-9e91-4559-929e-de6779ff7991 None admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:17:31.510602 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88ce175d-9e91-4559-929e-de6779ff7991 None admin] PATH_INFO: `/v3/projects` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:17:31.511693 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88ce175d-9e91-4559-929e-de6779ff7991 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80244) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 08:17:31.512208 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88ce175d-9e91-4559-929e-de6779ff7991 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:17:31.513901 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88ce175d-9e91-4559-929e-de6779ff7991 None admin] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:17:31.527506 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 379/762] 10.222.0.47 () {56 vars in 1148 bytes} [Thu Jul 21 08:17:31 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 08:17:31.528654 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88ce175d-9e91-4559-929e-de6779ff7991 None admin] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:17:31.585922 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c23eb4c-7d94-46b8-bed6-9c118494b787 None admin] Validating token access rules against request {{(pid=80245) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:17:31.619997 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG dbcounter [-] [80245] Writing DB stats keystone:SELECT=100 {{(pid=80245) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:17:31.626497 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f128c61-a94f-4c3d-b437-35e641f907ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80245) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:17:31.627184 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f128c61-a94f-4c3d-b437-35e641f907ef None admin] REQUEST_METHOD: `GET` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:17:31.627494 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f128c61-a94f-4c3d-b437-35e641f907ef None admin] SCRIPT_NAME: `/identity` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:17:31.627768 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f128c61-a94f-4c3d-b437-35e641f907ef None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:17:31.628617 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f128c61-a94f-4c3d-b437-35e641f907ef None admin] RBAC: Adding query filter params (role.id=50d1ec7f947f42acb65c57e07f2269d6, scope.project.id=4faef13aa50e4e36b6c265fb1398fa14, user.id=2ef7f1e91e554a1f88eb6a63e60a475f) {{(pid=80245) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 08:17:31.629463 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f128c61-a94f-4c3d-b437-35e641f907ef None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:17:31.631076 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f128c61-a94f-4c3d-b437-35e641f907ef None admin] RBAC: Authorization granted {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:17:31.647698 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: [pid: 80245|app: 0|req: 384/763] 10.222.0.47 () {56 vars in 1374 bytes} [Thu Jul 21 08:17:31 2022] GET /identity/v3/role_assignments?user.id=2ef7f1e91e554a1f88eb6a63e60a475f&scope.project.id=4faef13aa50e4e36b6c265fb1398fa14&role.id=50d1ec7f947f42acb65c57e07f2269d6 => generated 259 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 08:17:31.647698 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f128c61-a94f-4c3d-b437-35e641f907ef None admin] Authenticating user token {{(pid=80245) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:17:31.777638 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88ce175d-9e91-4559-929e-de6779ff7991 None admin] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:17:31.785628 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f128c61-a94f-4c3d-b437-35e641f907ef None admin] Validating token access rules against request {{(pid=80245) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:17:31.825681 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6179fd34-5788-4f22-b2b9-558c125be20d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:17:31.826356 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6179fd34-5788-4f22-b2b9-558c125be20d None admin] REQUEST_METHOD: `GET` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:17:31.826631 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6179fd34-5788-4f22-b2b9-558c125be20d None admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:17:31.826902 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6179fd34-5788-4f22-b2b9-558c125be20d None admin] PATH_INFO: `/v3/projects` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:17:31.827719 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6179fd34-5788-4f22-b2b9-558c125be20d None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80244) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 08:17:31.828024 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6179fd34-5788-4f22-b2b9-558c125be20d None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:17:31.829440 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6179fd34-5788-4f22-b2b9-558c125be20d None admin] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:17:31.829928 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efdcb556-50f3-477b-a005-6c4e40fc036b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80245) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:17:31.830829 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efdcb556-50f3-477b-a005-6c4e40fc036b None admin] REQUEST_METHOD: `PUT` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:17:31.831121 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efdcb556-50f3-477b-a005-6c4e40fc036b None admin] SCRIPT_NAME: `/identity` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:17:31.831434 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efdcb556-50f3-477b-a005-6c4e40fc036b None admin] PATH_INFO: `/v3/projects/4faef13aa50e4e36b6c265fb1398fa14/users/ac2b5b99a36442148fe743269054e991/roles/e29870f6e9e2411a9db0d5bf45f32ca5` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:17:31.834111 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efdcb556-50f3-477b-a005-6c4e40fc036b None admin] RBAC: Authorizing `identity:create_grant(project_id=4faef13aa50e4e36b6c265fb1398fa14, user_id=ac2b5b99a36442148fe743269054e991, role_id=e29870f6e9e2411a9db0d5bf45f32ca5)` {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:17:31.835607 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efdcb556-50f3-477b-a005-6c4e40fc036b None admin] RBAC: Authorization granted {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:17:31.843024 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 380/764] 10.222.0.47 () {56 vars in 1148 bytes} [Thu Jul 21 08:17:31 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 315 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 08:17:31.844630 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5a571ce-a36f-474a-a481-ceeb10522226 None None] REQUEST_METHOD: `GET` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:17:31.845097 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5a571ce-a36f-474a-a481-ceeb10522226 None None] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:17:31.845546 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5a571ce-a36f-474a-a481-ceeb10522226 None None] PATH_INFO: `/` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:17:31.847175 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 381/765] 10.222.0.47 () {54 vars in 926 bytes} [Thu Jul 21 08:17:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 08:17:31.853641 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abeef4c9-0848-459f-91b7-023efeef0e61 None None] REQUEST_METHOD: `POST` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:17:31.854610 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abeef4c9-0848-459f-91b7-023efeef0e61 None None] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:17:31.855154 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abeef4c9-0848-459f-91b7-023efeef0e61 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:17:31.860600 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: [pid: 80245|app: 0|req: 385/766] 10.222.0.47 () {58 vars in 1439 bytes} [Thu Jul 21 08:17:31 2022] PUT /identity/v3/projects/4faef13aa50e4e36b6c265fb1398fa14/users/ac2b5b99a36442148fe743269054e991/roles/e29870f6e9e2411a9db0d5bf45f32ca5 => generated 0 bytes in 222 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 08:17:31.861796 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efdcb556-50f3-477b-a005-6c4e40fc036b None admin] Authenticating user token {{(pid=80245) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:17:31.881440 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.auth.core [None req-abeef4c9-0848-459f-91b7-023efeef0e61 None None] MFA Rules not processed for user `2dce40b8d74c41ec94cc36e6ff05d412`. Rule list: `[]` (Enabled: `True`). {{(pid=80244) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 08:17:31.884479 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.fernet_utils [None req-abeef4c9-0848-459f-91b7-023efeef0e61 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80244) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 08:17:31.911772 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG dbcounter [-] [80244] Writing DB stats keystone:SELECT=100 {{(pid=80244) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:17:31.971284 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG dbcounter [-] [80245] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80245) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:17:31.999113 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 382/767] 10.222.0.47 () {58 vars in 1023 bytes} [Thu Jul 21 08:17:31 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 147 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 21 08:17:32.022658 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abeef4c9-0848-459f-91b7-023efeef0e61 None None] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:17:32.068606 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efdcb556-50f3-477b-a005-6c4e40fc036b None admin] Validating token access rules against request {{(pid=80245) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:17:32.126483 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abeef4c9-0848-459f-91b7-023efeef0e61 None None] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:17:32.130367 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f25a8b1-daf6-460b-a3cd-8c95ad86a50f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80245) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:17:32.131136 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f25a8b1-daf6-460b-a3cd-8c95ad86a50f None admin] REQUEST_METHOD: `GET` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:17:32.131453 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f25a8b1-daf6-460b-a3cd-8c95ad86a50f None admin] SCRIPT_NAME: `/identity` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:17:32.131734 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f25a8b1-daf6-460b-a3cd-8c95ad86a50f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:17:32.132578 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f25a8b1-daf6-460b-a3cd-8c95ad86a50f None admin] RBAC: Adding query filter params (role.id=e29870f6e9e2411a9db0d5bf45f32ca5, scope.project.id=4faef13aa50e4e36b6c265fb1398fa14, user.id=302351411fc84248a6b166b25d21609f) {{(pid=80245) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 08:17:32.132892 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f25a8b1-daf6-460b-a3cd-8c95ad86a50f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:17:32.134605 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f25a8b1-daf6-460b-a3cd-8c95ad86a50f None admin] RBAC: Authorization granted {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:17:32.150942 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: [pid: 80245|app: 0|req: 386/768] 10.222.0.47 () {56 vars in 1374 bytes} [Thu Jul 21 08:17:31 2022] GET /identity/v3/role_assignments?user.id=302351411fc84248a6b166b25d21609f&scope.project.id=4faef13aa50e4e36b6c265fb1398fa14&role.id=e29870f6e9e2411a9db0d5bf45f32ca5 => generated 259 bytes in 290 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 08:17:32.170810 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e228a79c-f815-49d7-951b-6d0567737650 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:17:32.172116 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e228a79c-f815-49d7-951b-6d0567737650 None admin] REQUEST_METHOD: `GET` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:17:32.172604 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e228a79c-f815-49d7-951b-6d0567737650 None admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:17:32.173075 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e228a79c-f815-49d7-951b-6d0567737650 None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:17:32.173950 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e228a79c-f815-49d7-951b-6d0567737650 None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:17:32.176302 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e228a79c-f815-49d7-951b-6d0567737650 None admin] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:17:32.189613 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application [None req-e228a79c-f815-49d7-951b-6d0567737650 None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Jul 21 08:17:32.189613 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 08:17:32.189613 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 21 08:17:32.189613 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 21 08:17:32.189613 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 21 08:17:32.189613 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 21 08:17:32.189613 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 21 08:17:32.189613 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 21 08:17:32.189613 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Jul 21 08:17:32.189613 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application Jul 21 08:17:32.189613 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 21 08:17:32.189613 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application Jul 21 08:17:32.189613 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 08:17:32.189613 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 08:17:32.189613 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 08:17:32.189613 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 08:17:32.190559 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 08:17:32.190559 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 08:17:32.190559 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 08:17:32.190559 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 08:17:32.190559 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 08:17:32.190559 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 08:17:32.190559 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 08:17:32.190559 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 21 08:17:32.190559 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 21 08:17:32.190559 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 21 08:17:32.190559 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 21 08:17:32.190559 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 08:17:32.190559 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 08:17:32.190559 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 08:17:32.190559 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 08:17:32.190559 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 08:17:32.192066 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 08:17:32.192066 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 08:17:32.192066 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application with Lock( Jul 21 08:17:32.192066 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 08:17:32.192066 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self._enter() Jul 21 08:17:32.192066 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 08:17:32.192066 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 08:17:32.192066 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 08:17:32.192066 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self.creator() Jul 21 08:17:32.192066 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 08:17:32.192066 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application created_value = creator( Jul 21 08:17:32.192066 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 21 08:17:32.192066 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 21 08:17:32.192066 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Jul 21 08:17:32.192066 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application Jul 21 08:17:32.195082 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 383/769] 10.222.0.47 () {56 vars in 1129 bytes} [Thu Jul 21 08:17:32 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 08:17:32.203823 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f25a8b1-daf6-460b-a3cd-8c95ad86a50f None admin] Authenticating user token {{(pid=80245) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:17:32.372601 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f25a8b1-daf6-460b-a3cd-8c95ad86a50f None admin] Validating token access rules against request {{(pid=80245) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:17:32.397131 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG dbcounter [-] [80245] Writing DB stats keystone:SELECT=100 {{(pid=80245) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:17:32.421785 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9219188e-9081-46e8-87ca-fa98f9098091 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80245) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:17:32.422473 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9219188e-9081-46e8-87ca-fa98f9098091 None admin] REQUEST_METHOD: `GET` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:17:32.422753 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9219188e-9081-46e8-87ca-fa98f9098091 None admin] SCRIPT_NAME: `/identity` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:17:32.423025 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9219188e-9081-46e8-87ca-fa98f9098091 None admin] PATH_INFO: `/v3/services` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:17:32.423753 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9219188e-9081-46e8-87ca-fa98f9098091 None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=80245) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 08:17:32.424057 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9219188e-9081-46e8-87ca-fa98f9098091 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:17:32.425417 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9219188e-9081-46e8-87ca-fa98f9098091 None admin] RBAC: Authorization granted {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:17:32.435781 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: [pid: 80245|app: 0|req: 387/770] 10.222.0.47 () {56 vars in 1124 bytes} [Thu Jul 21 08:17:32 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 235 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 08:17:32.441813 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e228a79c-f815-49d7-951b-6d0567737650 None admin] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:17:32.477705 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG dbcounter [-] [80244] Writing DB stats keystone:SELECT=100 {{(pid=80244) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:17:32.632663 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e228a79c-f815-49d7-951b-6d0567737650 None admin] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:17:32.674936 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad00aeaf-47af-4d80-b3fe-ec53ad29e996 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:17:32.676935 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad00aeaf-47af-4d80-b3fe-ec53ad29e996 None admin] REQUEST_METHOD: `GET` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:17:32.677439 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad00aeaf-47af-4d80-b3fe-ec53ad29e996 None admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:17:32.677909 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad00aeaf-47af-4d80-b3fe-ec53ad29e996 None admin] PATH_INFO: `/v3/services` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:17:32.680296 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad00aeaf-47af-4d80-b3fe-ec53ad29e996 None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=80244) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 08:17:32.680831 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad00aeaf-47af-4d80-b3fe-ec53ad29e996 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:17:32.682490 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad00aeaf-47af-4d80-b3fe-ec53ad29e996 None admin] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:17:32.688467 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 384/771] 10.222.0.47 () {56 vars in 1124 bytes} [Thu Jul 21 08:17:32 2022] GET /identity/v3/services?type=block-storage => generated 367 bytes in 248 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 08:17:32.700463 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9219188e-9081-46e8-87ca-fa98f9098091 None admin] Authenticating user token {{(pid=80245) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:17:32.854408 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9219188e-9081-46e8-87ca-fa98f9098091 None admin] Validating token access rules against request {{(pid=80245) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:17:32.906934 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dc64964-abce-4582-bfed-8db0921ac5e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80245) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:17:32.907700 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dc64964-abce-4582-bfed-8db0921ac5e0 None admin] REQUEST_METHOD: `GET` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:17:32.907993 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dc64964-abce-4582-bfed-8db0921ac5e0 None admin] SCRIPT_NAME: `/identity` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:17:32.908270 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dc64964-abce-4582-bfed-8db0921ac5e0 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:17:32.909076 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dc64964-abce-4582-bfed-8db0921ac5e0 None admin] RBAC: Adding query filter params (interface=public, service_id=05de249778d5423b9377cc3f8061b894, region_id=RegionOne) {{(pid=80245) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 08:17:32.909391 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dc64964-abce-4582-bfed-8db0921ac5e0 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:17:32.910965 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dc64964-abce-4582-bfed-8db0921ac5e0 None admin] RBAC: Authorization granted {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:17:32.919496 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: [pid: 80245|app: 0|req: 388/772] 10.222.0.47 () {56 vars in 1251 bytes} [Thu Jul 21 08:17:32 2022] GET /identity/v3/endpoints?service_id=05de249778d5423b9377cc3f8061b894&interface=public®ion_id=RegionOne => generated 194 bytes in 219 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 08:17:32.924709 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad00aeaf-47af-4d80-b3fe-ec53ad29e996 None admin] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:17:33.039114 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG dbcounter [-] [80244] Writing DB stats keystone:SELECT=100 {{(pid=80244) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:17:33.071385 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad00aeaf-47af-4d80-b3fe-ec53ad29e996 None admin] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:17:33.100192 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a233b10-f976-42d2-897c-2ecc5a568a08 None None] REQUEST_METHOD: `GET` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:17:33.100584 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a233b10-f976-42d2-897c-2ecc5a568a08 None None] SCRIPT_NAME: `/identity` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:17:33.100866 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a233b10-f976-42d2-897c-2ecc5a568a08 None None] PATH_INFO: `/` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:17:33.102525 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: [pid: 80245|app: 0|req: 389/773] 10.222.0.47 () {54 vars in 926 bytes} [Thu Jul 21 08:17:33 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 08:17:33.114118 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8df134ae-8692-4da5-8316-e6876ed5ca20 None None] REQUEST_METHOD: `POST` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:17:33.114468 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8df134ae-8692-4da5-8316-e6876ed5ca20 None None] SCRIPT_NAME: `/identity` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:17:33.114750 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8df134ae-8692-4da5-8316-e6876ed5ca20 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:17:33.128060 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02bc34ff-6c02-43a2-aac6-fdf3db9247af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:17:33.128783 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02bc34ff-6c02-43a2-aac6-fdf3db9247af None admin] REQUEST_METHOD: `GET` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:17:33.129071 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02bc34ff-6c02-43a2-aac6-fdf3db9247af None admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:17:33.129347 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02bc34ff-6c02-43a2-aac6-fdf3db9247af None admin] PATH_INFO: `/v3/services` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:17:33.130032 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02bc34ff-6c02-43a2-aac6-fdf3db9247af None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:17:33.131434 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02bc34ff-6c02-43a2-aac6-fdf3db9247af None admin] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:17:33.133676 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.auth.core [None req-8df134ae-8692-4da5-8316-e6876ed5ca20 None None] MFA Rules not processed for user `2dce40b8d74c41ec94cc36e6ff05d412`. Rule list: `[]` (Enabled: `True`). {{(pid=80245) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 08:17:33.135076 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.fernet_utils [None req-8df134ae-8692-4da5-8316-e6876ed5ca20 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80245) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 08:17:33.143363 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 385/774] 10.222.0.47 () {56 vars in 1087 bytes} [Thu Jul 21 08:17:32 2022] GET /identity/v3/services => generated 772 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 08:17:33.218968 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG dbcounter [-] [80245] Writing DB stats keystone:SELECT=100 {{(pid=80245) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:17:33.257429 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: [pid: 80245|app: 0|req: 390/775] 10.222.0.47 () {58 vars in 1023 bytes} [Thu Jul 21 08:17:33 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 145 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 21 08:17:33.290096 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02bc34ff-6c02-43a2-aac6-fdf3db9247af None admin] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:17:33.409600 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02bc34ff-6c02-43a2-aac6-fdf3db9247af None admin] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:17:33.450625 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee9f5229-80b2-4bde-8d8d-d610b1d46f00 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:17:33.451492 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee9f5229-80b2-4bde-8d8d-d610b1d46f00 None admin] REQUEST_METHOD: `GET` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:17:33.451735 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee9f5229-80b2-4bde-8d8d-d610b1d46f00 None admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:17:33.452083 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee9f5229-80b2-4bde-8d8d-d610b1d46f00 None admin] PATH_INFO: `/v3/services/network` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:17:33.452739 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee9f5229-80b2-4bde-8d8d-d610b1d46f00 None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:17:33.454130 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee9f5229-80b2-4bde-8d8d-d610b1d46f00 None admin] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:17:33.468044 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application [None req-ee9f5229-80b2-4bde-8d8d-d610b1d46f00 None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Jul 21 08:17:33.468044 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 08:17:33.468044 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 21 08:17:33.468044 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 21 08:17:33.468044 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 21 08:17:33.468044 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 21 08:17:33.468044 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 21 08:17:33.468044 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 21 08:17:33.468044 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Jul 21 08:17:33.468044 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application Jul 21 08:17:33.468044 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 21 08:17:33.468044 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application Jul 21 08:17:33.468044 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 08:17:33.468044 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 08:17:33.468044 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 08:17:33.468044 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 08:17:33.468044 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 08:17:33.469101 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 08:17:33.469101 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 08:17:33.469101 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 08:17:33.469101 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 08:17:33.469101 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 08:17:33.469101 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 08:17:33.469101 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 21 08:17:33.469101 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 21 08:17:33.469101 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 21 08:17:33.469101 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 21 08:17:33.469101 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 08:17:33.469101 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 08:17:33.469101 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 08:17:33.469101 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 08:17:33.469101 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 08:17:33.469101 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 08:17:33.470117 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 08:17:33.470117 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application with Lock( Jul 21 08:17:33.470117 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 08:17:33.470117 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self._enter() Jul 21 08:17:33.470117 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 08:17:33.470117 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 08:17:33.470117 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 08:17:33.470117 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self.creator() Jul 21 08:17:33.470117 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 08:17:33.470117 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application created_value = creator( Jul 21 08:17:33.470117 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 21 08:17:33.470117 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 21 08:17:33.470117 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Jul 21 08:17:33.470117 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application Jul 21 08:17:33.470117 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 386/776] 10.222.0.47 () {56 vars in 1111 bytes} [Thu Jul 21 08:17:33 2022] GET /identity/v3/services/network => generated 88 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 08:17:33.482177 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8df134ae-8692-4da5-8316-e6876ed5ca20 None None] Authenticating user token {{(pid=80245) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:17:33.626172 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8df134ae-8692-4da5-8316-e6876ed5ca20 None None] Validating token access rules against request {{(pid=80245) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:17:33.662608 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b3ccae5-4ecf-4878-a521-1759f13c7ed4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80245) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:17:33.663286 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b3ccae5-4ecf-4878-a521-1759f13c7ed4 None admin] REQUEST_METHOD: `GET` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:17:33.663589 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b3ccae5-4ecf-4878-a521-1759f13c7ed4 None admin] SCRIPT_NAME: `/identity` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:17:33.663862 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b3ccae5-4ecf-4878-a521-1759f13c7ed4 None admin] PATH_INFO: `/v3/services` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:17:33.664564 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b3ccae5-4ecf-4878-a521-1759f13c7ed4 None admin] RBAC: Adding query filter params (name=network) {{(pid=80245) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 08:17:33.664866 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b3ccae5-4ecf-4878-a521-1759f13c7ed4 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:17:33.666259 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b3ccae5-4ecf-4878-a521-1759f13c7ed4 None admin] RBAC: Authorization granted {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:17:33.671381 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: [pid: 80245|app: 0|req: 391/777] 10.222.0.47 () {56 vars in 1112 bytes} [Thu Jul 21 08:17:33 2022] GET /identity/v3/services?name=network => generated 124 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 08:17:33.676477 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee9f5229-80b2-4bde-8d8d-d610b1d46f00 None admin] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:17:33.680261 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG dbcounter [-] [80244] Writing DB stats keystone:SELECT=100 {{(pid=80244) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:17:33.819680 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee9f5229-80b2-4bde-8d8d-d610b1d46f00 None admin] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:17:33.861163 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6635a4f-cd75-47cd-89e0-3f717fd141d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:17:33.861832 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6635a4f-cd75-47cd-89e0-3f717fd141d8 None admin] REQUEST_METHOD: `GET` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:17:33.862116 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6635a4f-cd75-47cd-89e0-3f717fd141d8 None admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:17:33.862391 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6635a4f-cd75-47cd-89e0-3f717fd141d8 None admin] PATH_INFO: `/v3/services` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:17:33.863139 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6635a4f-cd75-47cd-89e0-3f717fd141d8 None admin] RBAC: Adding query filter params (type=network) {{(pid=80244) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 08:17:33.863473 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6635a4f-cd75-47cd-89e0-3f717fd141d8 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:17:33.864847 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6635a4f-cd75-47cd-89e0-3f717fd141d8 None admin] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:17:33.875314 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 387/778] 10.222.0.47 () {56 vars in 1112 bytes} [Thu Jul 21 08:17:33 2022] GET /identity/v3/services?type=network => generated 350 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 08:17:33.881683 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b3ccae5-4ecf-4878-a521-1759f13c7ed4 None admin] Authenticating user token {{(pid=80245) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:17:33.951054 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG dbcounter [-] [80245] Writing DB stats keystone:SELECT=100 {{(pid=80245) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:17:34.022115 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b3ccae5-4ecf-4878-a521-1759f13c7ed4 None admin] Validating token access rules against request {{(pid=80245) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:17:34.063146 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ce81a44-59f9-4250-96bf-aeeab3c68adf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80245) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:17:34.063860 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ce81a44-59f9-4250-96bf-aeeab3c68adf None admin] REQUEST_METHOD: `GET` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:17:34.064148 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ce81a44-59f9-4250-96bf-aeeab3c68adf None admin] SCRIPT_NAME: `/identity` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:17:34.064422 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ce81a44-59f9-4250-96bf-aeeab3c68adf None admin] PATH_INFO: `/v3/endpoints` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:17:34.065268 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ce81a44-59f9-4250-96bf-aeeab3c68adf None admin] RBAC: Adding query filter params (interface=public, service_id=99ad5c165f8743d6ae5effaef6254e9e, region_id=RegionOne) {{(pid=80245) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 08:17:34.065579 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ce81a44-59f9-4250-96bf-aeeab3c68adf None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:17:34.067835 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ce81a44-59f9-4250-96bf-aeeab3c68adf None admin] RBAC: Authorization granted {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:17:34.076080 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34667f5a-a9a4-4315-8d3e-d8c0dadf923e None None] REQUEST_METHOD: `GET` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:17:34.076548 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34667f5a-a9a4-4315-8d3e-d8c0dadf923e None None] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:17:34.076548 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34667f5a-a9a4-4315-8d3e-d8c0dadf923e None None] PATH_INFO: `/` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:17:34.078086 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 388/779] 10.222.0.47 () {54 vars in 926 bytes} [Thu Jul 21 08:17:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 08:17:34.078808 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: [pid: 80245|app: 0|req: 392/780] 10.222.0.47 () {56 vars in 1251 bytes} [Thu Jul 21 08:17:33 2022] GET /identity/v3/endpoints?service_id=99ad5c165f8743d6ae5effaef6254e9e&interface=public®ion_id=RegionOne => generated 194 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 08:17:34.085184 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14062dcf-15d8-48d7-ada8-f32f676a1141 None None] REQUEST_METHOD: `POST` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:17:34.085184 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14062dcf-15d8-48d7-ada8-f32f676a1141 None None] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:17:34.085401 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ce81a44-59f9-4250-96bf-aeeab3c68adf None admin] Authenticating user token {{(pid=80245) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:17:34.085478 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14062dcf-15d8-48d7-ada8-f32f676a1141 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:17:34.124453 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.auth.core [None req-14062dcf-15d8-48d7-ada8-f32f676a1141 None None] MFA Rules not processed for user `2dce40b8d74c41ec94cc36e6ff05d412`. Rule list: `[]` (Enabled: `True`). {{(pid=80244) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 08:17:34.125684 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.fernet_utils [None req-14062dcf-15d8-48d7-ada8-f32f676a1141 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80244) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 08:17:34.239758 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ce81a44-59f9-4250-96bf-aeeab3c68adf None admin] Validating token access rules against request {{(pid=80245) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:17:34.248412 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG dbcounter [-] [80244] Writing DB stats keystone:SELECT=100 {{(pid=80244) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:17:34.256086 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG dbcounter [-] [80245] Writing DB stats keystone:SELECT=100 {{(pid=80245) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:17:34.262945 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 389/781] 10.222.0.47 () {58 vars in 1023 bytes} [Thu Jul 21 08:17:34 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 179 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 21 08:17:34.288438 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1458a50e-5744-48ab-8d5f-6fc8e264d109 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80245) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:17:34.289164 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1458a50e-5744-48ab-8d5f-6fc8e264d109 None admin] REQUEST_METHOD: `GET` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:17:34.289446 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1458a50e-5744-48ab-8d5f-6fc8e264d109 None admin] SCRIPT_NAME: `/identity` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:17:34.289788 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1458a50e-5744-48ab-8d5f-6fc8e264d109 None admin] PATH_INFO: `/v3/services` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:17:34.290478 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1458a50e-5744-48ab-8d5f-6fc8e264d109 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:17:34.291900 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1458a50e-5744-48ab-8d5f-6fc8e264d109 None admin] RBAC: Authorization granted {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:17:34.297590 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: [pid: 80245|app: 0|req: 393/782] 10.222.0.47 () {56 vars in 1087 bytes} [Thu Jul 21 08:17:34 2022] GET /identity/v3/services => generated 772 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 21 08:17:34.298021 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14062dcf-15d8-48d7-ada8-f32f676a1141 None None] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:17:34.415302 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14062dcf-15d8-48d7-ada8-f32f676a1141 None None] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:17:34.459807 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f95ff6ae-6831-4c45-bd4e-e124bcfbcaf6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:17:34.461649 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f95ff6ae-6831-4c45-bd4e-e124bcfbcaf6 None admin] REQUEST_METHOD: `GET` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:17:34.462433 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f95ff6ae-6831-4c45-bd4e-e124bcfbcaf6 None admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:17:34.462941 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f95ff6ae-6831-4c45-bd4e-e124bcfbcaf6 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:17:34.481783 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f95ff6ae-6831-4c45-bd4e-e124bcfbcaf6 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 21 08:17:34.490267 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application [None req-f95ff6ae-6831-4c45-bd4e-e124bcfbcaf6 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 21 08:17:34.490267 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 08:17:34.490267 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 08:17:34.490267 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 08:17:34.490267 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 08:17:34.490267 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 08:17:34.490267 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 08:17:34.490267 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 08:17:34.490267 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 08:17:34.490267 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 08:17:34.490267 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 08:17:34.490267 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 08:17:34.490267 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 21 08:17:34.490267 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 21 08:17:34.490267 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 21 08:17:34.490267 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 21 08:17:34.491208 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 21 08:17:34.491208 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 21 08:17:34.491208 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 21 08:17:34.491208 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 21 08:17:34.491208 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 08:17:34.491208 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 08:17:34.491208 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 08:17:34.491208 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 08:17:34.491208 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 08:17:34.491208 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 08:17:34.491208 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 08:17:34.491208 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application with Lock( Jul 21 08:17:34.491208 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 08:17:34.491208 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self._enter() Jul 21 08:17:34.491208 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 08:17:34.491208 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 08:17:34.492069 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 08:17:34.492069 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self.creator() Jul 21 08:17:34.492069 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 08:17:34.492069 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application created_value = creator( Jul 21 08:17:34.492069 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 21 08:17:34.492069 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 21 08:17:34.492069 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 21 08:17:34.492069 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 21 08:17:34.492069 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 21 08:17:34.492069 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 21 08:17:34.492069 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 21 08:17:34.492069 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application Jul 21 08:17:34.496685 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 390/783] 10.222.0.47 () {56 vars in 1102 bytes} [Thu Jul 21 08:17:34 2022] GET /identity/v3/roles/service => generated 85 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 21 08:17:34.502813 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1458a50e-5744-48ab-8d5f-6fc8e264d109 None admin] Authenticating user token {{(pid=80245) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:17:34.623287 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1458a50e-5744-48ab-8d5f-6fc8e264d109 None admin] Validating token access rules against request {{(pid=80245) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:17:34.660892 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bc548fc-d862-47cb-a9ba-e4260634125d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80245) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:17:34.661839 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bc548fc-d862-47cb-a9ba-e4260634125d None admin] REQUEST_METHOD: `GET` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:17:34.662264 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bc548fc-d862-47cb-a9ba-e4260634125d None admin] SCRIPT_NAME: `/identity` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:17:34.662680 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bc548fc-d862-47cb-a9ba-e4260634125d None admin] PATH_INFO: `/v3/roles` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:17:34.663589 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bc548fc-d862-47cb-a9ba-e4260634125d None admin] RBAC: Adding query filter params (name=service) {{(pid=80245) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 08:17:34.664032 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bc548fc-d862-47cb-a9ba-e4260634125d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:17:34.665665 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bc548fc-d862-47cb-a9ba-e4260634125d None admin] RBAC: Authorization granted {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:17:34.675150 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: [pid: 80245|app: 0|req: 394/784] 10.222.0.47 () {56 vars in 1103 bytes} [Thu Jul 21 08:17:34 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 08:17:34.680390 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f95ff6ae-6831-4c45-bd4e-e124bcfbcaf6 None admin] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:17:34.686894 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6c928db-7a45-44b4-a3dc-e4f4c99da55f None None] REQUEST_METHOD: `GET` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:17:34.687180 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6c928db-7a45-44b4-a3dc-e4f4c99da55f None None] SCRIPT_NAME: `/identity` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:17:34.687487 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6c928db-7a45-44b4-a3dc-e4f4c99da55f None None] PATH_INFO: `/` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:17:34.689084 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: [pid: 80245|app: 0|req: 395/785] 10.222.0.47 () {54 vars in 926 bytes} [Thu Jul 21 08:17:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 21 08:17:34.694595 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1f53feb-0fe2-4892-adc4-3e2dc727cfbc None None] REQUEST_METHOD: `POST` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:17:34.694883 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1f53feb-0fe2-4892-adc4-3e2dc727cfbc None None] SCRIPT_NAME: `/identity` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:17:34.695157 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1f53feb-0fe2-4892-adc4-3e2dc727cfbc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:17:34.712535 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.auth.core [None req-d1f53feb-0fe2-4892-adc4-3e2dc727cfbc None None] MFA Rules not processed for user `2dce40b8d74c41ec94cc36e6ff05d412`. Rule list: `[]` (Enabled: `True`). {{(pid=80245) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 08:17:34.713718 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.fernet_utils [None req-d1f53feb-0fe2-4892-adc4-3e2dc727cfbc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80245) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 08:17:34.746332 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG dbcounter [-] [80245] Writing DB stats keystone:SELECT=100 {{(pid=80245) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:17:34.787705 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG dbcounter [-] [80244] Writing DB stats keystone:SELECT=100 {{(pid=80244) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:17:34.796164 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: [pid: 80245|app: 0|req: 396/786] 10.222.0.47 () {58 vars in 1023 bytes} [Thu Jul 21 08:17:34 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 21 08:17:34.797522 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b60a646-6bec-442b-8174-bad96370caba None None] REQUEST_METHOD: `GET` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:17:34.797806 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b60a646-6bec-442b-8174-bad96370caba None None] SCRIPT_NAME: `/identity` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:17:34.798083 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b60a646-6bec-442b-8174-bad96370caba None None] PATH_INFO: `/` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:17:34.807464 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: [pid: 80245|app: 0|req: 397/787] 10.222.0.47 () {54 vars in 926 bytes} [Thu Jul 21 08:17:34 2022] GET /identity => generated 270 bytes in 11 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 08:17:34.812890 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f95ff6ae-6831-4c45-bd4e-e124bcfbcaf6 None admin] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:17:34.817722 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b7b6228-ed09-447c-b930-201c5dcab1ad None None] REQUEST_METHOD: `POST` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:17:34.818006 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b7b6228-ed09-447c-b930-201c5dcab1ad None None] SCRIPT_NAME: `/identity` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:17:34.818282 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b7b6228-ed09-447c-b930-201c5dcab1ad None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:17:34.844183 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.auth.core [None req-8b7b6228-ed09-447c-b930-201c5dcab1ad None None] MFA Rules not processed for user `2dce40b8d74c41ec94cc36e6ff05d412`. Rule list: `[]` (Enabled: `True`). {{(pid=80245) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 08:17:34.845398 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.fernet_utils [None req-8b7b6228-ed09-447c-b930-201c5dcab1ad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80245) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 08:17:34.856406 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-174681ef-3a3f-402d-8283-95a53bd9b180 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:17:34.857621 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-174681ef-3a3f-402d-8283-95a53bd9b180 None admin] REQUEST_METHOD: `GET` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:17:34.857920 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-174681ef-3a3f-402d-8283-95a53bd9b180 None admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:17:34.858203 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-174681ef-3a3f-402d-8283-95a53bd9b180 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:17:34.873231 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-174681ef-3a3f-402d-8283-95a53bd9b180 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:17:34.875425 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-174681ef-3a3f-402d-8283-95a53bd9b180 None admin] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:17:34.900103 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application [None req-174681ef-3a3f-402d-8283-95a53bd9b180 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 08:17:34.900103 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 08:17:34.900103 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 08:17:34.900103 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 08:17:34.900103 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 08:17:34.900103 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 08:17:34.900103 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 08:17:34.900103 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 08:17:34.900103 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 08:17:34.900103 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 08:17:34.900103 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 08:17:34.900103 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 08:17:34.900103 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 21 08:17:34.900103 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 21 08:17:34.900103 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 21 08:17:34.901026 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 21 08:17:34.901026 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 08:17:34.901026 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 08:17:34.901026 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 08:17:34.901026 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 08:17:34.901026 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 08:17:34.901026 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 08:17:34.901026 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 08:17:34.901026 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application with Lock( Jul 21 08:17:34.901026 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 08:17:34.901026 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self._enter() Jul 21 08:17:34.901026 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 08:17:34.901026 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 08:17:34.901026 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 08:17:34.901026 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self.creator() Jul 21 08:17:34.901026 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 08:17:34.901026 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application created_value = creator( Jul 21 08:17:34.901893 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 21 08:17:34.901893 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 21 08:17:34.901893 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 08:17:34.901893 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application Jul 21 08:17:34.902109 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 391/788] 10.222.0.47 () {56 vars in 1108 bytes} [Thu Jul 21 08:17:34 2022] GET /identity/v3/domains/Default => generated 87 bytes in 222 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 21 08:17:34.903043 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-174681ef-3a3f-402d-8283-95a53bd9b180 None admin] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:17:34.927526 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: [pid: 80245|app: 0|req: 398/789] 10.222.0.47 () {58 vars in 1023 bytes} [Thu Jul 21 08:17:34 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 21 08:17:34.928339 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b7b6228-ed09-447c-b930-201c5dcab1ad None None] Authenticating user token {{(pid=80245) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:17:35.050292 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG dbcounter [-] [80245] Writing DB stats keystone:SELECT=100 {{(pid=80245) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:17:35.052964 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-174681ef-3a3f-402d-8283-95a53bd9b180 None admin] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:17:35.060723 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b7b6228-ed09-447c-b930-201c5dcab1ad None None] Validating token access rules against request {{(pid=80245) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:17:35.067689 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG dbcounter [-] [80244] Writing DB stats keystone:SELECT=100 {{(pid=80244) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:17:35.095848 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84b9908d-2e75-4cf6-aa38-e5eaa965cee1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:17:35.098969 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84b9908d-2e75-4cf6-aa38-e5eaa965cee1 None admin] REQUEST_METHOD: `GET` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:17:35.100335 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84b9908d-2e75-4cf6-aa38-e5eaa965cee1 None admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:17:35.101375 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84b9908d-2e75-4cf6-aa38-e5eaa965cee1 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:17:35.105197 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05245388-67cc-4d4f-9c34-5096bcc524dc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80245) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:17:35.106200 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05245388-67cc-4d4f-9c34-5096bcc524dc None admin] REQUEST_METHOD: `GET` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:17:35.106673 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05245388-67cc-4d4f-9c34-5096bcc524dc None admin] SCRIPT_NAME: `/identity` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:17:35.107087 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05245388-67cc-4d4f-9c34-5096bcc524dc None admin] PATH_INFO: `/v3/domains` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:17:35.107998 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05245388-67cc-4d4f-9c34-5096bcc524dc None admin] RBAC: Adding query filter params (name=Default) {{(pid=80245) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 08:17:35.108445 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05245388-67cc-4d4f-9c34-5096bcc524dc None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:17:35.109977 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05245388-67cc-4d4f-9c34-5096bcc524dc None admin] RBAC: Authorization granted {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:17:35.115395 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84b9908d-2e75-4cf6-aa38-e5eaa965cee1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 21 08:17:35.124713 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application [None req-84b9908d-2e75-4cf6-aa38-e5eaa965cee1 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 21 08:17:35.124713 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 08:17:35.124713 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 08:17:35.124713 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 08:17:35.124713 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 08:17:35.124713 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 08:17:35.124713 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 08:17:35.124713 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 08:17:35.124713 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 08:17:35.124713 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 08:17:35.124713 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 08:17:35.124713 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 08:17:35.124713 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 21 08:17:35.124713 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 21 08:17:35.124713 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 21 08:17:35.124713 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 21 08:17:35.126223 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 21 08:17:35.126223 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 21 08:17:35.126223 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 21 08:17:35.126223 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 21 08:17:35.126223 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 08:17:35.126223 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 08:17:35.126223 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 08:17:35.126223 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 08:17:35.126223 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 08:17:35.126223 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 08:17:35.126223 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 08:17:35.126223 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application with Lock( Jul 21 08:17:35.126223 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 08:17:35.126223 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self._enter() Jul 21 08:17:35.126223 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 08:17:35.126223 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 08:17:35.127985 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 08:17:35.127985 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self.creator() Jul 21 08:17:35.127985 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 08:17:35.127985 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application created_value = creator( Jul 21 08:17:35.127985 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 21 08:17:35.127985 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 21 08:17:35.127985 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 21 08:17:35.127985 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 21 08:17:35.127985 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 21 08:17:35.127985 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 21 08:17:35.127985 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 21 08:17:35.127985 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application Jul 21 08:17:35.131211 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: [pid: 80245|app: 0|req: 399/791] 10.222.0.47 () {56 vars in 1109 bytes} [Thu Jul 21 08:17:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 08:17:35.131361 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 392/791] 10.222.0.47 () {56 vars in 1096 bytes} [Thu Jul 21 08:17:34 2022] GET /identity/v3/roles/admin => generated 83 bytes in 224 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 08:17:35.132310 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84b9908d-2e75-4cf6-aa38-e5eaa965cee1 None admin] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:17:35.136715 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05245388-67cc-4d4f-9c34-5096bcc524dc None admin] Authenticating user token {{(pid=80245) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:17:35.239683 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84b9908d-2e75-4cf6-aa38-e5eaa965cee1 None admin] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:17:35.267785 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05245388-67cc-4d4f-9c34-5096bcc524dc None admin] Validating token access rules against request {{(pid=80245) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:17:35.287820 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afbd9a0e-5d38-4068-8054-72d99c75a0ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:17:35.288687 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afbd9a0e-5d38-4068-8054-72d99c75a0ca None admin] REQUEST_METHOD: `GET` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:17:35.288975 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afbd9a0e-5d38-4068-8054-72d99c75a0ca None admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:17:35.289299 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afbd9a0e-5d38-4068-8054-72d99c75a0ca None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:17:35.304609 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afbd9a0e-5d38-4068-8054-72d99c75a0ca None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 21 08:17:35.305965 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b6f90da-1ec5-4704-931f-6d0f811bf0cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80245) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:17:35.306876 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b6f90da-1ec5-4704-931f-6d0f811bf0cd None admin] REQUEST_METHOD: `GET` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:17:35.307457 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b6f90da-1ec5-4704-931f-6d0f811bf0cd None admin] SCRIPT_NAME: `/identity` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:17:35.307925 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b6f90da-1ec5-4704-931f-6d0f811bf0cd None admin] PATH_INFO: `/v3/roles` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:17:35.308831 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b6f90da-1ec5-4704-931f-6d0f811bf0cd None admin] RBAC: Adding query filter params (name=admin) {{(pid=80245) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 08:17:35.309321 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b6f90da-1ec5-4704-931f-6d0f811bf0cd None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:17:35.310858 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b6f90da-1ec5-4704-931f-6d0f811bf0cd None admin] RBAC: Authorization granted {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:17:35.312792 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application [None req-afbd9a0e-5d38-4068-8054-72d99c75a0ca None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 21 08:17:35.312792 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 08:17:35.312792 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 08:17:35.312792 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 08:17:35.312792 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 08:17:35.312792 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 08:17:35.312792 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 08:17:35.312792 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 08:17:35.312792 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 08:17:35.312792 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 08:17:35.312792 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 08:17:35.312792 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 08:17:35.312792 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 21 08:17:35.312792 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 21 08:17:35.312792 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 21 08:17:35.312792 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 21 08:17:35.313699 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 21 08:17:35.313699 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 21 08:17:35.313699 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 21 08:17:35.313699 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 21 08:17:35.313699 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 08:17:35.313699 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 08:17:35.313699 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 08:17:35.313699 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 08:17:35.313699 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 08:17:35.313699 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 08:17:35.313699 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 08:17:35.313699 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application with Lock( Jul 21 08:17:35.313699 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 08:17:35.313699 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self._enter() Jul 21 08:17:35.313699 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 08:17:35.313699 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 08:17:35.314530 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 08:17:35.314530 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self.creator() Jul 21 08:17:35.314530 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 08:17:35.314530 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application created_value = creator( Jul 21 08:17:35.314530 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 21 08:17:35.314530 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 21 08:17:35.314530 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 21 08:17:35.314530 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 21 08:17:35.314530 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 21 08:17:35.314530 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 21 08:17:35.314530 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 21 08:17:35.314530 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application Jul 21 08:17:35.319402 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 393/792] 10.222.0.47 () {56 vars in 1096 bytes} [Thu Jul 21 08:17:35 2022] GET /identity/v3/roles/admin => generated 83 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 08:17:35.323346 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afbd9a0e-5d38-4068-8054-72d99c75a0ca None admin] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:17:35.330521 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: [pid: 80245|app: 0|req: 400/793] 10.222.0.47 () {56 vars in 1099 bytes} [Thu Jul 21 08:17:35 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 08:17:35.330521 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b6f90da-1ec5-4704-931f-6d0f811bf0cd None admin] Authenticating user token {{(pid=80245) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:17:35.348728 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG dbcounter [-] [80245] Writing DB stats keystone:SELECT=100 {{(pid=80245) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:17:35.374404 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG dbcounter [-] [80244] Writing DB stats keystone:SELECT=100 {{(pid=80244) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:17:35.445490 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b6f90da-1ec5-4704-931f-6d0f811bf0cd None admin] Validating token access rules against request {{(pid=80245) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:17:35.473772 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afbd9a0e-5d38-4068-8054-72d99c75a0ca None admin] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:17:35.496713 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97a098ed-7fd7-4a66-a8d5-a89f6beb1ee1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80245) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:17:35.497956 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97a098ed-7fd7-4a66-a8d5-a89f6beb1ee1 None admin] REQUEST_METHOD: `GET` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:17:35.498449 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97a098ed-7fd7-4a66-a8d5-a89f6beb1ee1 None admin] SCRIPT_NAME: `/identity` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:17:35.499354 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97a098ed-7fd7-4a66-a8d5-a89f6beb1ee1 None admin] PATH_INFO: `/v3/roles` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:17:35.500325 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97a098ed-7fd7-4a66-a8d5-a89f6beb1ee1 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80245) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 08:17:35.500832 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97a098ed-7fd7-4a66-a8d5-a89f6beb1ee1 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:17:35.502436 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97a098ed-7fd7-4a66-a8d5-a89f6beb1ee1 None admin] RBAC: Authorization granted {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:17:35.520448 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: [pid: 80245|app: 0|req: 401/794] 10.222.0.47 () {56 vars in 1099 bytes} [Thu Jul 21 08:17:35 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 08:17:35.520448 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97a098ed-7fd7-4a66-a8d5-a89f6beb1ee1 None admin] Authenticating user token {{(pid=80245) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:17:35.522726 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2a27571-bcf0-4403-a31d-ac34cfbbd3cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:17:35.523577 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2a27571-bcf0-4403-a31d-ac34cfbbd3cd None admin] REQUEST_METHOD: `GET` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:17:35.523867 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2a27571-bcf0-4403-a31d-ac34cfbbd3cd None admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:17:35.524142 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2a27571-bcf0-4403-a31d-ac34cfbbd3cd None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:17:35.532451 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2a27571-bcf0-4403-a31d-ac34cfbbd3cd None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:17:35.533919 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2a27571-bcf0-4403-a31d-ac34cfbbd3cd None admin] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:17:35.550306 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application [None req-c2a27571-bcf0-4403-a31d-ac34cfbbd3cd None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 21 08:17:35.550306 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 08:17:35.550306 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 08:17:35.550306 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 08:17:35.550306 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 08:17:35.550306 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 08:17:35.550306 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 08:17:35.550306 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 08:17:35.550306 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 08:17:35.550306 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 08:17:35.550306 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 08:17:35.550306 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 08:17:35.550306 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 21 08:17:35.550306 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 21 08:17:35.550306 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 21 08:17:35.550306 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 21 08:17:35.552075 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 08:17:35.552075 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 08:17:35.552075 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 21 08:17:35.552075 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 21 08:17:35.552075 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 21 08:17:35.552075 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 21 08:17:35.552075 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 08:17:35.552075 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 08:17:35.552075 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 08:17:35.552075 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 08:17:35.552075 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 08:17:35.552075 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application with Lock( Jul 21 08:17:35.552075 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 08:17:35.552075 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self._enter() Jul 21 08:17:35.552075 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 08:17:35.552075 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 08:17:35.552075 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 08:17:35.552075 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self.creator() Jul 21 08:17:35.553494 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 08:17:35.553494 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application created_value = creator( Jul 21 08:17:35.553494 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 21 08:17:35.553494 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 21 08:17:35.553494 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 21 08:17:35.553494 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 21 08:17:35.553494 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 21 08:17:35.553494 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 21 08:17:35.553494 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 21 08:17:35.553494 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application Jul 21 08:17:35.553494 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 394/795] 10.222.0.47 () {56 vars in 1117 bytes} [Thu Jul 21 08:17:35 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 229 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 08:17:35.553494 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2a27571-bcf0-4403-a31d-ac34cfbbd3cd None admin] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:17:35.630280 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG dbcounter [-] [80245] Writing DB stats keystone:SELECT=100 {{(pid=80245) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:17:35.653321 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG dbcounter [-] [80244] Writing DB stats keystone:SELECT=100 {{(pid=80244) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:17:35.660733 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97a098ed-7fd7-4a66-a8d5-a89f6beb1ee1 None admin] Validating token access rules against request {{(pid=80245) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:17:35.667701 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2a27571-bcf0-4403-a31d-ac34cfbbd3cd None admin] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:17:35.708863 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83ba6da1-4392-4bd6-9c15-fe1837d2f5f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80245) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:17:35.709807 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83ba6da1-4392-4bd6-9c15-fe1837d2f5f7 None admin] REQUEST_METHOD: `GET` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:17:35.710190 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83ba6da1-4392-4bd6-9c15-fe1837d2f5f7 None admin] SCRIPT_NAME: `/identity` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:17:35.710471 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83ba6da1-4392-4bd6-9c15-fe1837d2f5f7 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:17:35.715854 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-465d7abf-af9b-4ba2-9591-990a65b16d0b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:17:35.717027 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-465d7abf-af9b-4ba2-9591-990a65b16d0b None admin] REQUEST_METHOD: `GET` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:17:35.717499 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-465d7abf-af9b-4ba2-9591-990a65b16d0b None admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:17:35.717959 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-465d7abf-af9b-4ba2-9591-990a65b16d0b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:17:35.724413 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83ba6da1-4392-4bd6-9c15-fe1837d2f5f7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:17:35.726000 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83ba6da1-4392-4bd6-9c15-fe1837d2f5f7 None admin] RBAC: Authorization granted {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:17:35.734303 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-465d7abf-af9b-4ba2-9591-990a65b16d0b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:17:35.736872 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-465d7abf-af9b-4ba2-9591-990a65b16d0b None admin] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:17:35.744302 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application [None req-83ba6da1-4392-4bd6-9c15-fe1837d2f5f7 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 08:17:35.744302 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 08:17:35.744302 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 08:17:35.744302 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 08:17:35.744302 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 08:17:35.744302 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 08:17:35.744302 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 08:17:35.744302 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 08:17:35.744302 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 08:17:35.744302 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 08:17:35.744302 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 08:17:35.744302 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 08:17:35.744302 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 21 08:17:35.744302 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 21 08:17:35.744302 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 21 08:17:35.745208 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 21 08:17:35.745208 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 08:17:35.745208 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 08:17:35.745208 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 08:17:35.745208 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 08:17:35.745208 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 08:17:35.745208 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 08:17:35.745208 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 08:17:35.745208 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application with Lock( Jul 21 08:17:35.745208 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 08:17:35.745208 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application return self._enter() Jul 21 08:17:35.745208 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 08:17:35.745208 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 08:17:35.745208 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 08:17:35.745208 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application return self.creator() Jul 21 08:17:35.745208 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 08:17:35.745208 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application created_value = creator( Jul 21 08:17:35.746071 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 21 08:17:35.746071 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 21 08:17:35.746071 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 08:17:35.746071 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application Jul 21 08:17:35.750302 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: [pid: 80245|app: 0|req: 402/796] 10.222.0.47 () {56 vars in 1108 bytes} [Thu Jul 21 08:17:35 2022] GET /identity/v3/domains/Default => generated 87 bytes in 225 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 08:17:35.751265 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83ba6da1-4392-4bd6-9c15-fe1837d2f5f7 None admin] Authenticating user token {{(pid=80245) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:17:35.760093 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application [None req-465d7abf-af9b-4ba2-9591-990a65b16d0b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 08:17:35.760093 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 08:17:35.760093 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 08:17:35.760093 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 08:17:35.760093 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 08:17:35.760093 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 08:17:35.760093 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 08:17:35.760093 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 08:17:35.760093 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 08:17:35.760093 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 08:17:35.760093 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 08:17:35.760093 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 08:17:35.760093 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 21 08:17:35.760093 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 21 08:17:35.760093 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 21 08:17:35.760965 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 21 08:17:35.760965 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 08:17:35.760965 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 08:17:35.760965 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 08:17:35.760965 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 08:17:35.760965 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 08:17:35.760965 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 08:17:35.760965 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 08:17:35.760965 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application with Lock( Jul 21 08:17:35.760965 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 08:17:35.760965 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self._enter() Jul 21 08:17:35.760965 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 08:17:35.760965 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 08:17:35.760965 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 08:17:35.760965 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self.creator() Jul 21 08:17:35.760965 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 08:17:35.760965 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application created_value = creator( Jul 21 08:17:35.761844 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 21 08:17:35.761844 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 21 08:17:35.761844 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 08:17:35.761844 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application Jul 21 08:17:35.764199 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 395/797] 10.222.0.47 () {56 vars in 1108 bytes} [Thu Jul 21 08:17:35 2022] GET /identity/v3/domains/Default => generated 87 bytes in 211 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 08:17:35.765189 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-465d7abf-af9b-4ba2-9591-990a65b16d0b None admin] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:17:35.884025 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83ba6da1-4392-4bd6-9c15-fe1837d2f5f7 None admin] Validating token access rules against request {{(pid=80245) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:17:35.887317 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-465d7abf-af9b-4ba2-9591-990a65b16d0b None admin] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:17:35.914247 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG dbcounter [-] [80245] Writing DB stats keystone:SELECT=100 {{(pid=80245) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:17:35.922565 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1f2170d-1705-4158-8e80-1ec9fc052220 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80245) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:17:35.923235 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1f2170d-1705-4158-8e80-1ec9fc052220 None admin] REQUEST_METHOD: `GET` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:17:35.923542 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1f2170d-1705-4158-8e80-1ec9fc052220 None admin] SCRIPT_NAME: `/identity` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:17:35.923824 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1f2170d-1705-4158-8e80-1ec9fc052220 None admin] PATH_INFO: `/v3/users` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:17:35.924654 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1f2170d-1705-4158-8e80-1ec9fc052220 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80245) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 08:17:35.924958 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1f2170d-1705-4158-8e80-1ec9fc052220 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:17:35.926905 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1f2170d-1705-4158-8e80-1ec9fc052220 None admin] RBAC: Authorization granted {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:17:35.930029 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbe67f37-9dfe-4d5f-bc12-b9e2d39ea269 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:17:35.930444 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbe67f37-9dfe-4d5f-bc12-b9e2d39ea269 None admin] REQUEST_METHOD: `GET` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:17:35.930720 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbe67f37-9dfe-4d5f-bc12-b9e2d39ea269 None admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:17:35.930995 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbe67f37-9dfe-4d5f-bc12-b9e2d39ea269 None admin] PATH_INFO: `/v3/domains` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:17:35.931745 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbe67f37-9dfe-4d5f-bc12-b9e2d39ea269 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80244) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 08:17:35.932057 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbe67f37-9dfe-4d5f-bc12-b9e2d39ea269 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:17:35.933413 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbe67f37-9dfe-4d5f-bc12-b9e2d39ea269 None admin] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:17:35.953543 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: [pid: 80245|app: 0|req: 403/798] 10.222.0.47 () {56 vars in 1149 bytes} [Thu Jul 21 08:17:35 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 08:17:35.954057 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 396/799] 10.222.0.47 () {56 vars in 1109 bytes} [Thu Jul 21 08:17:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 08:17:35.954176 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1f2170d-1705-4158-8e80-1ec9fc052220 None admin] Authenticating user token {{(pid=80245) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:17:35.954275 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a9f76f6-0a5b-43b6-97e5-b3ea99f0f62f None None] REQUEST_METHOD: `GET` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:17:35.954275 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a9f76f6-0a5b-43b6-97e5-b3ea99f0f62f None None] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:17:35.954275 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a9f76f6-0a5b-43b6-97e5-b3ea99f0f62f None None] PATH_INFO: `/` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:17:35.954275 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 397/800] 10.222.0.47 () {54 vars in 926 bytes} [Thu Jul 21 08:17:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 08:17:35.956528 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97634cd1-28ee-41ee-b810-e121cb9cd00b None None] REQUEST_METHOD: `POST` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:17:35.957000 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97634cd1-28ee-41ee-b810-e121cb9cd00b None None] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:17:35.957160 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97634cd1-28ee-41ee-b810-e121cb9cd00b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:17:35.962176 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG dbcounter [-] [80244] Writing DB stats keystone:SELECT=100 {{(pid=80244) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:17:35.975813 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.auth.core [None req-97634cd1-28ee-41ee-b810-e121cb9cd00b None None] MFA Rules not processed for user `2dce40b8d74c41ec94cc36e6ff05d412`. Rule list: `[]` (Enabled: `True`). {{(pid=80244) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 08:17:35.977245 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.fernet_utils [None req-97634cd1-28ee-41ee-b810-e121cb9cd00b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80244) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 08:17:36.052958 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 398/801] 10.222.0.47 () {58 vars in 1023 bytes} [Thu Jul 21 08:17:35 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 21 08:17:36.053804 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97634cd1-28ee-41ee-b810-e121cb9cd00b None None] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:17:36.074969 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1f2170d-1705-4158-8e80-1ec9fc052220 None admin] Validating token access rules against request {{(pid=80245) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:17:36.110704 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8979aa84-3b44-4835-a86d-70989e4dc2ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80245) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:17:36.111520 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8979aa84-3b44-4835-a86d-70989e4dc2ed None admin] REQUEST_METHOD: `GET` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:17:36.111803 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8979aa84-3b44-4835-a86d-70989e4dc2ed None admin] SCRIPT_NAME: `/identity` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:17:36.112084 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8979aa84-3b44-4835-a86d-70989e4dc2ed None admin] PATH_INFO: `/v3/domains` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:17:36.112804 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8979aa84-3b44-4835-a86d-70989e4dc2ed None admin] RBAC: Adding query filter params (name=Default) {{(pid=80245) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 08:17:36.113829 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8979aa84-3b44-4835-a86d-70989e4dc2ed None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:17:36.115124 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8979aa84-3b44-4835-a86d-70989e4dc2ed None admin] RBAC: Authorization granted {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:17:36.130761 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: [pid: 80245|app: 0|req: 404/802] 10.222.0.47 () {56 vars in 1109 bytes} [Thu Jul 21 08:17:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 08:17:36.130761 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8979aa84-3b44-4835-a86d-70989e4dc2ed None admin] Authenticating user token {{(pid=80245) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:17:36.172111 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97634cd1-28ee-41ee-b810-e121cb9cd00b None None] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:17:36.179261 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG dbcounter [-] [80245] Writing DB stats keystone:SELECT=100 {{(pid=80245) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:17:36.205426 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG dbcounter [-] [80244] Writing DB stats keystone:SELECT=100 {{(pid=80244) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:17:36.212576 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a1e38e1-98d5-4e04-ac8a-bcf298128012 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:17:36.213737 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a1e38e1-98d5-4e04-ac8a-bcf298128012 None admin] REQUEST_METHOD: `GET` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:17:36.214029 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a1e38e1-98d5-4e04-ac8a-bcf298128012 None admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:17:36.214307 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a1e38e1-98d5-4e04-ac8a-bcf298128012 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:17:36.225759 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a1e38e1-98d5-4e04-ac8a-bcf298128012 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:17:36.227266 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a1e38e1-98d5-4e04-ac8a-bcf298128012 None admin] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:17:36.237775 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8979aa84-3b44-4835-a86d-70989e4dc2ed None admin] Validating token access rules against request {{(pid=80245) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:17:36.247107 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application [None req-2a1e38e1-98d5-4e04-ac8a-bcf298128012 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 08:17:36.247107 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 08:17:36.247107 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 08:17:36.247107 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 08:17:36.247107 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 08:17:36.247107 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 08:17:36.247107 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 08:17:36.247107 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 08:17:36.247107 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 08:17:36.247107 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 08:17:36.247107 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 08:17:36.247107 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 08:17:36.247107 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 21 08:17:36.247107 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 21 08:17:36.247107 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 21 08:17:36.248036 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 21 08:17:36.248036 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 08:17:36.248036 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 08:17:36.248036 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 08:17:36.248036 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 08:17:36.248036 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 08:17:36.248036 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 08:17:36.248036 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 08:17:36.248036 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application with Lock( Jul 21 08:17:36.248036 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 08:17:36.248036 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self._enter() Jul 21 08:17:36.248036 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 08:17:36.248036 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 08:17:36.248036 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 08:17:36.248036 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self.creator() Jul 21 08:17:36.248036 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 08:17:36.248036 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application created_value = creator( Jul 21 08:17:36.248910 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 21 08:17:36.248910 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 21 08:17:36.248910 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 08:17:36.248910 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application Jul 21 08:17:36.248910 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 399/803] 10.222.0.47 () {56 vars in 1108 bytes} [Thu Jul 21 08:17:36 2022] GET /identity/v3/domains/Default => generated 87 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 08:17:36.249189 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a1e38e1-98d5-4e04-ac8a-bcf298128012 None admin] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:17:36.279837 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aab689ad-24b2-41a8-b63d-5e20070cab92 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80245) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:17:36.280614 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aab689ad-24b2-41a8-b63d-5e20070cab92 None admin] REQUEST_METHOD: `GET` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:17:36.280927 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aab689ad-24b2-41a8-b63d-5e20070cab92 None admin] SCRIPT_NAME: `/identity` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:17:36.281164 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aab689ad-24b2-41a8-b63d-5e20070cab92 None admin] PATH_INFO: `/v3/users/swift` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:17:36.289870 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aab689ad-24b2-41a8-b63d-5e20070cab92 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:17:36.291356 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aab689ad-24b2-41a8-b63d-5e20070cab92 None admin] RBAC: Authorization granted {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:17:36.306492 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application [None req-aab689ad-24b2-41a8-b63d-5e20070cab92 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Jul 21 08:17:36.306492 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 08:17:36.306492 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 08:17:36.306492 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 08:17:36.306492 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 08:17:36.306492 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 08:17:36.306492 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 08:17:36.306492 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 08:17:36.306492 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 08:17:36.306492 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 08:17:36.306492 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 08:17:36.306492 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 08:17:36.306492 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 21 08:17:36.306492 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 21 08:17:36.306492 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 21 08:17:36.306492 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 21 08:17:36.307668 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 08:17:36.307668 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 08:17:36.307668 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 21 08:17:36.307668 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 21 08:17:36.307668 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 21 08:17:36.307668 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 21 08:17:36.307668 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 08:17:36.307668 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 08:17:36.307668 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 08:17:36.307668 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 08:17:36.307668 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 08:17:36.307668 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application with Lock( Jul 21 08:17:36.307668 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 08:17:36.307668 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application return self._enter() Jul 21 08:17:36.307668 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 08:17:36.307668 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 08:17:36.307668 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 08:17:36.307668 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application return self.creator() Jul 21 08:17:36.308598 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 08:17:36.308598 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application created_value = creator( Jul 21 08:17:36.308598 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 21 08:17:36.308598 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 21 08:17:36.308598 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 21 08:17:36.308598 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 21 08:17:36.308598 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 21 08:17:36.308598 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 21 08:17:36.308598 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Jul 21 08:17:36.308598 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application Jul 21 08:17:36.308598 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: [pid: 80245|app: 0|req: 405/804] 10.222.0.47 () {56 vars in 1096 bytes} [Thu Jul 21 08:17:36 2022] GET /identity/v3/users/swift => generated 83 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 08:17:36.313091 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aab689ad-24b2-41a8-b63d-5e20070cab92 None admin] Authenticating user token {{(pid=80245) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:17:36.367450 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a1e38e1-98d5-4e04-ac8a-bcf298128012 None admin] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:17:36.404303 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c84238b8-b9d6-4993-b80a-1ae63f820f96 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:17:36.405407 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c84238b8-b9d6-4993-b80a-1ae63f820f96 None admin] REQUEST_METHOD: `GET` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:17:36.405951 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c84238b8-b9d6-4993-b80a-1ae63f820f96 None admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:17:36.406231 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c84238b8-b9d6-4993-b80a-1ae63f820f96 None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:17:36.406923 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c84238b8-b9d6-4993-b80a-1ae63f820f96 None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:17:36.408334 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c84238b8-b9d6-4993-b80a-1ae63f820f96 None admin] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:17:36.421389 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application [None req-c84238b8-b9d6-4993-b80a-1ae63f820f96 None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Jul 21 08:17:36.421389 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 08:17:36.421389 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 21 08:17:36.421389 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 21 08:17:36.421389 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 21 08:17:36.421389 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 21 08:17:36.421389 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 21 08:17:36.421389 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 21 08:17:36.421389 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Jul 21 08:17:36.421389 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application Jul 21 08:17:36.421389 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 21 08:17:36.421389 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application Jul 21 08:17:36.421389 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 08:17:36.421389 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 08:17:36.421389 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 08:17:36.421389 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 08:17:36.422323 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 08:17:36.422323 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 08:17:36.422323 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 08:17:36.422323 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 08:17:36.422323 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 08:17:36.422323 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 08:17:36.422323 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 08:17:36.422323 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 21 08:17:36.422323 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 21 08:17:36.422323 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 21 08:17:36.422323 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 21 08:17:36.422323 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 08:17:36.422323 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 08:17:36.422323 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 08:17:36.422323 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 08:17:36.422323 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 08:17:36.423152 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 08:17:36.423152 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 08:17:36.423152 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application with Lock( Jul 21 08:17:36.423152 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 08:17:36.423152 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self._enter() Jul 21 08:17:36.423152 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 08:17:36.423152 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 08:17:36.423152 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 08:17:36.423152 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self.creator() Jul 21 08:17:36.423152 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 08:17:36.423152 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application created_value = creator( Jul 21 08:17:36.423152 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 21 08:17:36.423152 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 21 08:17:36.423152 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Jul 21 08:17:36.423152 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application Jul 21 08:17:36.423931 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aab689ad-24b2-41a8-b63d-5e20070cab92 None admin] Validating token access rules against request {{(pid=80245) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:17:36.427689 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 400/805] 10.222.0.47 () {56 vars in 1129 bytes} [Thu Jul 21 08:17:36 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 08:17:36.428638 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c84238b8-b9d6-4993-b80a-1ae63f820f96 None admin] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:17:36.430829 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG dbcounter [-] [80245] Writing DB stats keystone:SELECT=100 {{(pid=80245) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:17:36.462760 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bab2c9f3-a8c0-40c4-a31c-dad484d34f61 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80245) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:17:36.463592 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bab2c9f3-a8c0-40c4-a31c-dad484d34f61 None admin] REQUEST_METHOD: `GET` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:17:36.463874 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bab2c9f3-a8c0-40c4-a31c-dad484d34f61 None admin] SCRIPT_NAME: `/identity` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:17:36.464147 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bab2c9f3-a8c0-40c4-a31c-dad484d34f61 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:17:36.476949 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bab2c9f3-a8c0-40c4-a31c-dad484d34f61 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:17:36.478793 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bab2c9f3-a8c0-40c4-a31c-dad484d34f61 None admin] RBAC: Authorization granted {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:17:36.487682 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG dbcounter [-] [80244] Writing DB stats keystone:SELECT=100 {{(pid=80244) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:17:36.493806 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application [None req-bab2c9f3-a8c0-40c4-a31c-dad484d34f61 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jul 21 08:17:36.493806 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 08:17:36.493806 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 08:17:36.493806 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 08:17:36.493806 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 08:17:36.493806 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 08:17:36.493806 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 08:17:36.493806 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 08:17:36.493806 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 08:17:36.493806 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 08:17:36.493806 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 08:17:36.493806 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 08:17:36.493806 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 21 08:17:36.493806 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 21 08:17:36.493806 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 21 08:17:36.493806 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 21 08:17:36.494717 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 08:17:36.494717 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 08:17:36.494717 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 21 08:17:36.494717 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 21 08:17:36.494717 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 21 08:17:36.494717 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 21 08:17:36.494717 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 08:17:36.494717 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 08:17:36.494717 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 08:17:36.494717 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 08:17:36.494717 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 08:17:36.494717 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application with Lock( Jul 21 08:17:36.494717 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 08:17:36.494717 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application return self._enter() Jul 21 08:17:36.494717 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 08:17:36.494717 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 08:17:36.494717 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 08:17:36.494717 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application return self.creator() Jul 21 08:17:36.495653 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 08:17:36.495653 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application created_value = creator( Jul 21 08:17:36.495653 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 21 08:17:36.495653 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 21 08:17:36.495653 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 21 08:17:36.495653 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 21 08:17:36.495653 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 21 08:17:36.495653 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 21 08:17:36.495653 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jul 21 08:17:36.495653 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application Jul 21 08:17:36.495653 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: [pid: 80245|app: 0|req: 406/806] 10.222.0.47 () {56 vars in 1093 bytes} [Thu Jul 21 08:17:36 2022] GET /identity/v3/users/nova => generated 82 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 08:17:36.496214 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bab2c9f3-a8c0-40c4-a31c-dad484d34f61 None admin] Authenticating user token {{(pid=80245) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:17:36.545831 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c84238b8-b9d6-4993-b80a-1ae63f820f96 None admin] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:17:36.582656 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af1eedba-cf6e-4843-87a7-c4c91c376eb0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:17:36.583236 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af1eedba-cf6e-4843-87a7-c4c91c376eb0 None admin] REQUEST_METHOD: `GET` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:17:36.583545 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af1eedba-cf6e-4843-87a7-c4c91c376eb0 None admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:17:36.583819 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af1eedba-cf6e-4843-87a7-c4c91c376eb0 None admin] PATH_INFO: `/v3/domains` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:17:36.584533 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af1eedba-cf6e-4843-87a7-c4c91c376eb0 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80244) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 08:17:36.584836 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af1eedba-cf6e-4843-87a7-c4c91c376eb0 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:17:36.586195 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af1eedba-cf6e-4843-87a7-c4c91c376eb0 None admin] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:17:36.599353 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 401/807] 10.222.0.47 () {56 vars in 1109 bytes} [Thu Jul 21 08:17:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 08:17:36.600206 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af1eedba-cf6e-4843-87a7-c4c91c376eb0 None admin] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:17:36.617896 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bab2c9f3-a8c0-40c4-a31c-dad484d34f61 None admin] Validating token access rules against request {{(pid=80245) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:17:36.655508 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45415180-d741-47d2-831e-5925b6dc1e62 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80245) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:17:36.656574 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45415180-d741-47d2-831e-5925b6dc1e62 None admin] REQUEST_METHOD: `GET` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:17:36.657058 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45415180-d741-47d2-831e-5925b6dc1e62 None admin] SCRIPT_NAME: `/identity` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:17:36.657357 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45415180-d741-47d2-831e-5925b6dc1e62 None admin] PATH_INFO: `/v3/users` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:17:36.658203 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45415180-d741-47d2-831e-5925b6dc1e62 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80245) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 08:17:36.658511 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45415180-d741-47d2-831e-5925b6dc1e62 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:17:36.660578 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45415180-d741-47d2-831e-5925b6dc1e62 None admin] RBAC: Authorization granted {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:17:36.678035 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: [pid: 80245|app: 0|req: 407/808] 10.222.0.47 () {56 vars in 1135 bytes} [Thu Jul 21 08:17:36 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 08:17:36.678947 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45415180-d741-47d2-831e-5925b6dc1e62 None admin] Authenticating user token {{(pid=80245) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:17:36.704152 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG dbcounter [-] [80245] Writing DB stats keystone:SELECT=100 {{(pid=80245) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:17:36.708445 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af1eedba-cf6e-4843-87a7-c4c91c376eb0 None admin] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:17:36.717687 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG dbcounter [-] [80244] Writing DB stats keystone:SELECT=100 {{(pid=80244) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:17:36.743442 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32fda272-124d-401e-94d4-eaa3ea5adce8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:17:36.744122 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32fda272-124d-401e-94d4-eaa3ea5adce8 None admin] REQUEST_METHOD: `GET` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:17:36.744403 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32fda272-124d-401e-94d4-eaa3ea5adce8 None admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:17:36.744683 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32fda272-124d-401e-94d4-eaa3ea5adce8 None admin] PATH_INFO: `/v3/services` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:17:36.745396 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32fda272-124d-401e-94d4-eaa3ea5adce8 None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=80244) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 08:17:36.745707 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32fda272-124d-401e-94d4-eaa3ea5adce8 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:17:36.747074 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32fda272-124d-401e-94d4-eaa3ea5adce8 None admin] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:17:36.751654 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 402/809] 10.222.0.47 () {56 vars in 1124 bytes} [Thu Jul 21 08:17:36 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 08:17:36.752433 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32fda272-124d-401e-94d4-eaa3ea5adce8 None admin] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:17:36.781274 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45415180-d741-47d2-831e-5925b6dc1e62 None admin] Validating token access rules against request {{(pid=80245) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:17:36.814339 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a70d14c-f84e-44da-9e66-3761c582df1e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80245) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:17:36.815004 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a70d14c-f84e-44da-9e66-3761c582df1e None admin] REQUEST_METHOD: `GET` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:17:36.815282 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a70d14c-f84e-44da-9e66-3761c582df1e None admin] SCRIPT_NAME: `/identity` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:17:36.815584 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a70d14c-f84e-44da-9e66-3761c582df1e None admin] PATH_INFO: `/v3/users` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:17:36.816925 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a70d14c-f84e-44da-9e66-3761c582df1e None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80245) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 08:17:36.817229 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a70d14c-f84e-44da-9e66-3761c582df1e None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:17:36.818646 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a70d14c-f84e-44da-9e66-3761c582df1e None admin] RBAC: Authorization granted {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:17:36.836049 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: [pid: 80245|app: 0|req: 408/810] 10.222.0.47 () {56 vars in 1133 bytes} [Thu Jul 21 08:17:36 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 08:17:36.837269 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22dab15d-9284-46f6-a63f-d4725ba90e55 None None] REQUEST_METHOD: `GET` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:17:36.837564 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22dab15d-9284-46f6-a63f-d4725ba90e55 None None] SCRIPT_NAME: `/identity` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:17:36.837843 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22dab15d-9284-46f6-a63f-d4725ba90e55 None None] PATH_INFO: `/` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:17:36.840148 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: [pid: 80245|app: 0|req: 409/811] 10.222.0.47 () {54 vars in 926 bytes} [Thu Jul 21 08:17:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 08:17:36.840148 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22dab15d-9284-46f6-a63f-d4725ba90e55 None None] Authenticating user token {{(pid=80245) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:17:36.853234 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32fda272-124d-401e-94d4-eaa3ea5adce8 None admin] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:17:36.889951 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2166f37-86b6-4894-833f-a0fe12068409 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:17:36.890764 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2166f37-86b6-4894-833f-a0fe12068409 None admin] REQUEST_METHOD: `GET` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:17:36.891045 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2166f37-86b6-4894-833f-a0fe12068409 None admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:17:36.891352 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2166f37-86b6-4894-833f-a0fe12068409 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:17:36.899051 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2166f37-86b6-4894-833f-a0fe12068409 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:17:36.900530 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2166f37-86b6-4894-833f-a0fe12068409 None admin] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:17:36.914232 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application [None req-c2166f37-86b6-4894-833f-a0fe12068409 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 21 08:17:36.914232 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 08:17:36.914232 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 08:17:36.914232 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 08:17:36.914232 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 08:17:36.914232 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 08:17:36.914232 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 08:17:36.914232 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 08:17:36.914232 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 08:17:36.914232 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 08:17:36.914232 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 08:17:36.914232 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 08:17:36.914232 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 21 08:17:36.914232 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 21 08:17:36.914232 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 21 08:17:36.915292 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 21 08:17:36.915292 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 08:17:36.915292 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 08:17:36.915292 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 08:17:36.915292 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 08:17:36.915292 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 08:17:36.915292 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 08:17:36.915292 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 08:17:36.915292 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application with Lock( Jul 21 08:17:36.915292 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 08:17:36.915292 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self._enter() Jul 21 08:17:36.915292 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 08:17:36.915292 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 08:17:36.915292 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 08:17:36.915292 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self.creator() Jul 21 08:17:36.915292 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 08:17:36.915292 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application created_value = creator( Jul 21 08:17:36.916272 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 21 08:17:36.916272 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 21 08:17:36.916272 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 21 08:17:36.916272 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 21 08:17:36.916272 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 21 08:17:36.916272 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 21 08:17:36.916272 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 21 08:17:36.916272 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application Jul 21 08:17:36.916272 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 403/812] 10.222.0.47 () {56 vars in 1111 bytes} [Thu Jul 21 08:17:36 2022] GET /identity/v3/projects/service => generated 88 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 08:17:36.916761 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2166f37-86b6-4894-833f-a0fe12068409 None admin] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:17:36.927395 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG dbcounter [-] [80245] Writing DB stats keystone:SELECT=100 {{(pid=80245) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:17:36.947284 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22dab15d-9284-46f6-a63f-d4725ba90e55 None None] Validating token access rules against request {{(pid=80245) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:17:36.948008 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG dbcounter [-] [80244] Writing DB stats keystone:SELECT=100 {{(pid=80244) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:17:36.981210 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a56911bc-c949-4138-88f8-da6f1384b7f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80245) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:17:36.982008 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a56911bc-c949-4138-88f8-da6f1384b7f8 None admin] REQUEST_METHOD: `GET` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:17:36.982291 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a56911bc-c949-4138-88f8-da6f1384b7f8 None admin] SCRIPT_NAME: `/identity` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:17:36.982568 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a56911bc-c949-4138-88f8-da6f1384b7f8 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:17:36.992665 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a56911bc-c949-4138-88f8-da6f1384b7f8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:17:36.994116 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a56911bc-c949-4138-88f8-da6f1384b7f8 None admin] RBAC: Authorization granted {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:17:37.012123 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application [None req-a56911bc-c949-4138-88f8-da6f1384b7f8 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 08:17:37.012123 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 08:17:37.012123 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 08:17:37.012123 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 08:17:37.012123 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 08:17:37.012123 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 08:17:37.012123 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 08:17:37.012123 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 08:17:37.012123 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 08:17:37.012123 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 08:17:37.012123 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 08:17:37.012123 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 08:17:37.012123 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 21 08:17:37.012123 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 21 08:17:37.012123 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 21 08:17:37.013008 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 21 08:17:37.013008 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 08:17:37.013008 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 08:17:37.013008 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 08:17:37.013008 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 08:17:37.013008 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 08:17:37.013008 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 08:17:37.013008 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 08:17:37.013008 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application with Lock( Jul 21 08:17:37.013008 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 08:17:37.013008 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application return self._enter() Jul 21 08:17:37.013008 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 08:17:37.013008 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 08:17:37.013008 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 08:17:37.013008 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application return self.creator() Jul 21 08:17:37.013008 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 08:17:37.013008 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application created_value = creator( Jul 21 08:17:37.013904 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 21 08:17:37.013904 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 21 08:17:37.013904 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 08:17:37.013904 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application Jul 21 08:17:37.013904 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: [pid: 80245|app: 0|req: 410/813] 10.222.0.47 () {56 vars in 1108 bytes} [Thu Jul 21 08:17:36 2022] GET /identity/v3/domains/Default => generated 87 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 08:17:37.014174 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a56911bc-c949-4138-88f8-da6f1384b7f8 None admin] Authenticating user token {{(pid=80245) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:17:37.022028 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2166f37-86b6-4894-833f-a0fe12068409 None admin] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:17:37.054739 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70e8b974-49a5-4a60-b1bf-888e20bebbe8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:17:37.055431 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70e8b974-49a5-4a60-b1bf-888e20bebbe8 None admin] REQUEST_METHOD: `GET` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:17:37.055721 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70e8b974-49a5-4a60-b1bf-888e20bebbe8 None admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:17:37.056001 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70e8b974-49a5-4a60-b1bf-888e20bebbe8 None admin] PATH_INFO: `/v3/services` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:17:37.056706 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70e8b974-49a5-4a60-b1bf-888e20bebbe8 None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=80244) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 08:17:37.057015 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70e8b974-49a5-4a60-b1bf-888e20bebbe8 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:17:37.058378 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70e8b974-49a5-4a60-b1bf-888e20bebbe8 None admin] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:17:37.063182 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 404/814] 10.222.0.47 () {56 vars in 1124 bytes} [Thu Jul 21 08:17:36 2022] GET /identity/v3/services?type=block-storage => generated 367 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 08:17:37.064428 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-468f04e4-5c7b-427c-96b5-f3cc8ed151b5 None None] REQUEST_METHOD: `POST` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:17:37.064719 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-468f04e4-5c7b-427c-96b5-f3cc8ed151b5 None None] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:17:37.065001 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-468f04e4-5c7b-427c-96b5-f3cc8ed151b5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:17:37.081075 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.auth.core [None req-468f04e4-5c7b-427c-96b5-f3cc8ed151b5 None None] MFA Rules not processed for user `2dce40b8d74c41ec94cc36e6ff05d412`. Rule list: `[]` (Enabled: `True`). {{(pid=80244) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 08:17:37.082814 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.fernet_utils [None req-468f04e4-5c7b-427c-96b5-f3cc8ed151b5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80244) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 08:17:37.114935 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a56911bc-c949-4138-88f8-da6f1384b7f8 None admin] Validating token access rules against request {{(pid=80245) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:17:37.147937 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 405/815] 10.222.0.47 () {58 vars in 1023 bytes} [Thu Jul 21 08:17:37 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 21 08:17:37.148773 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-468f04e4-5c7b-427c-96b5-f3cc8ed151b5 None None] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:17:37.162895 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cc81199-76c0-473a-8dd5-0f42965ef4c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80245) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:17:37.164048 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cc81199-76c0-473a-8dd5-0f42965ef4c6 None admin] REQUEST_METHOD: `GET` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:17:37.164605 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cc81199-76c0-473a-8dd5-0f42965ef4c6 None admin] SCRIPT_NAME: `/identity` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:17:37.164891 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cc81199-76c0-473a-8dd5-0f42965ef4c6 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:17:37.168465 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG dbcounter [-] [80245] Writing DB stats keystone:SELECT=100 {{(pid=80245) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:17:37.175140 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG dbcounter [-] [80244] Writing DB stats keystone:SELECT=100 {{(pid=80244) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:17:37.252805 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-468f04e4-5c7b-427c-96b5-f3cc8ed151b5 None None] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:17:37.288391 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9864b6d4-d58d-431d-81de-38869c875ce7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:17:37.289052 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9864b6d4-d58d-431d-81de-38869c875ce7 None admin] REQUEST_METHOD: `GET` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:17:37.289332 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9864b6d4-d58d-431d-81de-38869c875ce7 None admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:17:37.289603 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9864b6d4-d58d-431d-81de-38869c875ce7 None admin] PATH_INFO: `/v3/projects` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:17:37.290396 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9864b6d4-d58d-431d-81de-38869c875ce7 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80244) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 08:17:37.290702 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9864b6d4-d58d-431d-81de-38869c875ce7 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:17:37.292164 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9864b6d4-d58d-431d-81de-38869c875ce7 None admin] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:17:37.304227 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 406/816] 10.222.0.47 () {56 vars in 1148 bytes} [Thu Jul 21 08:17:37 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 08:17:37.305025 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9864b6d4-d58d-431d-81de-38869c875ce7 None admin] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:17:37.350368 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cc81199-76c0-473a-8dd5-0f42965ef4c6 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:17:37.351849 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cc81199-76c0-473a-8dd5-0f42965ef4c6 None admin] RBAC: Authorization granted {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:17:37.392340 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG dbcounter [-] [80244] Writing DB stats keystone:SELECT=100 {{(pid=80244) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:17:37.407472 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9864b6d4-d58d-431d-81de-38869c875ce7 None admin] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:17:37.440401 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ab79c97-1054-41e9-b864-c820e0eecc3a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:17:37.441619 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ab79c97-1054-41e9-b864-c820e0eecc3a None admin] REQUEST_METHOD: `GET` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:17:37.441905 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ab79c97-1054-41e9-b864-c820e0eecc3a None admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:17:37.442178 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ab79c97-1054-41e9-b864-c820e0eecc3a None admin] PATH_INFO: `/v3/domains` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:17:37.442893 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ab79c97-1054-41e9-b864-c820e0eecc3a None admin] RBAC: Adding query filter params (name=Default) {{(pid=80244) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 08:17:37.443192 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ab79c97-1054-41e9-b864-c820e0eecc3a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:17:37.444583 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ab79c97-1054-41e9-b864-c820e0eecc3a None admin] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:17:37.450729 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: [pid: 80245|app: 0|req: 411/817] 10.222.0.47 () {58 vars in 1282 bytes} [Thu Jul 21 08:17:37 2022] GET /identity/v3/auth/tokens => generated 1002 bytes in 437 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 21 08:17:37.451585 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cc81199-76c0-473a-8dd5-0f42965ef4c6 None admin] Authenticating user token {{(pid=80245) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:17:37.456845 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 407/818] 10.222.0.47 () {56 vars in 1109 bytes} [Thu Jul 21 08:17:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 08:17:37.457678 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ab79c97-1054-41e9-b864-c820e0eecc3a None admin] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:17:37.494292 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG dbcounter [-] [80245] Writing DB stats keystone:SELECT=100 {{(pid=80245) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:17:37.561877 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cc81199-76c0-473a-8dd5-0f42965ef4c6 None admin] Validating token access rules against request {{(pid=80245) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:17:37.569349 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ab79c97-1054-41e9-b864-c820e0eecc3a None admin] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:17:37.594733 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5832ddca-6db0-4545-8340-c42ff877a709 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80245) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:17:37.595422 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5832ddca-6db0-4545-8340-c42ff877a709 None admin] REQUEST_METHOD: `POST` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:17:37.595705 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5832ddca-6db0-4545-8340-c42ff877a709 None admin] SCRIPT_NAME: `/identity` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:17:37.595983 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5832ddca-6db0-4545-8340-c42ff877a709 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:17:37.596719 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5832ddca-6db0-4545-8340-c42ff877a709 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:17:37.598566 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5832ddca-6db0-4545-8340-c42ff877a709 None admin] RBAC: Authorization granted {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:17:37.603404 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c54a5b6-9946-4c96-9294-340d98378cfe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:17:37.604200 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c54a5b6-9946-4c96-9294-340d98378cfe None admin] REQUEST_METHOD: `GET` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:17:37.604482 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c54a5b6-9946-4c96-9294-340d98378cfe None admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:17:37.604757 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c54a5b6-9946-4c96-9294-340d98378cfe None admin] PATH_INFO: `/v3/services/network` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:17:37.605404 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c54a5b6-9946-4c96-9294-340d98378cfe None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:17:37.606768 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c54a5b6-9946-4c96-9294-340d98378cfe None admin] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:17:37.619984 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application [None req-1c54a5b6-9946-4c96-9294-340d98378cfe None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Jul 21 08:17:37.619984 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 08:17:37.619984 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 21 08:17:37.619984 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 21 08:17:37.619984 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 21 08:17:37.619984 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 21 08:17:37.619984 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 21 08:17:37.619984 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 21 08:17:37.619984 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Jul 21 08:17:37.619984 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application Jul 21 08:17:37.619984 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 21 08:17:37.619984 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application Jul 21 08:17:37.619984 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 08:17:37.619984 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 08:17:37.619984 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 08:17:37.619984 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 08:17:37.619984 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 08:17:37.621120 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 08:17:37.621120 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 08:17:37.621120 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 08:17:37.621120 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 08:17:37.621120 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 08:17:37.621120 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 08:17:37.621120 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 21 08:17:37.621120 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 21 08:17:37.621120 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 21 08:17:37.621120 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 21 08:17:37.621120 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 08:17:37.621120 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 08:17:37.621120 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 08:17:37.621120 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 08:17:37.621120 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 08:17:37.621120 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 08:17:37.622195 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 08:17:37.622195 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application with Lock( Jul 21 08:17:37.622195 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 08:17:37.622195 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self._enter() Jul 21 08:17:37.622195 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 08:17:37.622195 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 08:17:37.622195 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 08:17:37.622195 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self.creator() Jul 21 08:17:37.622195 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 08:17:37.622195 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application created_value = creator( Jul 21 08:17:37.622195 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 21 08:17:37.622195 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 21 08:17:37.622195 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Jul 21 08:17:37.622195 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application Jul 21 08:17:37.623060 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: [pid: 80245|app: 0|req: 412/819] 10.222.0.47 () {60 vars in 1144 bytes} [Thu Jul 21 08:17:37 2022] POST /identity/v3/endpoints => generated 346 bytes in 169 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 08:17:37.623060 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5832ddca-6db0-4545-8340-c42ff877a709 None admin] Authenticating user token {{(pid=80245) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:17:37.628849 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 408/820] 10.222.0.47 () {56 vars in 1111 bytes} [Thu Jul 21 08:17:37 2022] GET /identity/v3/services/network => generated 88 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 08:17:37.629979 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c54a5b6-9946-4c96-9294-340d98378cfe None admin] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:17:37.644569 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG dbcounter [-] [80244] Writing DB stats keystone:SELECT=100 {{(pid=80244) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:17:37.733940 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG dbcounter [-] [80245] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80245) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:17:37.753656 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5832ddca-6db0-4545-8340-c42ff877a709 None admin] Validating token access rules against request {{(pid=80245) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:17:37.754940 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c54a5b6-9946-4c96-9294-340d98378cfe None admin] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:17:37.793486 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb7e77de-7e73-4491-9bb8-1c50127fec4b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80245) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:17:37.794477 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb7e77de-7e73-4491-9bb8-1c50127fec4b None admin] REQUEST_METHOD: `PUT` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:17:37.794610 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f153aeb-ecdc-4cb3-90d3-c77bf85034e8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:17:37.794805 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb7e77de-7e73-4491-9bb8-1c50127fec4b None admin] SCRIPT_NAME: `/identity` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:17:37.795035 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb7e77de-7e73-4491-9bb8-1c50127fec4b None admin] PATH_INFO: `/v3/projects/4faef13aa50e4e36b6c265fb1398fa14/users/2ef7f1e91e554a1f88eb6a63e60a475f/roles/50d1ec7f947f42acb65c57e07f2269d6` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:17:37.795268 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f153aeb-ecdc-4cb3-90d3-c77bf85034e8 None admin] REQUEST_METHOD: `GET` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:17:37.795579 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f153aeb-ecdc-4cb3-90d3-c77bf85034e8 None admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:17:37.795854 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f153aeb-ecdc-4cb3-90d3-c77bf85034e8 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:17:37.809514 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f153aeb-ecdc-4cb3-90d3-c77bf85034e8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:17:37.810679 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f153aeb-ecdc-4cb3-90d3-c77bf85034e8 None admin] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:17:37.814518 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb7e77de-7e73-4491-9bb8-1c50127fec4b None admin] RBAC: Authorizing `identity:create_grant(project_id=4faef13aa50e4e36b6c265fb1398fa14, user_id=2ef7f1e91e554a1f88eb6a63e60a475f, role_id=50d1ec7f947f42acb65c57e07f2269d6)` {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:17:37.816023 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb7e77de-7e73-4491-9bb8-1c50127fec4b None admin] RBAC: Authorization granted {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:17:37.831492 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: [pid: 80245|app: 0|req: 413/821] 10.222.0.47 () {58 vars in 1439 bytes} [Thu Jul 21 08:17:37 2022] PUT /identity/v3/projects/4faef13aa50e4e36b6c265fb1398fa14/users/2ef7f1e91e554a1f88eb6a63e60a475f/roles/50d1ec7f947f42acb65c57e07f2269d6 => generated 0 bytes in 210 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 08:17:37.831965 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb7e77de-7e73-4491-9bb8-1c50127fec4b None admin] Authenticating user token {{(pid=80245) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:17:37.832240 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application [None req-3f153aeb-ecdc-4cb3-90d3-c77bf85034e8 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 08:17:37.832240 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 08:17:37.832240 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 08:17:37.832240 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 08:17:37.832240 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 08:17:37.832240 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 08:17:37.832240 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 08:17:37.832240 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 08:17:37.832240 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 08:17:37.832240 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 08:17:37.832240 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 08:17:37.832240 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 08:17:37.832240 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 21 08:17:37.832240 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 21 08:17:37.832240 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 21 08:17:37.834078 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 21 08:17:37.834078 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 08:17:37.834078 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 08:17:37.834078 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 08:17:37.834078 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 08:17:37.834078 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 08:17:37.834078 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 08:17:37.834078 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 08:17:37.834078 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application with Lock( Jul 21 08:17:37.834078 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 08:17:37.834078 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self._enter() Jul 21 08:17:37.834078 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 08:17:37.834078 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 08:17:37.834078 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 08:17:37.834078 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self.creator() Jul 21 08:17:37.834078 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 08:17:37.834078 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application created_value = creator( Jul 21 08:17:37.834977 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 21 08:17:37.834977 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 21 08:17:37.834977 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 08:17:37.834977 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application Jul 21 08:17:37.834977 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 409/822] 10.222.0.47 () {56 vars in 1108 bytes} [Thu Jul 21 08:17:37 2022] GET /identity/v3/domains/Default => generated 87 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 08:17:37.834977 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f153aeb-ecdc-4cb3-90d3-c77bf85034e8 None admin] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:17:37.922726 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG dbcounter [-] [80244] Writing DB stats keystone:SELECT=100 {{(pid=80244) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:17:37.968219 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f153aeb-ecdc-4cb3-90d3-c77bf85034e8 None admin] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:17:37.985730 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb7e77de-7e73-4491-9bb8-1c50127fec4b None admin] Validating token access rules against request {{(pid=80245) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:17:38.008105 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-617a31ad-5ca2-4664-8877-ef5a72b551c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:17:38.009080 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-617a31ad-5ca2-4664-8877-ef5a72b551c9 None admin] REQUEST_METHOD: `GET` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:17:38.009367 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-617a31ad-5ca2-4664-8877-ef5a72b551c9 None admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:17:38.009647 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-617a31ad-5ca2-4664-8877-ef5a72b551c9 None admin] PATH_INFO: `/v3/services` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:17:38.010367 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-617a31ad-5ca2-4664-8877-ef5a72b551c9 None admin] RBAC: Adding query filter params (name=network) {{(pid=80244) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 08:17:38.010676 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-617a31ad-5ca2-4664-8877-ef5a72b551c9 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:17:38.012624 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-617a31ad-5ca2-4664-8877-ef5a72b551c9 None admin] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:17:38.019548 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 410/823] 10.222.0.47 () {56 vars in 1112 bytes} [Thu Jul 21 08:17:37 2022] GET /identity/v3/services?name=network => generated 124 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 08:17:38.019548 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-617a31ad-5ca2-4664-8877-ef5a72b551c9 None admin] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:17:38.033911 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab9a66f8-6e3f-4085-9205-b7a06113910a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80245) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:17:38.035363 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab9a66f8-6e3f-4085-9205-b7a06113910a None admin] REQUEST_METHOD: `GET` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:17:38.035625 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab9a66f8-6e3f-4085-9205-b7a06113910a None admin] SCRIPT_NAME: `/identity` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:17:38.035999 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab9a66f8-6e3f-4085-9205-b7a06113910a None admin] PATH_INFO: `/v3/projects/service` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:17:38.043232 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab9a66f8-6e3f-4085-9205-b7a06113910a None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:17:38.045564 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG dbcounter [-] [80245] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80245) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:17:38.046528 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab9a66f8-6e3f-4085-9205-b7a06113910a None admin] RBAC: Authorization granted {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:17:38.064805 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application [None req-ab9a66f8-6e3f-4085-9205-b7a06113910a None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 21 08:17:38.064805 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 08:17:38.064805 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 08:17:38.064805 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 08:17:38.064805 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 08:17:38.064805 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 08:17:38.064805 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 08:17:38.064805 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 08:17:38.064805 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 08:17:38.064805 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 08:17:38.064805 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 08:17:38.064805 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 08:17:38.064805 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 21 08:17:38.064805 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 21 08:17:38.064805 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 21 08:17:38.065720 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 21 08:17:38.065720 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 08:17:38.065720 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 08:17:38.065720 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 08:17:38.065720 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 08:17:38.065720 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 08:17:38.065720 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 08:17:38.065720 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 08:17:38.065720 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application with Lock( Jul 21 08:17:38.065720 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 08:17:38.065720 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application return self._enter() Jul 21 08:17:38.065720 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 08:17:38.065720 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 08:17:38.065720 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 08:17:38.065720 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application return self.creator() Jul 21 08:17:38.065720 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 08:17:38.065720 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application created_value = creator( Jul 21 08:17:38.066816 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 21 08:17:38.066816 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 21 08:17:38.066816 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 21 08:17:38.066816 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 21 08:17:38.066816 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 21 08:17:38.066816 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 21 08:17:38.066816 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 21 08:17:38.066816 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application Jul 21 08:17:38.066816 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: [pid: 80245|app: 0|req: 414/824] 10.222.0.47 () {56 vars in 1111 bytes} [Thu Jul 21 08:17:37 2022] GET /identity/v3/projects/service => generated 88 bytes in 235 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 08:17:38.067291 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab9a66f8-6e3f-4085-9205-b7a06113910a None admin] Authenticating user token {{(pid=80245) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:17:38.140901 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-617a31ad-5ca2-4664-8877-ef5a72b551c9 None admin] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:17:38.169485 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG dbcounter [-] [80244] Writing DB stats keystone:SELECT=100 {{(pid=80244) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:17:38.180289 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35c6129a-3004-4700-9314-6eec26259c90 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:17:38.181056 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35c6129a-3004-4700-9314-6eec26259c90 None admin] REQUEST_METHOD: `GET` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:17:38.181151 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35c6129a-3004-4700-9314-6eec26259c90 None admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:17:38.181420 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35c6129a-3004-4700-9314-6eec26259c90 None admin] PATH_INFO: `/v3/domains` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:17:38.182151 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35c6129a-3004-4700-9314-6eec26259c90 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80244) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 08:17:38.182461 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35c6129a-3004-4700-9314-6eec26259c90 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:17:38.183980 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35c6129a-3004-4700-9314-6eec26259c90 None admin] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:17:38.197749 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 411/825] 10.222.0.47 () {56 vars in 1109 bytes} [Thu Jul 21 08:17:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 08:17:38.198794 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35c6129a-3004-4700-9314-6eec26259c90 None admin] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:17:38.206140 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab9a66f8-6e3f-4085-9205-b7a06113910a None admin] Validating token access rules against request {{(pid=80245) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:17:38.246004 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1712740d-68aa-46c8-8d46-8363222f149b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80245) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:17:38.247113 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1712740d-68aa-46c8-8d46-8363222f149b None admin] REQUEST_METHOD: `GET` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:17:38.247618 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1712740d-68aa-46c8-8d46-8363222f149b None admin] SCRIPT_NAME: `/identity` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:17:38.248105 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1712740d-68aa-46c8-8d46-8363222f149b None admin] PATH_INFO: `/v3/services` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:17:38.248998 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1712740d-68aa-46c8-8d46-8363222f149b None admin] RBAC: Adding query filter params (type=network) {{(pid=80245) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 08:17:38.249510 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1712740d-68aa-46c8-8d46-8363222f149b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:17:38.251076 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1712740d-68aa-46c8-8d46-8363222f149b None admin] RBAC: Authorization granted {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:17:38.258793 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: [pid: 80245|app: 0|req: 415/826] 10.222.0.47 () {56 vars in 1112 bytes} [Thu Jul 21 08:17:38 2022] GET /identity/v3/services?type=network => generated 350 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 08:17:38.260008 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1712740d-68aa-46c8-8d46-8363222f149b None admin] Authenticating user token {{(pid=80245) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:17:38.323098 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG dbcounter [-] [80245] Writing DB stats keystone:SELECT=100 {{(pid=80245) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:17:38.326764 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35c6129a-3004-4700-9314-6eec26259c90 None admin] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:17:38.365272 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-537b67cb-8ece-4914-bbe6-e2fcbc122096 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:17:38.365999 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-537b67cb-8ece-4914-bbe6-e2fcbc122096 None admin] REQUEST_METHOD: `GET` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:17:38.366291 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-537b67cb-8ece-4914-bbe6-e2fcbc122096 None admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:17:38.366571 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-537b67cb-8ece-4914-bbe6-e2fcbc122096 None admin] PATH_INFO: `/v3/projects` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:17:38.367392 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-537b67cb-8ece-4914-bbe6-e2fcbc122096 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80244) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 08:17:38.367709 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-537b67cb-8ece-4914-bbe6-e2fcbc122096 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:17:38.368716 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1712740d-68aa-46c8-8d46-8363222f149b None admin] Validating token access rules against request {{(pid=80245) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:17:38.369333 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-537b67cb-8ece-4914-bbe6-e2fcbc122096 None admin] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:17:38.386724 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 412/827] 10.222.0.47 () {56 vars in 1148 bytes} [Thu Jul 21 08:17:38 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 08:17:38.388138 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-537b67cb-8ece-4914-bbe6-e2fcbc122096 None admin] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:17:38.406936 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c912fc81-3482-402d-aafc-d2f742122397 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80245) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:17:38.408481 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c912fc81-3482-402d-aafc-d2f742122397 None admin] REQUEST_METHOD: `GET` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:17:38.409384 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c912fc81-3482-402d-aafc-d2f742122397 None admin] SCRIPT_NAME: `/identity` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:17:38.409893 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c912fc81-3482-402d-aafc-d2f742122397 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:17:38.417632 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c912fc81-3482-402d-aafc-d2f742122397 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:17:38.419445 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c912fc81-3482-402d-aafc-d2f742122397 None admin] RBAC: Authorization granted {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:17:38.435653 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application [None req-c912fc81-3482-402d-aafc-d2f742122397 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 21 08:17:38.435653 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 08:17:38.435653 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 08:17:38.435653 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 08:17:38.435653 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 08:17:38.435653 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 08:17:38.435653 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 08:17:38.435653 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 08:17:38.435653 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 08:17:38.435653 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 08:17:38.435653 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 08:17:38.435653 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 08:17:38.435653 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 21 08:17:38.435653 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 21 08:17:38.435653 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 21 08:17:38.436550 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 21 08:17:38.436550 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 08:17:38.436550 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 08:17:38.436550 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 08:17:38.436550 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 08:17:38.436550 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 08:17:38.436550 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 08:17:38.436550 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 08:17:38.436550 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application with Lock( Jul 21 08:17:38.436550 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 08:17:38.436550 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application return self._enter() Jul 21 08:17:38.436550 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 08:17:38.436550 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 08:17:38.436550 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 08:17:38.436550 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application return self.creator() Jul 21 08:17:38.436550 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 08:17:38.436550 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application created_value = creator( Jul 21 08:17:38.437420 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 21 08:17:38.437420 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 21 08:17:38.437420 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 21 08:17:38.437420 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 21 08:17:38.437420 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 21 08:17:38.437420 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 21 08:17:38.437420 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 21 08:17:38.437420 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application Jul 21 08:17:38.442322 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: [pid: 80245|app: 0|req: 416/828] 10.222.0.47 () {56 vars in 1111 bytes} [Thu Jul 21 08:17:38 2022] GET /identity/v3/projects/service => generated 88 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 08:17:38.443340 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c912fc81-3482-402d-aafc-d2f742122397 None admin] Authenticating user token {{(pid=80245) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:17:38.446825 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG dbcounter [-] [80244] Writing DB stats keystone:SELECT=100 {{(pid=80244) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:17:38.522950 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-537b67cb-8ece-4914-bbe6-e2fcbc122096 None admin] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:17:38.557464 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee2042a6-1501-473a-86b2-3fddc23d5f61 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:17:38.558057 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c912fc81-3482-402d-aafc-d2f742122397 None admin] Validating token access rules against request {{(pid=80245) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:17:38.558393 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee2042a6-1501-473a-86b2-3fddc23d5f61 None admin] REQUEST_METHOD: `POST` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:17:38.558789 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee2042a6-1501-473a-86b2-3fddc23d5f61 None admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:17:38.559169 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee2042a6-1501-473a-86b2-3fddc23d5f61 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:17:38.560051 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee2042a6-1501-473a-86b2-3fddc23d5f61 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:17:38.562026 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee2042a6-1501-473a-86b2-3fddc23d5f61 None admin] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:17:38.585232 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG dbcounter [-] [80245] Writing DB stats keystone:SELECT=100 {{(pid=80245) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:17:38.595366 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 413/829] 10.222.0.47 () {60 vars in 1144 bytes} [Thu Jul 21 08:17:38 2022] POST /identity/v3/endpoints => generated 327 bytes in 200 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 08:17:38.595366 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee2042a6-1501-473a-86b2-3fddc23d5f61 None admin] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:17:38.643545 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14b9ba19-dbc3-4b5e-835c-7d89ac3b4c70 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80245) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:17:38.644568 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14b9ba19-dbc3-4b5e-835c-7d89ac3b4c70 None admin] REQUEST_METHOD: `PUT` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:17:38.644891 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14b9ba19-dbc3-4b5e-835c-7d89ac3b4c70 None admin] SCRIPT_NAME: `/identity` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:17:38.645170 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14b9ba19-dbc3-4b5e-835c-7d89ac3b4c70 None admin] PATH_INFO: `/v3/projects/4faef13aa50e4e36b6c265fb1398fa14/users/302351411fc84248a6b166b25d21609f/roles/e29870f6e9e2411a9db0d5bf45f32ca5` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:17:38.647255 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14b9ba19-dbc3-4b5e-835c-7d89ac3b4c70 None admin] RBAC: Authorizing `identity:create_grant(project_id=4faef13aa50e4e36b6c265fb1398fa14, user_id=302351411fc84248a6b166b25d21609f, role_id=e29870f6e9e2411a9db0d5bf45f32ca5)` {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:17:38.648752 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14b9ba19-dbc3-4b5e-835c-7d89ac3b4c70 None admin] RBAC: Authorization granted {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:17:38.679707 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: [pid: 80245|app: 0|req: 417/830] 10.222.0.47 () {58 vars in 1439 bytes} [Thu Jul 21 08:17:38 2022] PUT /identity/v3/projects/4faef13aa50e4e36b6c265fb1398fa14/users/302351411fc84248a6b166b25d21609f/roles/e29870f6e9e2411a9db0d5bf45f32ca5 => generated 0 bytes in 237 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 08:17:38.764830 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee2042a6-1501-473a-86b2-3fddc23d5f61 None admin] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:17:38.766787 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG dbcounter [-] [80244] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80244) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:17:38.812001 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba9a8cc9-1eba-402e-8759-a01109b9e654 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:17:38.812690 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba9a8cc9-1eba-402e-8759-a01109b9e654 None admin] REQUEST_METHOD: `GET` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:17:38.812969 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba9a8cc9-1eba-402e-8759-a01109b9e654 None admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:17:38.813245 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba9a8cc9-1eba-402e-8759-a01109b9e654 None admin] PATH_INFO: `/v3/projects` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:17:38.814052 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba9a8cc9-1eba-402e-8759-a01109b9e654 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80244) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 08:17:38.814358 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba9a8cc9-1eba-402e-8759-a01109b9e654 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:17:38.815908 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba9a8cc9-1eba-402e-8759-a01109b9e654 None admin] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:17:38.830574 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 414/831] 10.222.0.47 () {56 vars in 1148 bytes} [Thu Jul 21 08:17:38 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 242 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 08:17:38.836780 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14b9ba19-dbc3-4b5e-835c-7d89ac3b4c70 None admin] Authenticating user token {{(pid=80245) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:17:38.961424 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14b9ba19-dbc3-4b5e-835c-7d89ac3b4c70 None admin] Validating token access rules against request {{(pid=80245) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:17:39.001674 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe829c18-9588-4d00-bec2-f83ac2ba2db3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80245) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:17:39.003075 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe829c18-9588-4d00-bec2-f83ac2ba2db3 None admin] REQUEST_METHOD: `GET` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:17:39.003590 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe829c18-9588-4d00-bec2-f83ac2ba2db3 None admin] SCRIPT_NAME: `/identity` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:17:39.004052 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe829c18-9588-4d00-bec2-f83ac2ba2db3 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:17:39.005056 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe829c18-9588-4d00-bec2-f83ac2ba2db3 None admin] RBAC: Adding query filter params (role.id=e29870f6e9e2411a9db0d5bf45f32ca5, scope.project.id=4faef13aa50e4e36b6c265fb1398fa14, user.id=ac2b5b99a36442148fe743269054e991) {{(pid=80245) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 08:17:39.005562 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe829c18-9588-4d00-bec2-f83ac2ba2db3 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:17:39.007157 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe829c18-9588-4d00-bec2-f83ac2ba2db3 None admin] RBAC: Authorization granted {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:17:39.014530 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: [pid: 80245|app: 0|req: 418/832] 10.222.0.47 () {56 vars in 1374 bytes} [Thu Jul 21 08:17:38 2022] GET /identity/v3/role_assignments?user.id=ac2b5b99a36442148fe743269054e991&scope.project.id=4faef13aa50e4e36b6c265fb1398fa14&role.id=e29870f6e9e2411a9db0d5bf45f32ca5 => generated 608 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 08:17:39.748358 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2800b05-c8fc-4537-b368-8cc11df5d62b None None] REQUEST_METHOD: `GET` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:17:39.748836 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2800b05-c8fc-4537-b368-8cc11df5d62b None None] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:17:39.748836 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2800b05-c8fc-4537-b368-8cc11df5d62b None None] PATH_INFO: `/` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:17:39.750467 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 415/833] 10.222.0.47 () {54 vars in 926 bytes} [Thu Jul 21 08:17:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 08:17:39.756462 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2034db44-f4b5-47df-8773-26130d736de1 None None] REQUEST_METHOD: `POST` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:17:39.756752 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2034db44-f4b5-47df-8773-26130d736de1 None None] SCRIPT_NAME: `/identity` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:17:39.757033 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2034db44-f4b5-47df-8773-26130d736de1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:17:39.774650 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.auth.core [None req-2034db44-f4b5-47df-8773-26130d736de1 None None] MFA Rules not processed for user `2dce40b8d74c41ec94cc36e6ff05d412`. Rule list: `[]` (Enabled: `True`). {{(pid=80245) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 08:17:39.775870 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.fernet_utils [None req-2034db44-f4b5-47df-8773-26130d736de1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80245) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 08:17:39.811821 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG dbcounter [-] [80245] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80245) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:17:39.848502 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: [pid: 80245|app: 0|req: 419/834] 10.222.0.47 () {58 vars in 1023 bytes} [Thu Jul 21 08:17:39 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 21 08:17:39.870335 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2800b05-c8fc-4537-b368-8cc11df5d62b None None] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:17:39.991339 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2800b05-c8fc-4537-b368-8cc11df5d62b None None] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:17:40.029071 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a851b57-6d3a-456a-9382-5d38a097da1d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:17:40.029805 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a851b57-6d3a-456a-9382-5d38a097da1d None admin] REQUEST_METHOD: `GET` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:17:40.030093 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a851b57-6d3a-456a-9382-5d38a097da1d None admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:17:40.030372 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a851b57-6d3a-456a-9382-5d38a097da1d None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:17:40.031033 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a851b57-6d3a-456a-9382-5d38a097da1d None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:17:40.032448 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a851b57-6d3a-456a-9382-5d38a097da1d None admin] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:17:40.044882 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application [None req-0a851b57-6d3a-456a-9382-5d38a097da1d None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Jul 21 08:17:40.044882 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 08:17:40.044882 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 21 08:17:40.044882 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 21 08:17:40.044882 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 21 08:17:40.044882 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 21 08:17:40.044882 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 21 08:17:40.044882 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 21 08:17:40.044882 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Jul 21 08:17:40.044882 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application Jul 21 08:17:40.044882 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 21 08:17:40.044882 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application Jul 21 08:17:40.044882 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 08:17:40.044882 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 08:17:40.044882 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 08:17:40.044882 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 08:17:40.044882 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 08:17:40.045851 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 08:17:40.045851 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 08:17:40.045851 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 08:17:40.045851 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 08:17:40.045851 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 08:17:40.045851 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 08:17:40.045851 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 21 08:17:40.045851 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 21 08:17:40.045851 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 21 08:17:40.045851 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 21 08:17:40.045851 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 08:17:40.045851 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 08:17:40.045851 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 08:17:40.045851 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 08:17:40.045851 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 08:17:40.045851 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 08:17:40.046670 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 08:17:40.046670 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application with Lock( Jul 21 08:17:40.046670 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 08:17:40.046670 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self._enter() Jul 21 08:17:40.046670 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 08:17:40.046670 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 08:17:40.046670 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 08:17:40.046670 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self.creator() Jul 21 08:17:40.046670 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 08:17:40.046670 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application created_value = creator( Jul 21 08:17:40.046670 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 21 08:17:40.046670 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 21 08:17:40.046670 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Jul 21 08:17:40.046670 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application Jul 21 08:17:40.046670 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 416/835] 10.222.0.47 () {56 vars in 1114 bytes} [Thu Jul 21 08:17:39 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 08:17:40.052140 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2034db44-f4b5-47df-8773-26130d736de1 None None] Authenticating user token {{(pid=80245) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:17:40.111087 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01afcba6-33f4-4b5a-9149-0ae684fea292 None None] REQUEST_METHOD: `GET` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:17:40.111520 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01afcba6-33f4-4b5a-9149-0ae684fea292 None None] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:17:40.111885 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01afcba6-33f4-4b5a-9149-0ae684fea292 None None] PATH_INFO: `/` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:17:40.115309 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 417/836] 10.222.0.47 () {54 vars in 926 bytes} [Thu Jul 21 08:17:40 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 08:17:40.119426 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5389c166-506f-4f76-a795-d767720f3acd None None] REQUEST_METHOD: `POST` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:17:40.119731 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5389c166-506f-4f76-a795-d767720f3acd None None] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:17:40.120088 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5389c166-506f-4f76-a795-d767720f3acd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:17:40.138029 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.auth.core [None req-5389c166-506f-4f76-a795-d767720f3acd None None] MFA Rules not processed for user `2dce40b8d74c41ec94cc36e6ff05d412`. Rule list: `[]` (Enabled: `True`). {{(pid=80244) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 08:17:40.139411 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.fernet_utils [None req-5389c166-506f-4f76-a795-d767720f3acd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80244) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 08:17:40.161625 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2034db44-f4b5-47df-8773-26130d736de1 None None] Validating token access rules against request {{(pid=80245) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:17:40.161816 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG dbcounter [-] [80244] Writing DB stats keystone:SELECT=100 {{(pid=80244) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:17:40.198384 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-237092fa-4d93-49b0-ab91-1b5fdbbc144c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80245) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:17:40.199558 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-237092fa-4d93-49b0-ab91-1b5fdbbc144c None admin] REQUEST_METHOD: `GET` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:17:40.200036 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-237092fa-4d93-49b0-ab91-1b5fdbbc144c None admin] SCRIPT_NAME: `/identity` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:17:40.200490 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-237092fa-4d93-49b0-ab91-1b5fdbbc144c None admin] PATH_INFO: `/v3/services` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:17:40.201410 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-237092fa-4d93-49b0-ab91-1b5fdbbc144c None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=80245) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 08:17:40.201906 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-237092fa-4d93-49b0-ab91-1b5fdbbc144c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:17:40.203648 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-237092fa-4d93-49b0-ab91-1b5fdbbc144c None admin] RBAC: Authorization granted {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:17:40.209869 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: [pid: 80245|app: 0|req: 420/837] 10.222.0.47 () {56 vars in 1114 bytes} [Thu Jul 21 08:17:40 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 08:17:40.221100 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 418/838] 10.222.0.47 () {58 vars in 1023 bytes} [Thu Jul 21 08:17:40 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 21 08:17:40.221286 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-237092fa-4d93-49b0-ab91-1b5fdbbc144c None admin] Authenticating user token {{(pid=80245) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:17:40.240550 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5389c166-506f-4f76-a795-d767720f3acd None None] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:17:40.264377 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG dbcounter [-] [80245] Writing DB stats keystone:SELECT=100 {{(pid=80245) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:17:40.340220 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-237092fa-4d93-49b0-ab91-1b5fdbbc144c None admin] Validating token access rules against request {{(pid=80245) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:17:40.350321 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5389c166-506f-4f76-a795-d767720f3acd None None] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:17:40.376464 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec7e22dc-ab00-409e-9171-22f4a5f68ccd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80245) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:17:40.377142 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec7e22dc-ab00-409e-9171-22f4a5f68ccd None admin] REQUEST_METHOD: `GET` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:17:40.377432 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec7e22dc-ab00-409e-9171-22f4a5f68ccd None admin] SCRIPT_NAME: `/identity` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:17:40.377706 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec7e22dc-ab00-409e-9171-22f4a5f68ccd None admin] PATH_INFO: `/v3/services` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:17:40.378957 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec7e22dc-ab00-409e-9171-22f4a5f68ccd None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=80245) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 08:17:40.379280 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec7e22dc-ab00-409e-9171-22f4a5f68ccd None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:17:40.380693 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec7e22dc-ab00-409e-9171-22f4a5f68ccd None admin] RBAC: Authorization granted {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:17:40.390706 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b63e7cb-375f-4bcb-9671-38a00f1be8a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:17:40.391456 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: [pid: 80245|app: 0|req: 421/839] 10.222.0.47 () {56 vars in 1114 bytes} [Thu Jul 21 08:17:40 2022] GET /identity/v3/services?type=volumev3 => generated 125 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 08:17:40.392167 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b63e7cb-375f-4bcb-9671-38a00f1be8a7 None admin] REQUEST_METHOD: `GET` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:17:40.392652 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b63e7cb-375f-4bcb-9671-38a00f1be8a7 None admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:17:40.393129 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b63e7cb-375f-4bcb-9671-38a00f1be8a7 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:17:40.406533 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b63e7cb-375f-4bcb-9671-38a00f1be8a7 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 21 08:17:40.415039 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application [None req-1b63e7cb-375f-4bcb-9671-38a00f1be8a7 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 21 08:17:40.415039 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 08:17:40.415039 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 08:17:40.415039 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 08:17:40.415039 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 08:17:40.415039 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 08:17:40.415039 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 08:17:40.415039 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 08:17:40.415039 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 08:17:40.415039 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 08:17:40.415039 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 08:17:40.415039 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 08:17:40.415039 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 21 08:17:40.415039 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 21 08:17:40.415039 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 21 08:17:40.415039 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 21 08:17:40.416077 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 21 08:17:40.416077 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 21 08:17:40.416077 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 21 08:17:40.416077 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 21 08:17:40.416077 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 08:17:40.416077 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 08:17:40.416077 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 08:17:40.416077 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 08:17:40.416077 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 08:17:40.416077 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 08:17:40.416077 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 08:17:40.416077 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application with Lock( Jul 21 08:17:40.416077 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 08:17:40.416077 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self._enter() Jul 21 08:17:40.416077 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 08:17:40.416077 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 08:17:40.416941 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 08:17:40.416941 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self.creator() Jul 21 08:17:40.416941 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 08:17:40.416941 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application created_value = creator( Jul 21 08:17:40.416941 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 21 08:17:40.416941 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 21 08:17:40.416941 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 21 08:17:40.416941 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 21 08:17:40.416941 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 21 08:17:40.416941 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 21 08:17:40.416941 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 21 08:17:40.416941 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application Jul 21 08:17:40.419896 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 419/840] 10.222.0.47 () {56 vars in 1102 bytes} [Thu Jul 21 08:17:40 2022] GET /identity/v3/roles/service => generated 85 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 21 08:17:40.426854 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec7e22dc-ab00-409e-9171-22f4a5f68ccd None admin] Authenticating user token {{(pid=80245) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:17:40.533379 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG dbcounter [-] [80245] Writing DB stats keystone:SELECT=100 {{(pid=80245) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:17:40.538841 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec7e22dc-ab00-409e-9171-22f4a5f68ccd None admin] Validating token access rules against request {{(pid=80245) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:17:40.577973 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da0e8758-2f6e-47ac-a6b6-3b81c802cd70 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80245) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:17:40.578658 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da0e8758-2f6e-47ac-a6b6-3b81c802cd70 None admin] REQUEST_METHOD: `GET` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:17:40.578939 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da0e8758-2f6e-47ac-a6b6-3b81c802cd70 None admin] SCRIPT_NAME: `/identity` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:17:40.579213 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da0e8758-2f6e-47ac-a6b6-3b81c802cd70 None admin] PATH_INFO: `/v3/roles` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:17:40.579993 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da0e8758-2f6e-47ac-a6b6-3b81c802cd70 None admin] RBAC: Adding query filter params (name=service) {{(pid=80245) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 08:17:40.580296 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da0e8758-2f6e-47ac-a6b6-3b81c802cd70 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:17:40.581691 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da0e8758-2f6e-47ac-a6b6-3b81c802cd70 None admin] RBAC: Authorization granted {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:17:40.592703 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: [pid: 80245|app: 0|req: 422/841] 10.222.0.47 () {56 vars in 1103 bytes} [Thu Jul 21 08:17:40 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 21 08:17:40.598637 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b63e7cb-375f-4bcb-9671-38a00f1be8a7 None admin] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:17:40.628680 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG dbcounter [-] [80244] Writing DB stats keystone:SELECT=100 {{(pid=80244) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:17:40.713024 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b63e7cb-375f-4bcb-9671-38a00f1be8a7 None admin] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:17:40.749019 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6e22d36-f068-4ca6-acb3-2710e8676449 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:17:40.750639 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6e22d36-f068-4ca6-acb3-2710e8676449 None admin] REQUEST_METHOD: `GET` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:17:40.751175 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6e22d36-f068-4ca6-acb3-2710e8676449 None admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:17:40.751719 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6e22d36-f068-4ca6-acb3-2710e8676449 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:17:40.764417 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6e22d36-f068-4ca6-acb3-2710e8676449 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:17:40.766237 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6e22d36-f068-4ca6-acb3-2710e8676449 None admin] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:17:40.785368 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application [None req-d6e22d36-f068-4ca6-acb3-2710e8676449 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 08:17:40.785368 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 08:17:40.785368 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 08:17:40.785368 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 08:17:40.785368 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 08:17:40.785368 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 08:17:40.785368 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 08:17:40.785368 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 08:17:40.785368 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 08:17:40.785368 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 08:17:40.785368 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 08:17:40.785368 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 08:17:40.785368 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 21 08:17:40.785368 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 21 08:17:40.785368 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 21 08:17:40.786398 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 21 08:17:40.786398 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 08:17:40.786398 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 08:17:40.786398 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 08:17:40.786398 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 08:17:40.786398 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 08:17:40.786398 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 08:17:40.786398 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 08:17:40.786398 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application with Lock( Jul 21 08:17:40.786398 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 08:17:40.786398 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self._enter() Jul 21 08:17:40.786398 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 08:17:40.786398 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 08:17:40.786398 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 08:17:40.786398 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self.creator() Jul 21 08:17:40.786398 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 08:17:40.786398 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application created_value = creator( Jul 21 08:17:40.787264 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 21 08:17:40.787264 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 21 08:17:40.787264 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 08:17:40.787264 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application Jul 21 08:17:40.789555 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 420/842] 10.222.0.47 () {56 vars in 1108 bytes} [Thu Jul 21 08:17:40 2022] GET /identity/v3/domains/Default => generated 87 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 21 08:17:40.795292 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da0e8758-2f6e-47ac-a6b6-3b81c802cd70 None admin] Authenticating user token {{(pid=80245) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:17:40.905074 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da0e8758-2f6e-47ac-a6b6-3b81c802cd70 None admin] Validating token access rules against request {{(pid=80245) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:17:40.927211 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb4ae9c8-7614-4bd3-b9bb-7223b35477c9 None None] REQUEST_METHOD: `GET` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:17:40.927542 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb4ae9c8-7614-4bd3-b9bb-7223b35477c9 None None] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:17:40.927791 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb4ae9c8-7614-4bd3-b9bb-7223b35477c9 None None] PATH_INFO: `/` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:17:40.929767 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 421/843] 10.222.0.47 () {54 vars in 926 bytes} [Thu Jul 21 08:17:40 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 08:17:40.935232 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5f600bb-3db0-4301-829c-30eeef70d67b None None] REQUEST_METHOD: `POST` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:17:40.935494 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5f600bb-3db0-4301-829c-30eeef70d67b None None] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:17:40.935766 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5f600bb-3db0-4301-829c-30eeef70d67b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:17:40.944675 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-909e6d0b-7534-4072-aa63-7de2614e2017 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80245) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:17:40.945347 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-909e6d0b-7534-4072-aa63-7de2614e2017 None admin] REQUEST_METHOD: `GET` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:17:40.945631 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-909e6d0b-7534-4072-aa63-7de2614e2017 None admin] SCRIPT_NAME: `/identity` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:17:40.945906 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-909e6d0b-7534-4072-aa63-7de2614e2017 None admin] PATH_INFO: `/v3/domains` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:17:40.946633 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-909e6d0b-7534-4072-aa63-7de2614e2017 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80245) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 08:17:40.946941 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-909e6d0b-7534-4072-aa63-7de2614e2017 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:17:40.948357 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-909e6d0b-7534-4072-aa63-7de2614e2017 None admin] RBAC: Authorization granted {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:17:40.956374 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.auth.core [None req-f5f600bb-3db0-4301-829c-30eeef70d67b None None] MFA Rules not processed for user `2dce40b8d74c41ec94cc36e6ff05d412`. Rule list: `[]` (Enabled: `True`). {{(pid=80244) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 08:17:40.958491 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.fernet_utils [None req-f5f600bb-3db0-4301-829c-30eeef70d67b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80244) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 08:17:40.960953 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: [pid: 80245|app: 0|req: 423/844] 10.222.0.47 () {56 vars in 1109 bytes} [Thu Jul 21 08:17:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 08:17:40.967447 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-909e6d0b-7534-4072-aa63-7de2614e2017 None admin] Authenticating user token {{(pid=80245) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:17:40.996549 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG dbcounter [-] [80245] Writing DB stats keystone:SELECT=100 {{(pid=80245) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:17:41.030468 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG dbcounter [-] [80244] Writing DB stats keystone:SELECT=100 {{(pid=80244) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:17:41.043413 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 422/845] 10.222.0.47 () {58 vars in 1023 bytes} [Thu Jul 21 08:17:40 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 21 08:17:41.067165 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5f600bb-3db0-4301-829c-30eeef70d67b None None] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:17:41.101163 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-909e6d0b-7534-4072-aa63-7de2614e2017 None admin] Validating token access rules against request {{(pid=80245) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:17:41.138718 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60ebee77-e2fb-4260-bddb-65d4394f7937 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80245) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:17:41.139749 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60ebee77-e2fb-4260-bddb-65d4394f7937 None admin] REQUEST_METHOD: `GET` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:17:41.140192 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60ebee77-e2fb-4260-bddb-65d4394f7937 None admin] SCRIPT_NAME: `/identity` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:17:41.140551 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60ebee77-e2fb-4260-bddb-65d4394f7937 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:17:41.148738 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60ebee77-e2fb-4260-bddb-65d4394f7937 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:17:41.150380 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60ebee77-e2fb-4260-bddb-65d4394f7937 None admin] RBAC: Authorization granted {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:17:41.166442 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application [None req-60ebee77-e2fb-4260-bddb-65d4394f7937 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 21 08:17:41.166442 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 08:17:41.166442 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 08:17:41.166442 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 08:17:41.166442 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 08:17:41.166442 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 08:17:41.166442 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 08:17:41.166442 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 08:17:41.166442 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 08:17:41.166442 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 08:17:41.166442 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 08:17:41.166442 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 08:17:41.166442 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 21 08:17:41.166442 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 21 08:17:41.166442 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 21 08:17:41.166442 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 21 08:17:41.167439 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 08:17:41.167439 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 08:17:41.167439 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 21 08:17:41.167439 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 21 08:17:41.167439 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 21 08:17:41.167439 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 21 08:17:41.167439 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 08:17:41.167439 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 08:17:41.167439 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 08:17:41.167439 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 08:17:41.167439 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 08:17:41.167439 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application with Lock( Jul 21 08:17:41.167439 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 08:17:41.167439 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application return self._enter() Jul 21 08:17:41.167439 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 08:17:41.167439 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 08:17:41.167439 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 08:17:41.167439 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application return self.creator() Jul 21 08:17:41.168523 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 08:17:41.168523 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application created_value = creator( Jul 21 08:17:41.168523 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 21 08:17:41.168523 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 21 08:17:41.168523 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 21 08:17:41.168523 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 21 08:17:41.168523 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 21 08:17:41.168523 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 21 08:17:41.168523 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 21 08:17:41.168523 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application Jul 21 08:17:41.169041 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: [pid: 80245|app: 0|req: 424/846] 10.222.0.47 () {56 vars in 1117 bytes} [Thu Jul 21 08:17:40 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 21 08:17:41.170263 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e92c6836-31f9-412e-ac21-54af4c5eb97c None None] REQUEST_METHOD: `GET` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:17:41.170558 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e92c6836-31f9-412e-ac21-54af4c5eb97c None None] SCRIPT_NAME: `/identity` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:17:41.170837 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e92c6836-31f9-412e-ac21-54af4c5eb97c None None] PATH_INFO: `/` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:17:41.175878 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: [pid: 80245|app: 0|req: 425/847] 10.222.0.47 () {54 vars in 926 bytes} [Thu Jul 21 08:17:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 08:17:41.177490 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e92c6836-31f9-412e-ac21-54af4c5eb97c None None] Authenticating user token {{(pid=80245) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:17:41.185656 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5f600bb-3db0-4301-829c-30eeef70d67b None None] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:17:41.220516 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7140192-d145-4db8-b177-b382483bb523 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:17:41.221317 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7140192-d145-4db8-b177-b382483bb523 None admin] REQUEST_METHOD: `GET` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:17:41.221601 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7140192-d145-4db8-b177-b382483bb523 None admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:17:41.221876 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7140192-d145-4db8-b177-b382483bb523 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:17:41.233104 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7140192-d145-4db8-b177-b382483bb523 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 21 08:17:41.241358 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application [None req-a7140192-d145-4db8-b177-b382483bb523 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 21 08:17:41.241358 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 08:17:41.241358 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 08:17:41.241358 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 08:17:41.241358 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 08:17:41.241358 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 08:17:41.241358 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 08:17:41.241358 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 08:17:41.241358 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 08:17:41.241358 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 08:17:41.241358 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 08:17:41.241358 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 08:17:41.241358 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 21 08:17:41.241358 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 21 08:17:41.241358 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 21 08:17:41.241358 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 21 08:17:41.242286 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 21 08:17:41.242286 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 21 08:17:41.242286 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 21 08:17:41.242286 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 21 08:17:41.242286 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 08:17:41.242286 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 08:17:41.242286 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 08:17:41.242286 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 08:17:41.242286 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 08:17:41.242286 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 08:17:41.242286 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 08:17:41.242286 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application with Lock( Jul 21 08:17:41.242286 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 08:17:41.242286 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self._enter() Jul 21 08:17:41.242286 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 08:17:41.242286 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 08:17:41.243143 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 08:17:41.243143 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self.creator() Jul 21 08:17:41.243143 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 08:17:41.243143 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application created_value = creator( Jul 21 08:17:41.243143 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 21 08:17:41.243143 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 21 08:17:41.243143 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 21 08:17:41.243143 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 21 08:17:41.243143 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 21 08:17:41.243143 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 21 08:17:41.243143 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 21 08:17:41.243143 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application Jul 21 08:17:41.243143 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 423/848] 10.222.0.47 () {56 vars in 1096 bytes} [Thu Jul 21 08:17:41 2022] GET /identity/v3/roles/admin => generated 83 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 21 08:17:41.244347 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e8d568d-36c5-4ef4-adb4-dccdac5c911e None None] REQUEST_METHOD: `POST` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:17:41.244521 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e8d568d-36c5-4ef4-adb4-dccdac5c911e None None] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:17:41.244742 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e8d568d-36c5-4ef4-adb4-dccdac5c911e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:17:41.261997 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.auth.core [None req-3e8d568d-36c5-4ef4-adb4-dccdac5c911e None None] MFA Rules not processed for user `2dce40b8d74c41ec94cc36e6ff05d412`. Rule list: `[]` (Enabled: `True`). {{(pid=80244) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 08:17:41.263230 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.fernet_utils [None req-3e8d568d-36c5-4ef4-adb4-dccdac5c911e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80244) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 08:17:41.278608 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG dbcounter [-] [80245] Writing DB stats keystone:SELECT=100 {{(pid=80245) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:17:41.304273 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e92c6836-31f9-412e-ac21-54af4c5eb97c None None] Validating token access rules against request {{(pid=80245) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:17:41.330027 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG dbcounter [-] [80244] Writing DB stats keystone:SELECT=100 {{(pid=80244) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:17:41.344769 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8da2508-f036-44f8-a62b-9fadfa2a4a6a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80245) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:17:41.345561 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8da2508-f036-44f8-a62b-9fadfa2a4a6a None admin] REQUEST_METHOD: `GET` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:17:41.351575 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8da2508-f036-44f8-a62b-9fadfa2a4a6a None admin] SCRIPT_NAME: `/identity` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:17:41.351717 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 424/849] 10.222.0.47 () {58 vars in 1023 bytes} [Thu Jul 21 08:17:41 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 21 08:17:41.351797 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8da2508-f036-44f8-a62b-9fadfa2a4a6a None admin] PATH_INFO: `/v3/users` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:17:41.351896 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e8d568d-36c5-4ef4-adb4-dccdac5c911e None None] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:17:41.351969 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8da2508-f036-44f8-a62b-9fadfa2a4a6a None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80245) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 08:17:41.351969 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8da2508-f036-44f8-a62b-9fadfa2a4a6a None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:17:41.351969 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8da2508-f036-44f8-a62b-9fadfa2a4a6a None admin] RBAC: Authorization granted {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:17:41.377965 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: [pid: 80245|app: 0|req: 426/850] 10.222.0.47 () {56 vars in 1149 bytes} [Thu Jul 21 08:17:41 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 08:17:41.378638 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8da2508-f036-44f8-a62b-9fadfa2a4a6a None admin] Authenticating user token {{(pid=80245) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:17:41.457028 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e8d568d-36c5-4ef4-adb4-dccdac5c911e None None] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:17:41.486828 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8da2508-f036-44f8-a62b-9fadfa2a4a6a None admin] Validating token access rules against request {{(pid=80245) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:17:41.498532 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74cac707-6354-4809-9931-37a7ea1fd56e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:17:41.499206 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74cac707-6354-4809-9931-37a7ea1fd56e None admin] REQUEST_METHOD: `GET` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:17:41.499523 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74cac707-6354-4809-9931-37a7ea1fd56e None admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:17:41.499802 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74cac707-6354-4809-9931-37a7ea1fd56e None admin] PATH_INFO: `/v3/roles` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:17:41.500562 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74cac707-6354-4809-9931-37a7ea1fd56e None admin] RBAC: Adding query filter params (name=admin) {{(pid=80244) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 08:17:41.500871 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74cac707-6354-4809-9931-37a7ea1fd56e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:17:41.502243 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74cac707-6354-4809-9931-37a7ea1fd56e None admin] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:17:41.511722 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 425/851] 10.222.0.47 () {56 vars in 1099 bytes} [Thu Jul 21 08:17:41 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 08:17:41.517065 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74cac707-6354-4809-9931-37a7ea1fd56e None admin] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:17:41.523204 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb8776dc-25e2-4a8e-ae3a-109e0564c6aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80245) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:17:41.524072 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb8776dc-25e2-4a8e-ae3a-109e0564c6aa None admin] REQUEST_METHOD: `GET` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:17:41.524360 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb8776dc-25e2-4a8e-ae3a-109e0564c6aa None admin] SCRIPT_NAME: `/identity` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:17:41.524637 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb8776dc-25e2-4a8e-ae3a-109e0564c6aa None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:17:41.525293 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb8776dc-25e2-4a8e-ae3a-109e0564c6aa None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:17:41.526660 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb8776dc-25e2-4a8e-ae3a-109e0564c6aa None admin] RBAC: Authorization granted {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:17:41.540286 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application [None req-bb8776dc-25e2-4a8e-ae3a-109e0564c6aa None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jul 21 08:17:41.540286 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 08:17:41.540286 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 21 08:17:41.540286 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 21 08:17:41.540286 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 21 08:17:41.540286 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 21 08:17:41.540286 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 21 08:17:41.540286 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 21 08:17:41.540286 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jul 21 08:17:41.540286 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application Jul 21 08:17:41.540286 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 21 08:17:41.540286 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application Jul 21 08:17:41.540286 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 08:17:41.540286 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 08:17:41.540286 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 08:17:41.540286 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 08:17:41.541266 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 08:17:41.541266 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 08:17:41.541266 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 08:17:41.541266 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 08:17:41.541266 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 08:17:41.541266 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 08:17:41.541266 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 08:17:41.541266 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 21 08:17:41.541266 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 21 08:17:41.541266 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 21 08:17:41.541266 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 21 08:17:41.541266 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 08:17:41.541266 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 08:17:41.541266 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 08:17:41.541266 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 08:17:41.541266 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 08:17:41.542091 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 08:17:41.542091 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 08:17:41.542091 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application with Lock( Jul 21 08:17:41.542091 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 08:17:41.542091 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application return self._enter() Jul 21 08:17:41.542091 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 08:17:41.542091 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 08:17:41.542091 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 08:17:41.542091 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application return self.creator() Jul 21 08:17:41.542091 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 08:17:41.542091 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application created_value = creator( Jul 21 08:17:41.542091 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 21 08:17:41.542091 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 21 08:17:41.542091 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jul 21 08:17:41.542091 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application Jul 21 08:17:41.542091 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: [pid: 80245|app: 0|req: 427/852] 10.222.0.47 () {56 vars in 1132 bytes} [Thu Jul 21 08:17:41 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 08:17:41.542997 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb8776dc-25e2-4a8e-ae3a-109e0564c6aa None admin] Authenticating user token {{(pid=80245) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:17:41.545817 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG dbcounter [-] [80245] Writing DB stats keystone:SELECT=100 {{(pid=80245) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:17:41.578001 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG dbcounter [-] [80244] Writing DB stats keystone:SELECT=100 {{(pid=80244) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:17:41.635527 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74cac707-6354-4809-9931-37a7ea1fd56e None admin] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:17:41.657864 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb8776dc-25e2-4a8e-ae3a-109e0564c6aa None admin] Validating token access rules against request {{(pid=80245) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:17:41.671984 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59c3b9c5-c354-4ef3-bdb7-b923a8924f82 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:17:41.672642 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59c3b9c5-c354-4ef3-bdb7-b923a8924f82 None admin] REQUEST_METHOD: `GET` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:17:41.672927 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59c3b9c5-c354-4ef3-bdb7-b923a8924f82 None admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:17:41.673206 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59c3b9c5-c354-4ef3-bdb7-b923a8924f82 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:17:41.698009 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-633788bc-3ae8-452c-b51e-24fbca6582b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80245) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:17:41.698805 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-633788bc-3ae8-452c-b51e-24fbca6582b5 None admin] REQUEST_METHOD: `GET` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:17:41.699086 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-633788bc-3ae8-452c-b51e-24fbca6582b5 None admin] SCRIPT_NAME: `/identity` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:17:41.699444 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-633788bc-3ae8-452c-b51e-24fbca6582b5 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:17:41.711221 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-633788bc-3ae8-452c-b51e-24fbca6582b5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:17:41.712636 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-633788bc-3ae8-452c-b51e-24fbca6582b5 None admin] RBAC: Authorization granted {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:17:41.730703 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application [None req-633788bc-3ae8-452c-b51e-24fbca6582b5 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 08:17:41.730703 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 08:17:41.730703 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 08:17:41.730703 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 08:17:41.730703 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 08:17:41.730703 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 08:17:41.730703 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 08:17:41.730703 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 08:17:41.730703 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 08:17:41.730703 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 08:17:41.730703 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 08:17:41.730703 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 08:17:41.730703 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 21 08:17:41.730703 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 21 08:17:41.730703 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 21 08:17:41.732055 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 21 08:17:41.732055 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 08:17:41.732055 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 08:17:41.732055 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 08:17:41.732055 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 08:17:41.732055 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 08:17:41.732055 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 08:17:41.732055 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 08:17:41.732055 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application with Lock( Jul 21 08:17:41.732055 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 08:17:41.732055 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application return self._enter() Jul 21 08:17:41.732055 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 08:17:41.732055 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 08:17:41.732055 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 08:17:41.732055 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application return self.creator() Jul 21 08:17:41.732055 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 08:17:41.732055 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application created_value = creator( Jul 21 08:17:41.733203 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 21 08:17:41.733203 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 21 08:17:41.733203 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 08:17:41.733203 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application Jul 21 08:17:41.736702 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: [pid: 80245|app: 0|req: 428/853] 10.222.0.47 () {56 vars in 1108 bytes} [Thu Jul 21 08:17:41 2022] GET /identity/v3/domains/Default => generated 87 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 08:17:41.737503 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-633788bc-3ae8-452c-b51e-24fbca6582b5 None admin] Authenticating user token {{(pid=80245) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:17:41.747172 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59c3b9c5-c354-4ef3-bdb7-b923a8924f82 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:17:41.748762 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59c3b9c5-c354-4ef3-bdb7-b923a8924f82 None admin] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:17:41.796275 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG dbcounter [-] [80245] Writing DB stats keystone:SELECT=100 {{(pid=80245) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:17:41.799793 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG dbcounter [-] [80244] Writing DB stats keystone:SELECT=100 {{(pid=80244) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:17:41.848666 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-633788bc-3ae8-452c-b51e-24fbca6582b5 None admin] Validating token access rules against request {{(pid=80245) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:17:41.861490 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 426/854] 10.222.0.47 () {58 vars in 1282 bytes} [Thu Jul 21 08:17:41 2022] GET /identity/v3/auth/tokens => generated 1151 bytes in 345 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 21 08:17:41.861490 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59c3b9c5-c354-4ef3-bdb7-b923a8924f82 None admin] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:17:41.887368 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10ff1eeb-d582-4c5f-8070-bdaa0ee142a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80245) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:17:41.888119 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10ff1eeb-d582-4c5f-8070-bdaa0ee142a9 None admin] REQUEST_METHOD: `GET` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:17:41.888473 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10ff1eeb-d582-4c5f-8070-bdaa0ee142a9 None admin] SCRIPT_NAME: `/identity` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:17:41.888830 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10ff1eeb-d582-4c5f-8070-bdaa0ee142a9 None admin] PATH_INFO: `/v3/services` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:17:41.889608 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10ff1eeb-d582-4c5f-8070-bdaa0ee142a9 None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=80245) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 08:17:41.889917 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10ff1eeb-d582-4c5f-8070-bdaa0ee142a9 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:17:41.891303 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10ff1eeb-d582-4c5f-8070-bdaa0ee142a9 None admin] RBAC: Authorization granted {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:17:41.903044 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: [pid: 80245|app: 0|req: 429/855] 10.222.0.47 () {56 vars in 1126 bytes} [Thu Jul 21 08:17:41 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 08:17:41.903044 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10ff1eeb-d582-4c5f-8070-bdaa0ee142a9 None admin] Authenticating user token {{(pid=80245) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:17:41.980149 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59c3b9c5-c354-4ef3-bdb7-b923a8924f82 None admin] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:17:42.013403 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10ff1eeb-d582-4c5f-8070-bdaa0ee142a9 None admin] Validating token access rules against request {{(pid=80245) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:17:42.015670 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c993d021-f35a-493b-99d7-90821d626a85 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:17:42.016347 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c993d021-f35a-493b-99d7-90821d626a85 None admin] REQUEST_METHOD: `GET` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:17:42.016629 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c993d021-f35a-493b-99d7-90821d626a85 None admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:17:42.016905 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c993d021-f35a-493b-99d7-90821d626a85 None admin] PATH_INFO: `/v3/domains` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:17:42.017630 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c993d021-f35a-493b-99d7-90821d626a85 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80244) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 08:17:42.017936 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c993d021-f35a-493b-99d7-90821d626a85 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:17:42.019360 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c993d021-f35a-493b-99d7-90821d626a85 None admin] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:17:42.031516 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 427/856] 10.222.0.47 () {56 vars in 1109 bytes} [Thu Jul 21 08:17:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 08:17:42.032353 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c993d021-f35a-493b-99d7-90821d626a85 None admin] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:17:42.037866 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG dbcounter [-] [80245] Writing DB stats keystone:SELECT=100 {{(pid=80245) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:17:42.046916 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG dbcounter [-] [80244] Writing DB stats keystone:SELECT=100 {{(pid=80244) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:17:42.057194 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af307f53-e69c-47ed-80f8-91861c69b893 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80245) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:17:42.058002 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af307f53-e69c-47ed-80f8-91861c69b893 None admin] REQUEST_METHOD: `GET` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:17:42.058283 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af307f53-e69c-47ed-80f8-91861c69b893 None admin] SCRIPT_NAME: `/identity` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:17:42.058561 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af307f53-e69c-47ed-80f8-91861c69b893 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:17:42.070775 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af307f53-e69c-47ed-80f8-91861c69b893 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:17:42.072267 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af307f53-e69c-47ed-80f8-91861c69b893 None admin] RBAC: Authorization granted {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:17:42.090788 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application [None req-af307f53-e69c-47ed-80f8-91861c69b893 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 08:17:42.090788 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 08:17:42.090788 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 08:17:42.090788 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 08:17:42.090788 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 08:17:42.090788 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 08:17:42.090788 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 08:17:42.090788 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 08:17:42.090788 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 08:17:42.090788 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 08:17:42.090788 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 08:17:42.090788 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 08:17:42.090788 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 21 08:17:42.090788 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 21 08:17:42.090788 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 21 08:17:42.091841 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 21 08:17:42.091841 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 08:17:42.091841 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 08:17:42.091841 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 08:17:42.091841 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 08:17:42.091841 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 08:17:42.091841 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 08:17:42.091841 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 08:17:42.091841 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application with Lock( Jul 21 08:17:42.091841 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 08:17:42.091841 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application return self._enter() Jul 21 08:17:42.091841 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 08:17:42.091841 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 08:17:42.091841 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 08:17:42.091841 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application return self.creator() Jul 21 08:17:42.091841 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 08:17:42.091841 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application created_value = creator( Jul 21 08:17:42.092959 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 21 08:17:42.092959 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 21 08:17:42.092959 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 08:17:42.092959 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application Jul 21 08:17:42.092959 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: [pid: 80245|app: 0|req: 430/857] 10.222.0.47 () {56 vars in 1108 bytes} [Thu Jul 21 08:17:41 2022] GET /identity/v3/domains/Default => generated 87 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 08:17:42.094647 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af307f53-e69c-47ed-80f8-91861c69b893 None admin] Authenticating user token {{(pid=80245) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:17:42.148758 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c993d021-f35a-493b-99d7-90821d626a85 None admin] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:17:42.189961 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fa1bd27-865d-4a2b-9567-d253926588d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:17:42.192834 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fa1bd27-865d-4a2b-9567-d253926588d4 None admin] REQUEST_METHOD: `GET` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:17:42.193131 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fa1bd27-865d-4a2b-9567-d253926588d4 None admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:17:42.193398 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fa1bd27-865d-4a2b-9567-d253926588d4 None admin] PATH_INFO: `/v3/services` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:17:42.194116 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fa1bd27-865d-4a2b-9567-d253926588d4 None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=80244) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 08:17:42.194424 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fa1bd27-865d-4a2b-9567-d253926588d4 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:17:42.195978 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fa1bd27-865d-4a2b-9567-d253926588d4 None admin] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:17:42.211354 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 428/858] 10.222.0.47 () {56 vars in 1126 bytes} [Thu Jul 21 08:17:42 2022] GET /identity/v3/services?type=compute_legacy => generated 131 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 08:17:42.211354 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fa1bd27-865d-4a2b-9567-d253926588d4 None admin] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:17:42.221313 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af307f53-e69c-47ed-80f8-91861c69b893 None admin] Validating token access rules against request {{(pid=80245) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:17:42.276104 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58217ce6-4be8-4d60-98be-4b2d25d9d6db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80245) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:17:42.276842 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58217ce6-4be8-4d60-98be-4b2d25d9d6db None admin] REQUEST_METHOD: `GET` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:17:42.277126 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58217ce6-4be8-4d60-98be-4b2d25d9d6db None admin] SCRIPT_NAME: `/identity` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:17:42.277524 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58217ce6-4be8-4d60-98be-4b2d25d9d6db None admin] PATH_INFO: `/v3/users/swift` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:17:42.286776 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58217ce6-4be8-4d60-98be-4b2d25d9d6db None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:17:42.288286 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58217ce6-4be8-4d60-98be-4b2d25d9d6db None admin] RBAC: Authorization granted {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:17:42.305203 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application [None req-58217ce6-4be8-4d60-98be-4b2d25d9d6db None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Jul 21 08:17:42.305203 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 08:17:42.305203 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 08:17:42.305203 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 08:17:42.305203 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 08:17:42.305203 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 08:17:42.305203 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 08:17:42.305203 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 08:17:42.305203 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 08:17:42.305203 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 08:17:42.305203 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 08:17:42.305203 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 08:17:42.305203 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 21 08:17:42.305203 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 21 08:17:42.305203 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 21 08:17:42.305203 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 21 08:17:42.306230 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 08:17:42.306230 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 08:17:42.306230 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 21 08:17:42.306230 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 21 08:17:42.306230 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 21 08:17:42.306230 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 21 08:17:42.306230 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 08:17:42.306230 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 08:17:42.306230 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 08:17:42.306230 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 08:17:42.306230 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 08:17:42.306230 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application with Lock( Jul 21 08:17:42.306230 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 08:17:42.306230 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application return self._enter() Jul 21 08:17:42.306230 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 08:17:42.306230 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 08:17:42.306230 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 08:17:42.306230 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application return self.creator() Jul 21 08:17:42.307177 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 08:17:42.307177 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application created_value = creator( Jul 21 08:17:42.307177 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 21 08:17:42.307177 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 21 08:17:42.307177 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 21 08:17:42.307177 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 21 08:17:42.307177 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 21 08:17:42.307177 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 21 08:17:42.307177 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Jul 21 08:17:42.307177 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application Jul 21 08:17:42.311475 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: [pid: 80245|app: 0|req: 431/859] 10.222.0.47 () {56 vars in 1096 bytes} [Thu Jul 21 08:17:42 2022] GET /identity/v3/users/swift => generated 83 bytes in 218 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 08:17:42.312506 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58217ce6-4be8-4d60-98be-4b2d25d9d6db None admin] Authenticating user token {{(pid=80245) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:17:42.321830 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG dbcounter [-] [80244] Writing DB stats keystone:SELECT=100 {{(pid=80244) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:17:42.350532 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG dbcounter [-] [80245] Writing DB stats keystone:SELECT=100 {{(pid=80245) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:17:42.360370 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fa1bd27-865d-4a2b-9567-d253926588d4 None admin] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:17:42.411888 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15d69b53-ce54-4a39-9853-ebb4a4a7f670 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:17:42.412866 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15d69b53-ce54-4a39-9853-ebb4a4a7f670 None admin] REQUEST_METHOD: `GET` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:17:42.413315 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15d69b53-ce54-4a39-9853-ebb4a4a7f670 None admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:17:42.413753 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15d69b53-ce54-4a39-9853-ebb4a4a7f670 None admin] PATH_INFO: `/v3/domains` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:17:42.414488 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15d69b53-ce54-4a39-9853-ebb4a4a7f670 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80244) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 08:17:42.414796 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15d69b53-ce54-4a39-9853-ebb4a4a7f670 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:17:42.416236 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15d69b53-ce54-4a39-9853-ebb4a4a7f670 None admin] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:17:42.432369 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 429/860] 10.222.0.47 () {56 vars in 1109 bytes} [Thu Jul 21 08:17:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 227 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 08:17:42.435346 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58217ce6-4be8-4d60-98be-4b2d25d9d6db None admin] Validating token access rules against request {{(pid=80245) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:17:42.436925 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15d69b53-ce54-4a39-9853-ebb4a4a7f670 None admin] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:17:42.474769 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83faaf87-0e7e-4c58-9606-f477e10b6f95 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80245) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:17:42.475666 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83faaf87-0e7e-4c58-9606-f477e10b6f95 None admin] REQUEST_METHOD: `GET` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:17:42.476043 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83faaf87-0e7e-4c58-9606-f477e10b6f95 None admin] SCRIPT_NAME: `/identity` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:17:42.476411 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83faaf87-0e7e-4c58-9606-f477e10b6f95 None admin] PATH_INFO: `/v3/users` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:17:42.477350 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83faaf87-0e7e-4c58-9606-f477e10b6f95 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80245) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 08:17:42.477751 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83faaf87-0e7e-4c58-9606-f477e10b6f95 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:17:42.479289 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83faaf87-0e7e-4c58-9606-f477e10b6f95 None admin] RBAC: Authorization granted {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:17:42.499287 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: [pid: 80245|app: 0|req: 432/861] 10.222.0.47 () {56 vars in 1135 bytes} [Thu Jul 21 08:17:42 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 08:17:42.506501 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83faaf87-0e7e-4c58-9606-f477e10b6f95 None admin] Authenticating user token {{(pid=80245) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:17:42.580424 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15d69b53-ce54-4a39-9853-ebb4a4a7f670 None admin] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:17:42.614177 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG dbcounter [-] [80245] Writing DB stats keystone:SELECT=100 {{(pid=80245) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:17:42.617604 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee8de724-fe88-4889-9fac-39545cb6ce39 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:17:42.618410 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee8de724-fe88-4889-9fac-39545cb6ce39 None admin] REQUEST_METHOD: `GET` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:17:42.618692 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee8de724-fe88-4889-9fac-39545cb6ce39 None admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:17:42.619192 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee8de724-fe88-4889-9fac-39545cb6ce39 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:17:42.621157 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG dbcounter [-] [80244] Writing DB stats keystone:SELECT=100 {{(pid=80244) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:17:42.627965 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee8de724-fe88-4889-9fac-39545cb6ce39 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:17:42.629511 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83faaf87-0e7e-4c58-9606-f477e10b6f95 None admin] Validating token access rules against request {{(pid=80245) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:17:42.630378 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee8de724-fe88-4889-9fac-39545cb6ce39 None admin] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:17:42.647057 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application [None req-ee8de724-fe88-4889-9fac-39545cb6ce39 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 21 08:17:42.647057 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 08:17:42.647057 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 08:17:42.647057 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 08:17:42.647057 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 08:17:42.647057 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 08:17:42.647057 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 08:17:42.647057 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 08:17:42.647057 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 08:17:42.647057 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 08:17:42.647057 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 08:17:42.647057 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 08:17:42.647057 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 21 08:17:42.647057 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 21 08:17:42.647057 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 21 08:17:42.648212 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 21 08:17:42.648212 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 08:17:42.648212 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 08:17:42.648212 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 08:17:42.648212 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 08:17:42.648212 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 08:17:42.648212 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 08:17:42.648212 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 08:17:42.648212 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application with Lock( Jul 21 08:17:42.648212 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 08:17:42.648212 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self._enter() Jul 21 08:17:42.648212 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 08:17:42.648212 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 08:17:42.648212 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 08:17:42.648212 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self.creator() Jul 21 08:17:42.648212 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 08:17:42.648212 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application created_value = creator( Jul 21 08:17:42.649378 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 21 08:17:42.649378 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 21 08:17:42.649378 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 21 08:17:42.649378 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 21 08:17:42.649378 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 21 08:17:42.649378 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 21 08:17:42.649378 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 21 08:17:42.649378 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application Jul 21 08:17:42.653576 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 430/862] 10.222.0.47 () {56 vars in 1111 bytes} [Thu Jul 21 08:17:42 2022] GET /identity/v3/projects/service => generated 88 bytes in 216 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 21 08:17:42.654260 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1a59f3d-ffc7-47e0-810e-f3c0df4c40a4 None None] REQUEST_METHOD: `GET` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:17:42.654549 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1a59f3d-ffc7-47e0-810e-f3c0df4c40a4 None None] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:17:42.654830 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1a59f3d-ffc7-47e0-810e-f3c0df4c40a4 None None] PATH_INFO: `/` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:17:42.659547 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 431/863] 10.222.0.47 () {54 vars in 926 bytes} [Thu Jul 21 08:17:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 08:17:42.659547 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1a59f3d-ffc7-47e0-810e-f3c0df4c40a4 None None] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:17:42.671207 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e1949a1-8f35-4557-83b4-de3f5987d5d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80245) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:17:42.672017 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e1949a1-8f35-4557-83b4-de3f5987d5d9 None admin] REQUEST_METHOD: `GET` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:17:42.672377 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e1949a1-8f35-4557-83b4-de3f5987d5d9 None admin] SCRIPT_NAME: `/identity` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:17:42.672719 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e1949a1-8f35-4557-83b4-de3f5987d5d9 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:17:42.744933 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e1949a1-8f35-4557-83b4-de3f5987d5d9 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:17:42.746524 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e1949a1-8f35-4557-83b4-de3f5987d5d9 None admin] RBAC: Authorization granted {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:17:42.773079 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1a59f3d-ffc7-47e0-810e-f3c0df4c40a4 None None] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:17:42.812536 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd36c70a-62e8-4620-bbff-10d9927b4b66 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:17:42.813319 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd36c70a-62e8-4620-bbff-10d9927b4b66 None admin] REQUEST_METHOD: `GET` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:17:42.813501 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd36c70a-62e8-4620-bbff-10d9927b4b66 None admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:17:42.813766 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd36c70a-62e8-4620-bbff-10d9927b4b66 None admin] PATH_INFO: `/v3/projects` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:17:42.814560 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd36c70a-62e8-4620-bbff-10d9927b4b66 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80244) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 08:17:42.814872 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd36c70a-62e8-4620-bbff-10d9927b4b66 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:17:42.817100 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd36c70a-62e8-4620-bbff-10d9927b4b66 None admin] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:17:42.829334 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 432/864] 10.222.0.47 () {56 vars in 1148 bytes} [Thu Jul 21 08:17:42 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 08:17:42.830650 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-661e11d6-f0b4-4e02-8038-ea31534eb526 None None] REQUEST_METHOD: `POST` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:17:42.830885 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-661e11d6-f0b4-4e02-8038-ea31534eb526 None None] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:17:42.831164 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-661e11d6-f0b4-4e02-8038-ea31534eb526 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:17:42.839764 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG dbcounter [-] [80245] Writing DB stats keystone:SELECT=100 {{(pid=80245) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:17:42.850510 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.auth.core [None req-661e11d6-f0b4-4e02-8038-ea31534eb526 None None] MFA Rules not processed for user `2dce40b8d74c41ec94cc36e6ff05d412`. Rule list: `[]` (Enabled: `True`). {{(pid=80244) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 08:17:42.851731 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.fernet_utils [None req-661e11d6-f0b4-4e02-8038-ea31534eb526 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80244) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 08:17:42.863057 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: [pid: 80245|app: 0|req: 433/865] 10.222.0.47 () {58 vars in 1282 bytes} [Thu Jul 21 08:17:42 2022] GET /identity/v3/auth/tokens => generated 1151 bytes in 352 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 21 08:17:42.863057 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e1949a1-8f35-4557-83b4-de3f5987d5d9 None admin] Authenticating user token {{(pid=80245) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:17:42.910158 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG dbcounter [-] [80244] Writing DB stats keystone:SELECT=100 {{(pid=80244) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:17:42.931494 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 433/866] 10.222.0.47 () {58 vars in 1023 bytes} [Thu Jul 21 08:17:42 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 21 08:17:42.931494 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-661e11d6-f0b4-4e02-8038-ea31534eb526 None None] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:17:42.979339 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e1949a1-8f35-4557-83b4-de3f5987d5d9 None admin] Validating token access rules against request {{(pid=80245) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:17:43.017157 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1cf5bb8-0e63-47ef-89c7-44285d48b817 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80245) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:17:43.017820 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1cf5bb8-0e63-47ef-89c7-44285d48b817 None admin] REQUEST_METHOD: `GET` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:17:43.018484 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1cf5bb8-0e63-47ef-89c7-44285d48b817 None admin] SCRIPT_NAME: `/identity` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:17:43.018777 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1cf5bb8-0e63-47ef-89c7-44285d48b817 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:17:43.020235 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1cf5bb8-0e63-47ef-89c7-44285d48b817 None admin] RBAC: Adding query filter params (role.id=50d1ec7f947f42acb65c57e07f2269d6, scope.project.id=4faef13aa50e4e36b6c265fb1398fa14, user.id=2ef7f1e91e554a1f88eb6a63e60a475f) {{(pid=80245) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 08:17:43.021042 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1cf5bb8-0e63-47ef-89c7-44285d48b817 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:17:43.022926 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1cf5bb8-0e63-47ef-89c7-44285d48b817 None admin] RBAC: Authorization granted {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:17:43.029045 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: [pid: 80245|app: 0|req: 434/867] 10.222.0.47 () {56 vars in 1374 bytes} [Thu Jul 21 08:17:42 2022] GET /identity/v3/role_assignments?user.id=2ef7f1e91e554a1f88eb6a63e60a475f&scope.project.id=4faef13aa50e4e36b6c265fb1398fa14&role.id=50d1ec7f947f42acb65c57e07f2269d6 => generated 608 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 08:17:43.030075 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1cf5bb8-0e63-47ef-89c7-44285d48b817 None admin] Authenticating user token {{(pid=80245) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:17:43.049289 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-661e11d6-f0b4-4e02-8038-ea31534eb526 None None] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:17:43.091652 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32253042-b5e1-47cd-955e-1a0d7a48991c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:17:43.092328 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32253042-b5e1-47cd-955e-1a0d7a48991c None admin] REQUEST_METHOD: `GET` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:17:43.092615 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32253042-b5e1-47cd-955e-1a0d7a48991c None admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:17:43.092894 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32253042-b5e1-47cd-955e-1a0d7a48991c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:17:43.105034 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32253042-b5e1-47cd-955e-1a0d7a48991c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:17:43.106561 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32253042-b5e1-47cd-955e-1a0d7a48991c None admin] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:17:43.117094 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG dbcounter [-] [80245] Writing DB stats keystone:SELECT=100 {{(pid=80245) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:17:43.131794 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application [None req-32253042-b5e1-47cd-955e-1a0d7a48991c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 08:17:43.131794 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 08:17:43.131794 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 08:17:43.131794 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 08:17:43.131794 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 08:17:43.131794 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 08:17:43.131794 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 08:17:43.131794 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 08:17:43.131794 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 08:17:43.131794 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 08:17:43.131794 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 08:17:43.131794 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 08:17:43.131794 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 21 08:17:43.131794 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 21 08:17:43.131794 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 21 08:17:43.132762 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 21 08:17:43.132762 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 08:17:43.132762 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 08:17:43.132762 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 08:17:43.132762 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 08:17:43.132762 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 08:17:43.132762 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 08:17:43.132762 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 08:17:43.132762 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application with Lock( Jul 21 08:17:43.132762 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 08:17:43.132762 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self._enter() Jul 21 08:17:43.132762 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 08:17:43.132762 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 08:17:43.132762 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 08:17:43.132762 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self.creator() Jul 21 08:17:43.132762 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 08:17:43.132762 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application created_value = creator( Jul 21 08:17:43.133915 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 21 08:17:43.133915 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 21 08:17:43.133915 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 08:17:43.133915 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application Jul 21 08:17:43.136294 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 434/868] 10.222.0.47 () {56 vars in 1108 bytes} [Thu Jul 21 08:17:42 2022] GET /identity/v3/domains/Default => generated 87 bytes in 207 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 08:17:43.147388 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32253042-b5e1-47cd-955e-1a0d7a48991c None admin] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:17:43.179670 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1cf5bb8-0e63-47ef-89c7-44285d48b817 None admin] Validating token access rules against request {{(pid=80245) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:17:43.182366 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG dbcounter [-] [80244] Writing DB stats keystone:SELECT=100 {{(pid=80244) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:17:43.233793 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f902d36-c269-4e16-b25b-271763f5b236 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80245) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:17:43.234476 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f902d36-c269-4e16-b25b-271763f5b236 None admin] REQUEST_METHOD: `POST` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:17:43.234761 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f902d36-c269-4e16-b25b-271763f5b236 None admin] SCRIPT_NAME: `/identity` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:17:43.235044 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f902d36-c269-4e16-b25b-271763f5b236 None admin] PATH_INFO: `/v3/services` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:17:43.235933 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f902d36-c269-4e16-b25b-271763f5b236 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:17:43.237313 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f902d36-c269-4e16-b25b-271763f5b236 None admin] RBAC: Authorization granted {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:17:43.259385 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: [pid: 80245|app: 0|req: 435/869] 10.222.0.47 () {60 vars in 1141 bytes} [Thu Jul 21 08:17:43 2022] POST /identity/v3/services => generated 251 bytes in 226 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 08:17:43.270996 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32253042-b5e1-47cd-955e-1a0d7a48991c None admin] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:17:43.327426 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-283fc56a-7670-473d-af03-fd229c7f3fe3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:17:43.328168 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-283fc56a-7670-473d-af03-fd229c7f3fe3 None admin] REQUEST_METHOD: `GET` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:17:43.328451 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-283fc56a-7670-473d-af03-fd229c7f3fe3 None admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:17:43.328725 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-283fc56a-7670-473d-af03-fd229c7f3fe3 None admin] PATH_INFO: `/v3/domains` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:17:43.329506 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-283fc56a-7670-473d-af03-fd229c7f3fe3 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80244) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 08:17:43.329841 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-283fc56a-7670-473d-af03-fd229c7f3fe3 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:17:43.331276 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-283fc56a-7670-473d-af03-fd229c7f3fe3 None admin] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:17:43.346077 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 435/870] 10.222.0.47 () {56 vars in 1109 bytes} [Thu Jul 21 08:17:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 08:17:43.351912 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f902d36-c269-4e16-b25b-271763f5b236 None admin] Authenticating user token {{(pid=80245) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:17:43.465296 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f902d36-c269-4e16-b25b-271763f5b236 None admin] Validating token access rules against request {{(pid=80245) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:17:43.481478 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG dbcounter [-] [80245] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80245) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:17:43.507050 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba9f0d84-9033-47be-98a0-39931c29c2a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80245) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:17:43.507954 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba9f0d84-9033-47be-98a0-39931c29c2a5 None admin] REQUEST_METHOD: `GET` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:17:43.508491 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba9f0d84-9033-47be-98a0-39931c29c2a5 None admin] SCRIPT_NAME: `/identity` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:17:43.508766 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba9f0d84-9033-47be-98a0-39931c29c2a5 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:17:43.519721 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba9f0d84-9033-47be-98a0-39931c29c2a5 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:17:43.521159 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba9f0d84-9033-47be-98a0-39931c29c2a5 None admin] RBAC: Authorization granted {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:17:43.536069 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application [None req-ba9f0d84-9033-47be-98a0-39931c29c2a5 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 21 08:17:43.536069 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 08:17:43.536069 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 08:17:43.536069 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 08:17:43.536069 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 08:17:43.536069 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 08:17:43.536069 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 08:17:43.536069 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 08:17:43.536069 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 08:17:43.536069 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 08:17:43.536069 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 08:17:43.536069 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 08:17:43.536069 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 21 08:17:43.536069 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 21 08:17:43.536069 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 21 08:17:43.536972 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 21 08:17:43.536972 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 08:17:43.536972 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 08:17:43.536972 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 08:17:43.536972 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 08:17:43.536972 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 08:17:43.536972 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 08:17:43.536972 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 08:17:43.536972 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application with Lock( Jul 21 08:17:43.536972 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 08:17:43.536972 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application return self._enter() Jul 21 08:17:43.536972 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 08:17:43.536972 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 08:17:43.536972 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 08:17:43.536972 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application return self.creator() Jul 21 08:17:43.536972 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 08:17:43.536972 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application created_value = creator( Jul 21 08:17:43.538246 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 21 08:17:43.538246 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 21 08:17:43.538246 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 21 08:17:43.538246 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 21 08:17:43.538246 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 21 08:17:43.538246 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 21 08:17:43.538246 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 21 08:17:43.538246 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application Jul 21 08:17:43.538246 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: [pid: 80245|app: 0|req: 436/871] 10.222.0.47 () {56 vars in 1111 bytes} [Thu Jul 21 08:17:43 2022] GET /identity/v3/projects/service => generated 88 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 08:17:43.544668 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-283fc56a-7670-473d-af03-fd229c7f3fe3 None admin] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:17:43.640780 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG dbcounter [-] [80244] Writing DB stats keystone:SELECT=100 {{(pid=80244) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:17:43.654632 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-283fc56a-7670-473d-af03-fd229c7f3fe3 None admin] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:17:43.696411 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f0715d5-2931-4858-9640-69475adaeb4e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:17:43.697088 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f0715d5-2931-4858-9640-69475adaeb4e None admin] REQUEST_METHOD: `GET` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:17:43.697372 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f0715d5-2931-4858-9640-69475adaeb4e None admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:17:43.697900 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f0715d5-2931-4858-9640-69475adaeb4e None admin] PATH_INFO: `/v3/projects` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:17:43.698447 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f0715d5-2931-4858-9640-69475adaeb4e None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80244) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 08:17:43.698756 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f0715d5-2931-4858-9640-69475adaeb4e None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:17:43.700231 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f0715d5-2931-4858-9640-69475adaeb4e None admin] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:17:43.712829 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 436/872] 10.222.0.47 () {56 vars in 1148 bytes} [Thu Jul 21 08:17:43 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 08:17:43.718824 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba9f0d84-9033-47be-98a0-39931c29c2a5 None admin] Authenticating user token {{(pid=80245) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:17:43.829192 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba9f0d84-9033-47be-98a0-39931c29c2a5 None admin] Validating token access rules against request {{(pid=80245) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:17:43.865342 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fc214c5-2209-43ab-b69c-2b3b366c6860 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80245) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:17:43.866007 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fc214c5-2209-43ab-b69c-2b3b366c6860 None admin] REQUEST_METHOD: `GET` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:17:43.866305 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fc214c5-2209-43ab-b69c-2b3b366c6860 None admin] SCRIPT_NAME: `/identity` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:17:43.866565 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fc214c5-2209-43ab-b69c-2b3b366c6860 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:17:43.867414 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fc214c5-2209-43ab-b69c-2b3b366c6860 None admin] RBAC: Adding query filter params (role.id=e29870f6e9e2411a9db0d5bf45f32ca5, scope.project.id=4faef13aa50e4e36b6c265fb1398fa14, user.id=302351411fc84248a6b166b25d21609f) {{(pid=80245) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 08:17:43.867723 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fc214c5-2209-43ab-b69c-2b3b366c6860 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:17:43.869155 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fc214c5-2209-43ab-b69c-2b3b366c6860 None admin] RBAC: Authorization granted {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:17:43.875851 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: [pid: 80245|app: 0|req: 437/873] 10.222.0.47 () {56 vars in 1374 bytes} [Thu Jul 21 08:17:43 2022] GET /identity/v3/role_assignments?user.id=302351411fc84248a6b166b25d21609f&scope.project.id=4faef13aa50e4e36b6c265fb1398fa14&role.id=e29870f6e9e2411a9db0d5bf45f32ca5 => generated 608 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 08:17:44.360914 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67df74a2-43ea-4dc6-9a8d-b6d4678d8138 None None] REQUEST_METHOD: `GET` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:17:44.361455 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67df74a2-43ea-4dc6-9a8d-b6d4678d8138 None None] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:17:44.361455 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67df74a2-43ea-4dc6-9a8d-b6d4678d8138 None None] PATH_INFO: `/` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:17:44.363130 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 437/874] 10.222.0.47 () {54 vars in 926 bytes} [Thu Jul 21 08:17:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 08:17:44.368658 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43aa610f-d111-4557-b9b7-3da60b2a7cc3 None None] REQUEST_METHOD: `POST` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:17:44.369107 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43aa610f-d111-4557-b9b7-3da60b2a7cc3 None None] SCRIPT_NAME: `/identity` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:17:44.369192 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43aa610f-d111-4557-b9b7-3da60b2a7cc3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:17:44.387879 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.auth.core [None req-43aa610f-d111-4557-b9b7-3da60b2a7cc3 None None] MFA Rules not processed for user `2dce40b8d74c41ec94cc36e6ff05d412`. Rule list: `[]` (Enabled: `True`). {{(pid=80245) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 08:17:44.389102 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.fernet_utils [None req-43aa610f-d111-4557-b9b7-3da60b2a7cc3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80245) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 08:17:44.420507 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG dbcounter [-] [80245] Writing DB stats keystone:SELECT=100 {{(pid=80245) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:17:44.466769 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: [pid: 80245|app: 0|req: 438/875] 10.222.0.47 () {58 vars in 1023 bytes} [Thu Jul 21 08:17:44 2022] POST /identity/v3/auth/tokens => generated 1252 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 21 08:17:44.490738 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67df74a2-43ea-4dc6-9a8d-b6d4678d8138 None None] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:17:44.600739 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67df74a2-43ea-4dc6-9a8d-b6d4678d8138 None None] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:17:44.637606 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7669130-fe97-4222-a407-6b93e8fdbb86 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:17:44.638202 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7669130-fe97-4222-a407-6b93e8fdbb86 None admin] REQUEST_METHOD: `POST` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:17:44.638489 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7669130-fe97-4222-a407-6b93e8fdbb86 None admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:17:44.638768 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7669130-fe97-4222-a407-6b93e8fdbb86 None admin] PATH_INFO: `/v3/services` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:17:44.639588 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7669130-fe97-4222-a407-6b93e8fdbb86 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:17:44.640971 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7669130-fe97-4222-a407-6b93e8fdbb86 None admin] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:17:44.656300 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 438/876] 10.222.0.47 () {60 vars in 1141 bytes} [Thu Jul 21 08:17:44 2022] POST /identity/v3/services => generated 269 bytes in 166 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 08:17:45.367650 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd53d43c-b825-43c6-9b0d-e0b9c2345d35 None None] REQUEST_METHOD: `GET` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:17:45.368069 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd53d43c-b825-43c6-9b0d-e0b9c2345d35 None None] SCRIPT_NAME: `/identity` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:17:45.368247 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd53d43c-b825-43c6-9b0d-e0b9c2345d35 None None] PATH_INFO: `/` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:17:45.369897 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: [pid: 80245|app: 0|req: 439/877] 10.222.0.47 () {54 vars in 926 bytes} [Thu Jul 21 08:17:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 08:17:45.375626 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b660ede-3697-41ab-8d79-9d0e0c6462e0 None None] REQUEST_METHOD: `POST` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:17:45.376119 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b660ede-3697-41ab-8d79-9d0e0c6462e0 None None] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:17:45.376208 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b660ede-3697-41ab-8d79-9d0e0c6462e0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:17:45.394778 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.auth.core [None req-8b660ede-3697-41ab-8d79-9d0e0c6462e0 None None] MFA Rules not processed for user `2dce40b8d74c41ec94cc36e6ff05d412`. Rule list: `[]` (Enabled: `True`). {{(pid=80244) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 08:17:45.396018 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.fernet_utils [None req-8b660ede-3697-41ab-8d79-9d0e0c6462e0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80244) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 08:17:45.404063 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG dbcounter [-] [80244] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80244) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:17:45.420515 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e2fa49f-caae-4673-b159-d6d8637fe7d6 None None] REQUEST_METHOD: `GET` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:17:45.420865 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e2fa49f-caae-4673-b159-d6d8637fe7d6 None None] SCRIPT_NAME: `/identity` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:17:45.421067 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e2fa49f-caae-4673-b159-d6d8637fe7d6 None None] PATH_INFO: `/` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:17:45.422696 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: [pid: 80245|app: 0|req: 440/878] 10.222.0.47 () {54 vars in 926 bytes} [Thu Jul 21 08:17:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 21 08:17:45.427765 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd392f3a-dc34-4b78-989d-a1a7a94554d9 None None] REQUEST_METHOD: `POST` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:17:45.428049 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd392f3a-dc34-4b78-989d-a1a7a94554d9 None None] SCRIPT_NAME: `/identity` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:17:45.428325 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd392f3a-dc34-4b78-989d-a1a7a94554d9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:17:45.445547 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.auth.core [None req-fd392f3a-dc34-4b78-989d-a1a7a94554d9 None None] MFA Rules not processed for user `2dce40b8d74c41ec94cc36e6ff05d412`. Rule list: `[]` (Enabled: `True`). {{(pid=80245) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 08:17:45.446829 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.fernet_utils [None req-fd392f3a-dc34-4b78-989d-a1a7a94554d9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80245) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 08:17:45.474312 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 439/879] 10.222.0.47 () {58 vars in 1023 bytes} [Thu Jul 21 08:17:45 2022] POST /identity/v3/auth/tokens => generated 1362 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 21 08:17:45.501027 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b660ede-3697-41ab-8d79-9d0e0c6462e0 None None] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:17:45.529751 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: [pid: 80245|app: 0|req: 441/880] 10.222.0.47 () {58 vars in 1023 bytes} [Thu Jul 21 08:17:45 2022] POST /identity/v3/auth/tokens => generated 1362 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 21 08:17:45.554139 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd392f3a-dc34-4b78-989d-a1a7a94554d9 None None] Authenticating user token {{(pid=80245) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:17:45.607032 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b660ede-3697-41ab-8d79-9d0e0c6462e0 None None] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:17:45.642543 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee2dee91-6217-409e-9faa-8ec702eec2cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:17:45.643388 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee2dee91-6217-409e-9faa-8ec702eec2cb None admin] REQUEST_METHOD: `GET` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:17:45.643681 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee2dee91-6217-409e-9faa-8ec702eec2cb None admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:17:45.644495 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee2dee91-6217-409e-9faa-8ec702eec2cb None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:17:45.651829 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG dbcounter [-] [80244] Writing DB stats keystone:SELECT=100 {{(pid=80244) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:17:45.655839 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee2dee91-6217-409e-9faa-8ec702eec2cb None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 21 08:17:45.664660 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG dbcounter [-] [80245] Writing DB stats keystone:SELECT=100 {{(pid=80245) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:17:45.665388 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application [None req-ee2dee91-6217-409e-9faa-8ec702eec2cb None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 21 08:17:45.665388 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 08:17:45.665388 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 08:17:45.665388 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 08:17:45.665388 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 08:17:45.665388 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 08:17:45.665388 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 08:17:45.665388 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 08:17:45.665388 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 08:17:45.665388 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 08:17:45.665388 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 08:17:45.665388 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 08:17:45.665388 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 21 08:17:45.665388 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 21 08:17:45.665388 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 21 08:17:45.665388 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 21 08:17:45.666548 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 21 08:17:45.666548 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 21 08:17:45.666548 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 21 08:17:45.666548 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 21 08:17:45.666548 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 08:17:45.666548 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 08:17:45.666548 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 08:17:45.666548 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 08:17:45.666548 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 08:17:45.666548 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 08:17:45.666548 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 08:17:45.666548 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application with Lock( Jul 21 08:17:45.666548 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 08:17:45.666548 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self._enter() Jul 21 08:17:45.666548 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 08:17:45.666548 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 08:17:45.667643 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 08:17:45.667643 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self.creator() Jul 21 08:17:45.667643 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 08:17:45.667643 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application created_value = creator( Jul 21 08:17:45.667643 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 21 08:17:45.667643 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 21 08:17:45.667643 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 21 08:17:45.667643 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 21 08:17:45.667643 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 21 08:17:45.667643 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 21 08:17:45.667643 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 21 08:17:45.667643 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application Jul 21 08:17:45.674339 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 440/881] 10.222.0.47 () {56 vars in 1120 bytes} [Thu Jul 21 08:17:45 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 08:17:45.675705 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee2dee91-6217-409e-9faa-8ec702eec2cb None admin] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:17:45.676122 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd392f3a-dc34-4b78-989d-a1a7a94554d9 None None] Validating token access rules against request {{(pid=80245) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:17:45.713582 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db4ca41b-4271-47e6-86c8-7ba81e10c651 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80245) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:17:45.714659 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db4ca41b-4271-47e6-86c8-7ba81e10c651 None admin] REQUEST_METHOD: `GET` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:17:45.715093 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db4ca41b-4271-47e6-86c8-7ba81e10c651 None admin] SCRIPT_NAME: `/identity` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:17:45.715539 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db4ca41b-4271-47e6-86c8-7ba81e10c651 None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:17:45.716356 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db4ca41b-4271-47e6-86c8-7ba81e10c651 None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:17:45.717995 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db4ca41b-4271-47e6-86c8-7ba81e10c651 None admin] RBAC: Authorization granted {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:17:45.731542 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application [None req-db4ca41b-4271-47e6-86c8-7ba81e10c651 None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Jul 21 08:17:45.731542 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 08:17:45.731542 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 21 08:17:45.731542 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 21 08:17:45.731542 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 21 08:17:45.731542 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 21 08:17:45.731542 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 21 08:17:45.731542 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 21 08:17:45.731542 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Jul 21 08:17:45.731542 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application Jul 21 08:17:45.731542 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 21 08:17:45.731542 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application Jul 21 08:17:45.731542 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 08:17:45.731542 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 08:17:45.731542 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 08:17:45.731542 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 08:17:45.731542 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 08:17:45.732613 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 08:17:45.732613 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 08:17:45.732613 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 08:17:45.732613 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 08:17:45.732613 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 08:17:45.732613 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 08:17:45.732613 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 21 08:17:45.732613 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 21 08:17:45.732613 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 21 08:17:45.732613 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 21 08:17:45.732613 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 08:17:45.732613 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 08:17:45.732613 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 08:17:45.732613 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 08:17:45.732613 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 08:17:45.732613 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 08:17:45.733425 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 08:17:45.733425 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application with Lock( Jul 21 08:17:45.733425 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 08:17:45.733425 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application return self._enter() Jul 21 08:17:45.733425 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 08:17:45.733425 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 08:17:45.733425 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 08:17:45.733425 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application return self.creator() Jul 21 08:17:45.733425 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 08:17:45.733425 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application created_value = creator( Jul 21 08:17:45.733425 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 21 08:17:45.733425 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 21 08:17:45.733425 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Jul 21 08:17:45.733425 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application Jul 21 08:17:45.741128 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: [pid: 80245|app: 0|req: 442/882] 10.222.0.47 () {56 vars in 1114 bytes} [Thu Jul 21 08:17:45 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 21 08:17:45.742312 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db4ca41b-4271-47e6-86c8-7ba81e10c651 None admin] Authenticating user token {{(pid=80245) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:17:45.782135 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee2dee91-6217-409e-9faa-8ec702eec2cb None admin] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:17:45.817998 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ad5d224-96f6-42f1-ada2-a7f6319d1a2f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:17:45.818792 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ad5d224-96f6-42f1-ada2-a7f6319d1a2f None admin] REQUEST_METHOD: `GET` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:17:45.819209 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ad5d224-96f6-42f1-ada2-a7f6319d1a2f None admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:17:45.819552 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ad5d224-96f6-42f1-ada2-a7f6319d1a2f None admin] PATH_INFO: `/v3/roles` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:17:45.820399 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ad5d224-96f6-42f1-ada2-a7f6319d1a2f None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80244) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 08:17:45.820784 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ad5d224-96f6-42f1-ada2-a7f6319d1a2f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:17:45.822297 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ad5d224-96f6-42f1-ada2-a7f6319d1a2f None admin] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:17:45.836954 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 441/883] 10.222.0.47 () {56 vars in 1115 bytes} [Thu Jul 21 08:17:45 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 21 08:17:45.838283 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ad5d224-96f6-42f1-ada2-a7f6319d1a2f None admin] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:17:45.856653 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db4ca41b-4271-47e6-86c8-7ba81e10c651 None admin] Validating token access rules against request {{(pid=80245) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:17:45.897219 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61707359-887c-4748-af84-d4d92b05983e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80245) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:17:45.898266 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61707359-887c-4748-af84-d4d92b05983e None admin] REQUEST_METHOD: `GET` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:17:45.898764 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61707359-887c-4748-af84-d4d92b05983e None admin] SCRIPT_NAME: `/identity` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:17:45.899244 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61707359-887c-4748-af84-d4d92b05983e None admin] PATH_INFO: `/v3/services` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:17:45.900285 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61707359-887c-4748-af84-d4d92b05983e None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=80245) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 08:17:45.900897 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61707359-887c-4748-af84-d4d92b05983e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:17:45.902476 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61707359-887c-4748-af84-d4d92b05983e None admin] RBAC: Authorization granted {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:17:45.904025 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG dbcounter [-] [80244] Writing DB stats keystone:SELECT=100 {{(pid=80244) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:17:45.914491 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: [pid: 80245|app: 0|req: 443/884] 10.222.0.47 () {56 vars in 1114 bytes} [Thu Jul 21 08:17:45 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 08:17:45.915706 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61707359-887c-4748-af84-d4d92b05983e None admin] Authenticating user token {{(pid=80245) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:17:45.942044 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG dbcounter [-] [80245] Writing DB stats keystone:SELECT=100 {{(pid=80245) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:17:45.947980 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ad5d224-96f6-42f1-ada2-a7f6319d1a2f None admin] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:17:45.984256 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-092abc51-a02f-4b9b-908c-34a2c68284b9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:17:45.985059 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-092abc51-a02f-4b9b-908c-34a2c68284b9 None admin] REQUEST_METHOD: `GET` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:17:45.985346 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-092abc51-a02f-4b9b-908c-34a2c68284b9 None admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:17:45.985624 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-092abc51-a02f-4b9b-908c-34a2c68284b9 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:17:45.997025 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-092abc51-a02f-4b9b-908c-34a2c68284b9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:17:45.998509 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-092abc51-a02f-4b9b-908c-34a2c68284b9 None admin] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:17:46.018766 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application [None req-092abc51-a02f-4b9b-908c-34a2c68284b9 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 08:17:46.018766 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 08:17:46.018766 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 08:17:46.018766 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 08:17:46.018766 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 08:17:46.018766 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 08:17:46.018766 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 08:17:46.018766 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 08:17:46.018766 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 08:17:46.018766 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 08:17:46.018766 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 08:17:46.018766 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 08:17:46.018766 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 21 08:17:46.018766 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 21 08:17:46.018766 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 21 08:17:46.020329 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 21 08:17:46.020329 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 08:17:46.020329 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 08:17:46.020329 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 08:17:46.020329 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 08:17:46.020329 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 08:17:46.020329 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 08:17:46.020329 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 08:17:46.020329 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application with Lock( Jul 21 08:17:46.020329 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 08:17:46.020329 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self._enter() Jul 21 08:17:46.020329 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 08:17:46.020329 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 08:17:46.020329 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 08:17:46.020329 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self.creator() Jul 21 08:17:46.020329 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 08:17:46.020329 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application created_value = creator( Jul 21 08:17:46.021197 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 21 08:17:46.021197 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 21 08:17:46.021197 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 08:17:46.021197 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application Jul 21 08:17:46.025130 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 442/885] 10.222.0.47 () {56 vars in 1108 bytes} [Thu Jul 21 08:17:45 2022] GET /identity/v3/domains/Default => generated 87 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 08:17:46.026377 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-092abc51-a02f-4b9b-908c-34a2c68284b9 None admin] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:17:46.028284 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61707359-887c-4748-af84-d4d92b05983e None admin] Validating token access rules against request {{(pid=80245) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:17:46.064110 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15a1ff49-5b44-45ac-a7da-3babfad481e7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80245) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:17:46.064776 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15a1ff49-5b44-45ac-a7da-3babfad481e7 None admin] REQUEST_METHOD: `GET` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:17:46.065050 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15a1ff49-5b44-45ac-a7da-3babfad481e7 None admin] SCRIPT_NAME: `/identity` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:17:46.065323 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15a1ff49-5b44-45ac-a7da-3babfad481e7 None admin] PATH_INFO: `/v3/services` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:17:46.066530 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15a1ff49-5b44-45ac-a7da-3babfad481e7 None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=80245) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 08:17:46.066832 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15a1ff49-5b44-45ac-a7da-3babfad481e7 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:17:46.068213 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15a1ff49-5b44-45ac-a7da-3babfad481e7 None admin] RBAC: Authorization granted {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:17:46.073534 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: [pid: 80245|app: 0|req: 444/886] 10.222.0.47 () {56 vars in 1114 bytes} [Thu Jul 21 08:17:45 2022] GET /identity/v3/services?type=volumev3 => generated 362 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 08:17:46.079313 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bdf41b9-f6dd-4007-87d8-c595ec0e1481 None None] REQUEST_METHOD: `GET` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:17:46.079639 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bdf41b9-f6dd-4007-87d8-c595ec0e1481 None None] SCRIPT_NAME: `/identity` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:17:46.079915 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bdf41b9-f6dd-4007-87d8-c595ec0e1481 None None] PATH_INFO: `/` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:17:46.083772 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: [pid: 80245|app: 0|req: 445/887] 10.222.0.47 () {54 vars in 926 bytes} [Thu Jul 21 08:17:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 08:17:46.083772 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bdf41b9-f6dd-4007-87d8-c595ec0e1481 None None] Authenticating user token {{(pid=80245) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:17:46.133657 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-092abc51-a02f-4b9b-908c-34a2c68284b9 None admin] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:17:46.148664 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG dbcounter [-] [80244] Writing DB stats keystone:SELECT=100 {{(pid=80244) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:17:46.171267 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7aa2d98-a291-45d5-856f-ccba7a461f12 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:17:46.171964 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7aa2d98-a291-45d5-856f-ccba7a461f12 None admin] REQUEST_METHOD: `GET` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:17:46.172250 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7aa2d98-a291-45d5-856f-ccba7a461f12 None admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:17:46.172522 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7aa2d98-a291-45d5-856f-ccba7a461f12 None admin] PATH_INFO: `/v3/domains` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:17:46.173257 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7aa2d98-a291-45d5-856f-ccba7a461f12 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80244) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 08:17:46.173564 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7aa2d98-a291-45d5-856f-ccba7a461f12 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:17:46.175499 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7aa2d98-a291-45d5-856f-ccba7a461f12 None admin] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:17:46.180957 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG dbcounter [-] [80245] Writing DB stats keystone:SELECT=100 {{(pid=80245) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:17:46.188424 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 443/888] 10.222.0.47 () {56 vars in 1109 bytes} [Thu Jul 21 08:17:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 08:17:46.189587 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0469f376-ba0e-4166-83c2-31abaa736f47 None None] REQUEST_METHOD: `POST` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:17:46.189880 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0469f376-ba0e-4166-83c2-31abaa736f47 None None] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:17:46.190163 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0469f376-ba0e-4166-83c2-31abaa736f47 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:17:46.200196 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bdf41b9-f6dd-4007-87d8-c595ec0e1481 None None] Validating token access rules against request {{(pid=80245) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:17:46.207928 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.auth.core [None req-0469f376-ba0e-4166-83c2-31abaa736f47 None None] MFA Rules not processed for user `2dce40b8d74c41ec94cc36e6ff05d412`. Rule list: `[]` (Enabled: `True`). {{(pid=80244) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 08:17:46.209154 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.fernet_utils [None req-0469f376-ba0e-4166-83c2-31abaa736f47 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80244) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 08:17:46.237205 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-282ea78b-fa19-4c1c-9b08-a1571c80dd02 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80245) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:17:46.238570 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-282ea78b-fa19-4c1c-9b08-a1571c80dd02 None admin] REQUEST_METHOD: `GET` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:17:46.238860 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-282ea78b-fa19-4c1c-9b08-a1571c80dd02 None admin] SCRIPT_NAME: `/identity` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:17:46.239137 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-282ea78b-fa19-4c1c-9b08-a1571c80dd02 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:17:46.239982 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-282ea78b-fa19-4c1c-9b08-a1571c80dd02 None admin] RBAC: Adding query filter params (interface=public, service_id=5e8e062e32d24bb5baf04813b2acdb85, region_id=RegionOne) {{(pid=80245) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 08:17:46.240275 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-282ea78b-fa19-4c1c-9b08-a1571c80dd02 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:17:46.241645 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-282ea78b-fa19-4c1c-9b08-a1571c80dd02 None admin] RBAC: Authorization granted {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:17:46.246959 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: [pid: 80245|app: 0|req: 446/889] 10.222.0.47 () {56 vars in 1251 bytes} [Thu Jul 21 08:17:46 2022] GET /identity/v3/endpoints?service_id=5e8e062e32d24bb5baf04813b2acdb85&interface=public®ion_id=RegionOne => generated 194 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 08:17:46.247795 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-282ea78b-fa19-4c1c-9b08-a1571c80dd02 None admin] Authenticating user token {{(pid=80245) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:17:46.299833 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 444/890] 10.222.0.47 () {58 vars in 1023 bytes} [Thu Jul 21 08:17:46 2022] POST /identity/v3/auth/tokens => generated 1362 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 21 08:17:46.299833 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0469f376-ba0e-4166-83c2-31abaa736f47 None None] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:17:46.366087 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-282ea78b-fa19-4c1c-9b08-a1571c80dd02 None admin] Validating token access rules against request {{(pid=80245) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:17:46.403653 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8842f1c-5af7-4542-a766-f9f566d0ab47 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80245) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:17:46.404388 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8842f1c-5af7-4542-a766-f9f566d0ab47 None admin] REQUEST_METHOD: `GET` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:17:46.404699 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8842f1c-5af7-4542-a766-f9f566d0ab47 None admin] SCRIPT_NAME: `/identity` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:17:46.404941 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8842f1c-5af7-4542-a766-f9f566d0ab47 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:17:46.412856 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8842f1c-5af7-4542-a766-f9f566d0ab47 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:17:46.414285 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8842f1c-5af7-4542-a766-f9f566d0ab47 None admin] RBAC: Authorization granted {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:17:46.415017 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0469f376-ba0e-4166-83c2-31abaa736f47 None None] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:17:46.417235 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG dbcounter [-] [80244] Writing DB stats keystone:SELECT=100 {{(pid=80244) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:17:46.429848 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application [None req-b8842f1c-5af7-4542-a766-f9f566d0ab47 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 21 08:17:46.429848 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 08:17:46.429848 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 08:17:46.429848 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 08:17:46.429848 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 08:17:46.429848 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 08:17:46.429848 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 08:17:46.429848 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 08:17:46.429848 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 08:17:46.429848 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 08:17:46.429848 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 08:17:46.429848 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 08:17:46.429848 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 21 08:17:46.429848 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 21 08:17:46.429848 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 21 08:17:46.429848 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 21 08:17:46.430844 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 08:17:46.430844 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 08:17:46.430844 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 21 08:17:46.430844 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 21 08:17:46.430844 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 21 08:17:46.430844 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 21 08:17:46.430844 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 08:17:46.430844 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 08:17:46.430844 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 08:17:46.430844 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 08:17:46.430844 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 08:17:46.430844 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application with Lock( Jul 21 08:17:46.430844 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 08:17:46.430844 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application return self._enter() Jul 21 08:17:46.430844 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 08:17:46.430844 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 08:17:46.430844 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 08:17:46.430844 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application return self.creator() Jul 21 08:17:46.436844 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 08:17:46.436844 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application created_value = creator( Jul 21 08:17:46.436844 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 21 08:17:46.436844 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 21 08:17:46.436844 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 21 08:17:46.436844 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 21 08:17:46.436844 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 21 08:17:46.436844 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 21 08:17:46.436844 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 21 08:17:46.436844 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application Jul 21 08:17:46.436844 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: [pid: 80245|app: 0|req: 447/891] 10.222.0.47 () {56 vars in 1117 bytes} [Thu Jul 21 08:17:46 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 08:17:46.436844 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8842f1c-5af7-4542-a766-f9f566d0ab47 None admin] Authenticating user token {{(pid=80245) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:17:46.452404 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG dbcounter [-] [80245] Writing DB stats keystone:SELECT=100 {{(pid=80245) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:17:46.463289 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15d41b2d-b415-4f90-af17-30b1b62a8559 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:17:46.463994 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15d41b2d-b415-4f90-af17-30b1b62a8559 None admin] REQUEST_METHOD: `GET` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:17:46.464284 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15d41b2d-b415-4f90-af17-30b1b62a8559 None admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:17:46.464563 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15d41b2d-b415-4f90-af17-30b1b62a8559 None admin] PATH_INFO: `/v3/services` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:17:46.465379 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15d41b2d-b415-4f90-af17-30b1b62a8559 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:17:46.467345 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15d41b2d-b415-4f90-af17-30b1b62a8559 None admin] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:17:46.483352 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 445/892] 10.222.0.47 () {56 vars in 1087 bytes} [Thu Jul 21 08:17:46 2022] GET /identity/v3/services => generated 1268 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 08:17:46.483352 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15d41b2d-b415-4f90-af17-30b1b62a8559 None admin] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:17:46.586193 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8842f1c-5af7-4542-a766-f9f566d0ab47 None admin] Validating token access rules against request {{(pid=80245) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:17:46.606692 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15d41b2d-b415-4f90-af17-30b1b62a8559 None admin] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:17:46.628860 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0968d9a-1f5b-490a-864a-43e7892651f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80245) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:17:46.629979 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0968d9a-1f5b-490a-864a-43e7892651f7 None admin] REQUEST_METHOD: `GET` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:17:46.630424 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0968d9a-1f5b-490a-864a-43e7892651f7 None admin] SCRIPT_NAME: `/identity` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:17:46.630852 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0968d9a-1f5b-490a-864a-43e7892651f7 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:17:46.631730 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0968d9a-1f5b-490a-864a-43e7892651f7 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:17:46.633798 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0968d9a-1f5b-490a-864a-43e7892651f7 None admin] RBAC: Authorization granted {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:17:46.649046 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5335dbf0-b42c-41a3-92be-cf3bc5b27b3b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:17:46.649871 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5335dbf0-b42c-41a3-92be-cf3bc5b27b3b None admin] REQUEST_METHOD: `GET` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:17:46.650249 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5335dbf0-b42c-41a3-92be-cf3bc5b27b3b None admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:17:46.650618 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5335dbf0-b42c-41a3-92be-cf3bc5b27b3b None admin] PATH_INFO: `/v3/users` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:17:46.650874 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application [None req-f0968d9a-1f5b-490a-864a-43e7892651f7 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 21 08:17:46.650874 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 08:17:46.650874 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 21 08:17:46.650874 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 21 08:17:46.650874 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 21 08:17:46.650874 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 21 08:17:46.650874 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 21 08:17:46.650874 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 21 08:17:46.650874 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 21 08:17:46.650874 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application Jul 21 08:17:46.650874 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 21 08:17:46.650874 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application Jul 21 08:17:46.650874 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 08:17:46.650874 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 08:17:46.650874 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 08:17:46.650874 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 08:17:46.650874 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 08:17:46.652003 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 08:17:46.652003 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 08:17:46.652003 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 08:17:46.652003 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 08:17:46.652003 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 08:17:46.652003 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 08:17:46.652003 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 21 08:17:46.652003 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 21 08:17:46.652003 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 21 08:17:46.652003 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 21 08:17:46.652003 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 08:17:46.652003 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 08:17:46.652003 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 08:17:46.652003 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 08:17:46.652003 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 08:17:46.652003 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 08:17:46.652820 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 08:17:46.652820 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application with Lock( Jul 21 08:17:46.652820 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 08:17:46.652820 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application return self._enter() Jul 21 08:17:46.652820 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 08:17:46.652820 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 08:17:46.652820 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 08:17:46.652820 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application return self.creator() Jul 21 08:17:46.652820 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 08:17:46.652820 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application created_value = creator( Jul 21 08:17:46.652820 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 21 08:17:46.652820 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 21 08:17:46.652820 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 21 08:17:46.652820 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application Jul 21 08:17:46.653568 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5335dbf0-b42c-41a3-92be-cf3bc5b27b3b None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80244) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 08:17:46.653568 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5335dbf0-b42c-41a3-92be-cf3bc5b27b3b None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:17:46.655556 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: [pid: 80245|app: 0|req: 448/893] 10.222.0.47 () {56 vars in 1126 bytes} [Thu Jul 21 08:17:46 2022] GET /identity/v3/services/object-store => generated 93 bytes in 224 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 08:17:46.660265 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5335dbf0-b42c-41a3-92be-cf3bc5b27b3b None admin] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:17:46.661668 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0968d9a-1f5b-490a-864a-43e7892651f7 None admin] Authenticating user token {{(pid=80245) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:17:46.682690 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 446/894] 10.222.0.47 () {56 vars in 1149 bytes} [Thu Jul 21 08:17:46 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 08:17:46.687617 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5335dbf0-b42c-41a3-92be-cf3bc5b27b3b None admin] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:17:46.720471 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG dbcounter [-] [80244] Writing DB stats keystone:SELECT=100 {{(pid=80244) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:17:46.758689 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG dbcounter [-] [80245] Writing DB stats keystone:SELECT=100 {{(pid=80245) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:17:46.783670 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0968d9a-1f5b-490a-864a-43e7892651f7 None admin] Validating token access rules against request {{(pid=80245) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:17:46.814928 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5335dbf0-b42c-41a3-92be-cf3bc5b27b3b None admin] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:17:46.819385 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a45d6ca-b2de-4744-8783-fbb1b4055aa8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80245) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:17:46.820113 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a45d6ca-b2de-4744-8783-fbb1b4055aa8 None admin] REQUEST_METHOD: `GET` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:17:46.820400 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a45d6ca-b2de-4744-8783-fbb1b4055aa8 None admin] SCRIPT_NAME: `/identity` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:17:46.820676 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a45d6ca-b2de-4744-8783-fbb1b4055aa8 None admin] PATH_INFO: `/v3/services` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:17:46.821432 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a45d6ca-b2de-4744-8783-fbb1b4055aa8 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=80245) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 08:17:46.821788 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a45d6ca-b2de-4744-8783-fbb1b4055aa8 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:17:46.823157 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a45d6ca-b2de-4744-8783-fbb1b4055aa8 None admin] RBAC: Authorization granted {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:17:46.832593 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: [pid: 80245|app: 0|req: 449/895] 10.222.0.47 () {56 vars in 1122 bytes} [Thu Jul 21 08:17:46 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 21 08:17:46.832593 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfbc1c34-0bbe-468a-a604-c6c390b288cf None None] REQUEST_METHOD: `GET` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:17:46.832593 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfbc1c34-0bbe-468a-a604-c6c390b288cf None None] SCRIPT_NAME: `/identity` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:17:46.832593 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfbc1c34-0bbe-468a-a604-c6c390b288cf None None] PATH_INFO: `/` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:17:46.832593 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: [pid: 80245|app: 0|req: 450/896] 10.222.0.47 () {54 vars in 926 bytes} [Thu Jul 21 08:17:46 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 08:17:46.835933 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfbc1c34-0bbe-468a-a604-c6c390b288cf None None] Authenticating user token {{(pid=80245) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:17:46.856337 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66b803a3-be6f-4091-bbad-d08333569bd1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:17:46.856994 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66b803a3-be6f-4091-bbad-d08333569bd1 None admin] REQUEST_METHOD: `GET` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:17:46.857279 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66b803a3-be6f-4091-bbad-d08333569bd1 None admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:17:46.857552 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66b803a3-be6f-4091-bbad-d08333569bd1 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:17:46.936789 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66b803a3-be6f-4091-bbad-d08333569bd1 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:17:46.938168 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66b803a3-be6f-4091-bbad-d08333569bd1 None admin] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:17:46.951783 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfbc1c34-0bbe-468a-a604-c6c390b288cf None None] Validating token access rules against request {{(pid=80245) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:17:46.963397 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG dbcounter [-] [80244] Writing DB stats keystone:SELECT=100 {{(pid=80244) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:17:46.997360 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f28305a4-8d31-48a0-8bf1-ddf13ddc82d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80245) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:17:46.998045 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f28305a4-8d31-48a0-8bf1-ddf13ddc82d6 None admin] REQUEST_METHOD: `GET` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:17:46.998787 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f28305a4-8d31-48a0-8bf1-ddf13ddc82d6 None admin] SCRIPT_NAME: `/identity` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:17:46.999072 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f28305a4-8d31-48a0-8bf1-ddf13ddc82d6 None admin] PATH_INFO: `/v3/services` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:17:46.999981 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f28305a4-8d31-48a0-8bf1-ddf13ddc82d6 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=80245) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 08:17:47.000292 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f28305a4-8d31-48a0-8bf1-ddf13ddc82d6 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:17:47.001672 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f28305a4-8d31-48a0-8bf1-ddf13ddc82d6 None admin] RBAC: Authorization granted {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:17:47.013428 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: [pid: 80245|app: 0|req: 451/897] 10.222.0.47 () {56 vars in 1122 bytes} [Thu Jul 21 08:17:46 2022] GET /identity/v3/services?type=object-store => generated 129 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 08:17:47.013428 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6bec169-67b4-47f4-b286-56837273ece4 None None] REQUEST_METHOD: `POST` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:17:47.013428 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6bec169-67b4-47f4-b286-56837273ece4 None None] SCRIPT_NAME: `/identity` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:17:47.013428 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6bec169-67b4-47f4-b286-56837273ece4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:17:47.015371 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG dbcounter [-] [80245] Writing DB stats keystone:SELECT=100 {{(pid=80245) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:17:47.027862 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.auth.core [None req-d6bec169-67b4-47f4-b286-56837273ece4 None None] MFA Rules not processed for user `2dce40b8d74c41ec94cc36e6ff05d412`. Rule list: `[]` (Enabled: `True`). {{(pid=80245) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 08:17:47.029273 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.fernet_utils [None req-d6bec169-67b4-47f4-b286-56837273ece4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80245) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 08:17:47.067094 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 447/898] 10.222.0.47 () {58 vars in 1282 bytes} [Thu Jul 21 08:17:46 2022] GET /identity/v3/auth/tokens => generated 1362 bytes in 380 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 21 08:17:47.074670 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66b803a3-be6f-4091-bbad-d08333569bd1 None admin] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:17:47.128766 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: [pid: 80245|app: 0|req: 452/899] 10.222.0.47 () {58 vars in 1023 bytes} [Thu Jul 21 08:17:47 2022] POST /identity/v3/auth/tokens => generated 1362 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 21 08:17:47.151653 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6bec169-67b4-47f4-b286-56837273ece4 None None] Authenticating user token {{(pid=80245) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:17:47.215794 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66b803a3-be6f-4091-bbad-d08333569bd1 None admin] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:17:47.248436 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG dbcounter [-] [80244] Writing DB stats keystone:SELECT=100 {{(pid=80244) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:17:47.261987 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00b50c30-1453-48b5-b844-32186d212773 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:17:47.262894 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00b50c30-1453-48b5-b844-32186d212773 None admin] REQUEST_METHOD: `GET` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:17:47.263336 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00b50c30-1453-48b5-b844-32186d212773 None admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:17:47.263697 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00b50c30-1453-48b5-b844-32186d212773 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:17:47.271177 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6bec169-67b4-47f4-b286-56837273ece4 None None] Validating token access rules against request {{(pid=80245) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:17:47.277314 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00b50c30-1453-48b5-b844-32186d212773 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:17:47.278855 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00b50c30-1453-48b5-b844-32186d212773 None admin] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:17:47.297731 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application [None req-00b50c30-1453-48b5-b844-32186d212773 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 08:17:47.297731 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 08:17:47.297731 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 08:17:47.297731 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 08:17:47.297731 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 08:17:47.297731 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 08:17:47.297731 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 08:17:47.297731 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 08:17:47.297731 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 08:17:47.297731 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 08:17:47.297731 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 08:17:47.297731 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 08:17:47.297731 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 21 08:17:47.297731 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 21 08:17:47.297731 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 21 08:17:47.298871 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 21 08:17:47.298871 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 08:17:47.298871 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 08:17:47.298871 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 08:17:47.298871 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 08:17:47.298871 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 08:17:47.298871 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 08:17:47.298871 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 08:17:47.298871 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application with Lock( Jul 21 08:17:47.298871 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 08:17:47.298871 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self._enter() Jul 21 08:17:47.298871 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 08:17:47.298871 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 08:17:47.298871 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 08:17:47.298871 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self.creator() Jul 21 08:17:47.298871 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 08:17:47.298871 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application created_value = creator( Jul 21 08:17:47.299969 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 21 08:17:47.299969 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 21 08:17:47.299969 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 08:17:47.299969 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application Jul 21 08:17:47.301099 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 448/900] 10.222.0.47 () {56 vars in 1108 bytes} [Thu Jul 21 08:17:47 2022] GET /identity/v3/domains/Default => generated 87 bytes in 227 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 08:17:47.306432 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00b50c30-1453-48b5-b844-32186d212773 None admin] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:17:47.318431 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca4f8245-f578-48be-8346-ffb8c2aa6ca8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80245) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:17:47.319713 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca4f8245-f578-48be-8346-ffb8c2aa6ca8 None admin] REQUEST_METHOD: `GET` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:17:47.320158 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca4f8245-f578-48be-8346-ffb8c2aa6ca8 None admin] SCRIPT_NAME: `/identity` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:17:47.320576 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca4f8245-f578-48be-8346-ffb8c2aa6ca8 None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:17:47.321389 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca4f8245-f578-48be-8346-ffb8c2aa6ca8 None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:17:47.323019 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca4f8245-f578-48be-8346-ffb8c2aa6ca8 None admin] RBAC: Authorization granted {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:17:47.335646 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG dbcounter [-] [80245] Writing DB stats keystone:SELECT=100 {{(pid=80245) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:17:47.340882 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application [None req-ca4f8245-f578-48be-8346-ffb8c2aa6ca8 None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jul 21 08:17:47.340882 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 08:17:47.340882 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 21 08:17:47.340882 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 21 08:17:47.340882 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 21 08:17:47.340882 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 21 08:17:47.340882 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 21 08:17:47.340882 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 21 08:17:47.340882 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jul 21 08:17:47.340882 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application Jul 21 08:17:47.340882 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 21 08:17:47.340882 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application Jul 21 08:17:47.340882 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 08:17:47.340882 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 08:17:47.340882 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 08:17:47.340882 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 08:17:47.341991 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 08:17:47.341991 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 08:17:47.341991 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 08:17:47.341991 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 08:17:47.341991 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 08:17:47.341991 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 08:17:47.341991 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 08:17:47.341991 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 21 08:17:47.341991 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 21 08:17:47.341991 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 21 08:17:47.341991 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 21 08:17:47.341991 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 08:17:47.341991 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 08:17:47.341991 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 08:17:47.341991 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 08:17:47.341991 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 08:17:47.342903 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 08:17:47.342903 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 08:17:47.342903 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application with Lock( Jul 21 08:17:47.342903 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 08:17:47.342903 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application return self._enter() Jul 21 08:17:47.342903 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 08:17:47.342903 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 08:17:47.342903 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 08:17:47.342903 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application return self.creator() Jul 21 08:17:47.342903 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 08:17:47.342903 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application created_value = creator( Jul 21 08:17:47.342903 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 21 08:17:47.342903 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 21 08:17:47.342903 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jul 21 08:17:47.342903 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application Jul 21 08:17:47.345892 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: [pid: 80245|app: 0|req: 453/901] 10.222.0.47 () {56 vars in 1132 bytes} [Thu Jul 21 08:17:47 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 21 08:17:47.355379 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca4f8245-f578-48be-8346-ffb8c2aa6ca8 None admin] Authenticating user token {{(pid=80245) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:17:47.417977 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00b50c30-1453-48b5-b844-32186d212773 None admin] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:17:47.455481 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeb83caa-ca91-43a7-93e3-0fbf6644370f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:17:47.456960 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeb83caa-ca91-43a7-93e3-0fbf6644370f None admin] REQUEST_METHOD: `GET` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:17:47.457554 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeb83caa-ca91-43a7-93e3-0fbf6644370f None admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:17:47.457981 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeb83caa-ca91-43a7-93e3-0fbf6644370f None admin] PATH_INFO: `/v3/domains` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:17:47.458939 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aeb83caa-ca91-43a7-93e3-0fbf6644370f None admin] RBAC: Adding query filter params (name=Default) {{(pid=80244) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 08:17:47.459425 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aeb83caa-ca91-43a7-93e3-0fbf6644370f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:17:47.461062 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aeb83caa-ca91-43a7-93e3-0fbf6644370f None admin] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:17:47.474880 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 449/902] 10.222.0.47 () {56 vars in 1109 bytes} [Thu Jul 21 08:17:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 21 08:17:47.481584 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeb83caa-ca91-43a7-93e3-0fbf6644370f None admin] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:17:47.494447 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca4f8245-f578-48be-8346-ffb8c2aa6ca8 None admin] Validating token access rules against request {{(pid=80245) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:17:47.523531 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG dbcounter [-] [80244] Writing DB stats keystone:SELECT=100 {{(pid=80244) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:17:47.553140 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95d655a2-1162-4659-b1c8-556a7f7edb7a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80245) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:17:47.554043 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95d655a2-1162-4659-b1c8-556a7f7edb7a None admin] REQUEST_METHOD: `GET` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:17:47.554421 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95d655a2-1162-4659-b1c8-556a7f7edb7a None admin] SCRIPT_NAME: `/identity` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:17:47.554773 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95d655a2-1162-4659-b1c8-556a7f7edb7a None admin] PATH_INFO: `/v3/services` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:17:47.555830 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95d655a2-1162-4659-b1c8-556a7f7edb7a None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=80245) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 08:17:47.556618 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95d655a2-1162-4659-b1c8-556a7f7edb7a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:17:47.558357 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95d655a2-1162-4659-b1c8-556a7f7edb7a None admin] RBAC: Authorization granted {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:17:47.567368 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: [pid: 80245|app: 0|req: 454/903] 10.222.0.47 () {56 vars in 1126 bytes} [Thu Jul 21 08:17:47 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 08:17:47.571281 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95d655a2-1162-4659-b1c8-556a7f7edb7a None admin] Authenticating user token {{(pid=80245) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:17:47.604652 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeb83caa-ca91-43a7-93e3-0fbf6644370f None admin] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:17:47.650983 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG dbcounter [-] [80245] Writing DB stats keystone:SELECT=100 {{(pid=80245) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:17:47.652509 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27c70377-e08e-4292-9402-e2a7aa3f5613 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:17:47.653687 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27c70377-e08e-4292-9402-e2a7aa3f5613 None admin] REQUEST_METHOD: `GET` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:17:47.653998 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27c70377-e08e-4292-9402-e2a7aa3f5613 None admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:17:47.654276 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27c70377-e08e-4292-9402-e2a7aa3f5613 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:17:47.666017 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27c70377-e08e-4292-9402-e2a7aa3f5613 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:17:47.667552 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27c70377-e08e-4292-9402-e2a7aa3f5613 None admin] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:17:47.686952 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application [None req-27c70377-e08e-4292-9402-e2a7aa3f5613 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 21 08:17:47.686952 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 08:17:47.686952 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 08:17:47.686952 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 08:17:47.686952 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 08:17:47.686952 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 08:17:47.686952 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 08:17:47.686952 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 08:17:47.686952 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 08:17:47.686952 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 08:17:47.686952 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 08:17:47.686952 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 08:17:47.686952 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 21 08:17:47.686952 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 21 08:17:47.686952 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 21 08:17:47.688028 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 21 08:17:47.688028 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 08:17:47.688028 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 08:17:47.688028 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 08:17:47.688028 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 08:17:47.688028 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 08:17:47.688028 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 08:17:47.688028 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 08:17:47.688028 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application with Lock( Jul 21 08:17:47.688028 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 08:17:47.688028 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self._enter() Jul 21 08:17:47.688028 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 08:17:47.688028 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 08:17:47.688028 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 08:17:47.688028 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self.creator() Jul 21 08:17:47.688028 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 08:17:47.688028 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application created_value = creator( Jul 21 08:17:47.693951 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 21 08:17:47.693951 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 21 08:17:47.693951 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 21 08:17:47.693951 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 21 08:17:47.693951 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 21 08:17:47.693951 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 21 08:17:47.693951 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 21 08:17:47.693951 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application Jul 21 08:17:47.693951 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 450/904] 10.222.0.47 () {56 vars in 1111 bytes} [Thu Jul 21 08:17:47 2022] GET /identity/v3/projects/service => generated 88 bytes in 207 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 21 08:17:47.696322 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27c70377-e08e-4292-9402-e2a7aa3f5613 None admin] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:17:47.703762 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95d655a2-1162-4659-b1c8-556a7f7edb7a None admin] Validating token access rules against request {{(pid=80245) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:17:47.742969 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26a92cd7-de51-4192-b4df-79b63416070f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80245) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:17:47.744143 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26a92cd7-de51-4192-b4df-79b63416070f None admin] REQUEST_METHOD: `GET` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:17:47.744649 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26a92cd7-de51-4192-b4df-79b63416070f None admin] SCRIPT_NAME: `/identity` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:17:47.745143 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26a92cd7-de51-4192-b4df-79b63416070f None admin] PATH_INFO: `/v3/services` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:17:47.746153 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26a92cd7-de51-4192-b4df-79b63416070f None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=80245) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 08:17:47.746682 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26a92cd7-de51-4192-b4df-79b63416070f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:17:47.748398 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26a92cd7-de51-4192-b4df-79b63416070f None admin] RBAC: Authorization granted {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:17:47.754390 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: [pid: 80245|app: 0|req: 455/905] 10.222.0.47 () {56 vars in 1126 bytes} [Thu Jul 21 08:17:47 2022] GET /identity/v3/services?type=compute_legacy => generated 386 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 21 08:17:47.766402 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26a92cd7-de51-4192-b4df-79b63416070f None admin] Authenticating user token {{(pid=80245) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:17:47.823115 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG dbcounter [-] [80244] Writing DB stats keystone:SELECT=100 {{(pid=80244) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:17:47.830047 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27c70377-e08e-4292-9402-e2a7aa3f5613 None admin] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:17:47.868624 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aa76320-7051-483d-ba3c-3045d38e144e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:17:47.869435 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aa76320-7051-483d-ba3c-3045d38e144e None admin] REQUEST_METHOD: `GET` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:17:47.869814 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aa76320-7051-483d-ba3c-3045d38e144e None admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:17:47.870200 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aa76320-7051-483d-ba3c-3045d38e144e None admin] PATH_INFO: `/v3/projects` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:17:47.871069 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8aa76320-7051-483d-ba3c-3045d38e144e None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80244) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 08:17:47.871633 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8aa76320-7051-483d-ba3c-3045d38e144e None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:17:47.873041 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8aa76320-7051-483d-ba3c-3045d38e144e None admin] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:17:47.881948 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26a92cd7-de51-4192-b4df-79b63416070f None admin] Validating token access rules against request {{(pid=80245) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:17:47.886614 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 451/906] 10.222.0.47 () {56 vars in 1148 bytes} [Thu Jul 21 08:17:47 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 08:17:47.897058 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aa76320-7051-483d-ba3c-3045d38e144e None admin] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:17:47.916618 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG dbcounter [-] [80245] Writing DB stats keystone:SELECT=100 {{(pid=80245) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:17:47.929679 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c65aa94-9da8-4ddb-8690-ab793660ef71 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80245) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:17:47.930670 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c65aa94-9da8-4ddb-8690-ab793660ef71 None admin] REQUEST_METHOD: `GET` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:17:47.931385 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c65aa94-9da8-4ddb-8690-ab793660ef71 None admin] SCRIPT_NAME: `/identity` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:17:47.931819 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c65aa94-9da8-4ddb-8690-ab793660ef71 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:17:47.933165 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c65aa94-9da8-4ddb-8690-ab793660ef71 None admin] RBAC: Adding query filter params (interface=public, service_id=9a5a169cf53041538adfdc2fb3528931, region_id=RegionOne) {{(pid=80245) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 08:17:47.933864 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c65aa94-9da8-4ddb-8690-ab793660ef71 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:17:47.935417 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c65aa94-9da8-4ddb-8690-ab793660ef71 None admin] RBAC: Authorization granted {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:17:47.942543 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: [pid: 80245|app: 0|req: 456/907] 10.222.0.47 () {56 vars in 1251 bytes} [Thu Jul 21 08:17:47 2022] GET /identity/v3/endpoints?service_id=9a5a169cf53041538adfdc2fb3528931&interface=public®ion_id=RegionOne => generated 194 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 08:17:47.950224 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c65aa94-9da8-4ddb-8690-ab793660ef71 None admin] Authenticating user token {{(pid=80245) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:17:48.028777 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aa76320-7051-483d-ba3c-3045d38e144e None admin] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:17:48.064136 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8b2c802-e07f-438a-a3dd-86dd8d15c1ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:17:48.064800 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8b2c802-e07f-438a-a3dd-86dd8d15c1ea None admin] REQUEST_METHOD: `GET` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:17:48.065079 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8b2c802-e07f-438a-a3dd-86dd8d15c1ea None admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:17:48.065352 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8b2c802-e07f-438a-a3dd-86dd8d15c1ea None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:17:48.067065 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8b2c802-e07f-438a-a3dd-86dd8d15c1ea None admin] RBAC: Adding query filter params (role.id=4b8dff36c32b49618b303451de1d1f69, scope.project.id=4faef13aa50e4e36b6c265fb1398fa14, user.id=2ef7f1e91e554a1f88eb6a63e60a475f) {{(pid=80244) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 08:17:48.067414 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8b2c802-e07f-438a-a3dd-86dd8d15c1ea None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:17:48.068859 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8b2c802-e07f-438a-a3dd-86dd8d15c1ea None admin] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:17:48.074925 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 452/908] 10.222.0.47 () {56 vars in 1374 bytes} [Thu Jul 21 08:17:47 2022] GET /identity/v3/role_assignments?user.id=2ef7f1e91e554a1f88eb6a63e60a475f&scope.project.id=4faef13aa50e4e36b6c265fb1398fa14&role.id=4b8dff36c32b49618b303451de1d1f69 => generated 259 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 08:17:48.085732 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c65aa94-9da8-4ddb-8690-ab793660ef71 None admin] Validating token access rules against request {{(pid=80245) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:17:48.123335 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1454324-cdb5-4c16-8ee3-c3ad487ef6d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80245) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:17:48.124040 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1454324-cdb5-4c16-8ee3-c3ad487ef6d0 None admin] REQUEST_METHOD: `GET` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:17:48.124319 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1454324-cdb5-4c16-8ee3-c3ad487ef6d0 None admin] SCRIPT_NAME: `/identity` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:17:48.124584 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1454324-cdb5-4c16-8ee3-c3ad487ef6d0 None admin] PATH_INFO: `/v3/services` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:17:48.125262 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1454324-cdb5-4c16-8ee3-c3ad487ef6d0 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:17:48.126651 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1454324-cdb5-4c16-8ee3-c3ad487ef6d0 None admin] RBAC: Authorization granted {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:17:48.134234 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: [pid: 80245|app: 0|req: 457/909] 10.222.0.47 () {56 vars in 1087 bytes} [Thu Jul 21 08:17:47 2022] GET /identity/v3/services => generated 1268 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 21 08:17:48.637156 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d6aae48-dd4a-4716-8c77-95a0558e7e9f None None] REQUEST_METHOD: `GET` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:17:48.637997 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d6aae48-dd4a-4716-8c77-95a0558e7e9f None None] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:17:48.638449 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d6aae48-dd4a-4716-8c77-95a0558e7e9f None None] PATH_INFO: `/` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:17:48.640278 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 453/910] 10.222.0.47 () {54 vars in 926 bytes} [Thu Jul 21 08:17:48 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 08:17:48.645999 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2696975b-4adc-4871-9313-8287fb841b23 None None] REQUEST_METHOD: `POST` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:17:48.646835 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2696975b-4adc-4871-9313-8287fb841b23 None None] SCRIPT_NAME: `/identity` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:17:48.647315 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2696975b-4adc-4871-9313-8287fb841b23 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:17:48.664718 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.auth.core [None req-2696975b-4adc-4871-9313-8287fb841b23 None None] MFA Rules not processed for user `2dce40b8d74c41ec94cc36e6ff05d412`. Rule list: `[]` (Enabled: `True`). {{(pid=80245) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 08:17:48.666242 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.fernet_utils [None req-2696975b-4adc-4871-9313-8287fb841b23 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80245) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 08:17:48.721534 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG dbcounter [-] [80245] Writing DB stats keystone:SELECT=100 {{(pid=80245) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:17:48.742907 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: [pid: 80245|app: 0|req: 458/911] 10.222.0.47 () {58 vars in 1023 bytes} [Thu Jul 21 08:17:48 2022] POST /identity/v3/auth/tokens => generated 1362 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 21 08:17:48.770260 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d6aae48-dd4a-4716-8c77-95a0558e7e9f None None] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:17:48.793898 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG dbcounter [-] [80244] Writing DB stats keystone:SELECT=100 {{(pid=80244) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:17:48.878279 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d6aae48-dd4a-4716-8c77-95a0558e7e9f None None] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:17:48.914375 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-590f0f20-3be2-44c0-8924-5c3219749b5c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:17:48.915229 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-590f0f20-3be2-44c0-8924-5c3219749b5c None admin] REQUEST_METHOD: `GET` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:17:48.915544 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-590f0f20-3be2-44c0-8924-5c3219749b5c None admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:17:48.915815 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-590f0f20-3be2-44c0-8924-5c3219749b5c None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:17:48.916462 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-590f0f20-3be2-44c0-8924-5c3219749b5c None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:17:48.917825 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-590f0f20-3be2-44c0-8924-5c3219749b5c None admin] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:17:48.931974 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application [None req-590f0f20-3be2-44c0-8924-5c3219749b5c None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Jul 21 08:17:48.931974 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 08:17:48.931974 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 21 08:17:48.931974 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 21 08:17:48.931974 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 21 08:17:48.931974 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 21 08:17:48.931974 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 21 08:17:48.931974 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 21 08:17:48.931974 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Jul 21 08:17:48.931974 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application Jul 21 08:17:48.931974 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 21 08:17:48.931974 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application Jul 21 08:17:48.931974 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 08:17:48.931974 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 08:17:48.931974 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 08:17:48.931974 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 08:17:48.931974 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 08:17:48.933052 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 08:17:48.933052 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 08:17:48.933052 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 08:17:48.933052 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 08:17:48.933052 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 08:17:48.933052 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 08:17:48.933052 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 21 08:17:48.933052 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 21 08:17:48.933052 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 21 08:17:48.933052 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 21 08:17:48.933052 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 08:17:48.933052 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 08:17:48.933052 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 08:17:48.933052 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 08:17:48.933052 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 08:17:48.933052 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 08:17:48.934318 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 08:17:48.934318 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application with Lock( Jul 21 08:17:48.934318 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 08:17:48.934318 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self._enter() Jul 21 08:17:48.934318 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 08:17:48.934318 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 08:17:48.934318 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 08:17:48.934318 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self.creator() Jul 21 08:17:48.934318 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 08:17:48.934318 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application created_value = creator( Jul 21 08:17:48.934318 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 21 08:17:48.934318 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 21 08:17:48.934318 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Jul 21 08:17:48.934318 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application Jul 21 08:17:48.934318 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 454/912] 10.222.0.47 () {56 vars in 1114 bytes} [Thu Jul 21 08:17:48 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 08:17:48.939496 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2696975b-4adc-4871-9313-8287fb841b23 None None] Authenticating user token {{(pid=80245) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:17:49.048514 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2696975b-4adc-4871-9313-8287fb841b23 None None] Validating token access rules against request {{(pid=80245) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:17:49.092282 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-553d9d14-1189-4798-a1ca-7b8606627c4f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80245) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:17:49.093426 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-553d9d14-1189-4798-a1ca-7b8606627c4f None admin] REQUEST_METHOD: `GET` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:17:49.093917 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-553d9d14-1189-4798-a1ca-7b8606627c4f None admin] SCRIPT_NAME: `/identity` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:17:49.094450 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-553d9d14-1189-4798-a1ca-7b8606627c4f None admin] PATH_INFO: `/v3/services` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:17:49.095404 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-553d9d14-1189-4798-a1ca-7b8606627c4f None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=80245) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 08:17:49.095933 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-553d9d14-1189-4798-a1ca-7b8606627c4f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:17:49.098094 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-553d9d14-1189-4798-a1ca-7b8606627c4f None admin] RBAC: Authorization granted {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:17:49.103656 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: [pid: 80245|app: 0|req: 459/913] 10.222.0.47 () {56 vars in 1114 bytes} [Thu Jul 21 08:17:48 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 08:17:49.109423 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-590f0f20-3be2-44c0-8924-5c3219749b5c None admin] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:17:49.160597 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-850b66fa-3569-426f-ae5f-dd7ce75f8d1e None None] REQUEST_METHOD: `GET` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:17:49.161559 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-850b66fa-3569-426f-ae5f-dd7ce75f8d1e None None] SCRIPT_NAME: `/identity` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:17:49.162290 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-850b66fa-3569-426f-ae5f-dd7ce75f8d1e None None] PATH_INFO: `/` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:17:49.164545 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: [pid: 80245|app: 0|req: 460/914] 10.222.0.47 () {54 vars in 926 bytes} [Thu Jul 21 08:17:49 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 21 08:17:49.170756 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-470e5811-07aa-45e4-8c8a-3238894c9bbd None None] REQUEST_METHOD: `POST` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:17:49.171100 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-470e5811-07aa-45e4-8c8a-3238894c9bbd None None] SCRIPT_NAME: `/identity` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:17:49.171297 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-470e5811-07aa-45e4-8c8a-3238894c9bbd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:17:49.189140 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.auth.core [None req-470e5811-07aa-45e4-8c8a-3238894c9bbd None None] MFA Rules not processed for user `2dce40b8d74c41ec94cc36e6ff05d412`. Rule list: `[]` (Enabled: `True`). {{(pid=80245) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 08:17:49.191428 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.fernet_utils [None req-470e5811-07aa-45e4-8c8a-3238894c9bbd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80245) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 08:17:49.209429 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG dbcounter [-] [80244] Writing DB stats keystone:SELECT=100 {{(pid=80244) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:17:49.227186 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-590f0f20-3be2-44c0-8924-5c3219749b5c None admin] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:17:49.253502 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG dbcounter [-] [80245] Writing DB stats keystone:SELECT=100 {{(pid=80245) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:17:49.264833 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96648c52-38d8-4f4d-b509-0da3bde76c23 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:17:49.265658 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96648c52-38d8-4f4d-b509-0da3bde76c23 None admin] REQUEST_METHOD: `GET` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:17:49.266041 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96648c52-38d8-4f4d-b509-0da3bde76c23 None admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:17:49.266589 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96648c52-38d8-4f4d-b509-0da3bde76c23 None admin] PATH_INFO: `/v3/services` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:17:49.267443 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96648c52-38d8-4f4d-b509-0da3bde76c23 None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=80244) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 08:17:49.267849 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96648c52-38d8-4f4d-b509-0da3bde76c23 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:17:49.269983 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96648c52-38d8-4f4d-b509-0da3bde76c23 None admin] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:17:49.282728 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 455/915] 10.222.0.47 () {56 vars in 1114 bytes} [Thu Jul 21 08:17:49 2022] GET /identity/v3/services?type=volumev3 => generated 362 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 08:17:49.283792 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96648c52-38d8-4f4d-b509-0da3bde76c23 None admin] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:17:49.291746 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: [pid: 80245|app: 0|req: 461/916] 10.222.0.47 () {58 vars in 1023 bytes} [Thu Jul 21 08:17:49 2022] POST /identity/v3/auth/tokens => generated 1362 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 21 08:17:49.314921 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-470e5811-07aa-45e4-8c8a-3238894c9bbd None None] Authenticating user token {{(pid=80245) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:17:49.399870 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96648c52-38d8-4f4d-b509-0da3bde76c23 None admin] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:17:49.430650 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-470e5811-07aa-45e4-8c8a-3238894c9bbd None None] Validating token access rules against request {{(pid=80245) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:17:49.435880 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e776177-c282-4133-95fd-7b714b4482f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:17:49.436615 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e776177-c282-4133-95fd-7b714b4482f8 None admin] REQUEST_METHOD: `POST` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:17:49.436906 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e776177-c282-4133-95fd-7b714b4482f8 None admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:17:49.437180 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e776177-c282-4133-95fd-7b714b4482f8 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:17:49.437983 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e776177-c282-4133-95fd-7b714b4482f8 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:17:49.439383 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e776177-c282-4133-95fd-7b714b4482f8 None admin] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:17:49.467522 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 456/917] 10.222.0.47 () {60 vars in 1144 bytes} [Thu Jul 21 08:17:49 2022] POST /identity/v3/endpoints => generated 346 bytes in 179 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 08:17:49.476949 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f954c9f9-c372-4e15-9ebb-0c67b30ec293 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80245) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:17:49.478111 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f954c9f9-c372-4e15-9ebb-0c67b30ec293 None admin] REQUEST_METHOD: `POST` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:17:49.478619 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f954c9f9-c372-4e15-9ebb-0c67b30ec293 None admin] SCRIPT_NAME: `/identity` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:17:49.479127 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f954c9f9-c372-4e15-9ebb-0c67b30ec293 None admin] PATH_INFO: `/v3/services` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:17:49.480115 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f954c9f9-c372-4e15-9ebb-0c67b30ec293 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:17:49.482341 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f954c9f9-c372-4e15-9ebb-0c67b30ec293 None admin] RBAC: Authorization granted {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:17:49.501154 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: [pid: 80245|app: 0|req: 462/918] 10.222.0.47 () {60 vars in 1141 bytes} [Thu Jul 21 08:17:49 2022] POST /identity/v3/services => generated 241 bytes in 187 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 08:17:50.235277 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a12f85a3-249e-4948-b4b3-eddcbd776e0f None None] REQUEST_METHOD: `GET` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:17:50.235807 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a12f85a3-249e-4948-b4b3-eddcbd776e0f None None] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:17:50.235915 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a12f85a3-249e-4948-b4b3-eddcbd776e0f None None] PATH_INFO: `/` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:17:50.238003 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 457/919] 10.222.0.47 () {54 vars in 926 bytes} [Thu Jul 21 08:17:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 08:17:50.243237 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a193440-9d0c-4858-a637-debe3a72d0a1 None None] REQUEST_METHOD: `POST` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:17:50.243703 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a193440-9d0c-4858-a637-debe3a72d0a1 None None] SCRIPT_NAME: `/identity` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:17:50.243887 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a193440-9d0c-4858-a637-debe3a72d0a1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:17:50.269235 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0709be9e-e63f-4c94-9c43-722dbeb76d5a None None] REQUEST_METHOD: `GET` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:17:50.269543 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0709be9e-e63f-4c94-9c43-722dbeb76d5a None None] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:17:50.269768 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.auth.core [None req-8a193440-9d0c-4858-a637-debe3a72d0a1 None None] MFA Rules not processed for user `2dce40b8d74c41ec94cc36e6ff05d412`. Rule list: `[]` (Enabled: `True`). {{(pid=80245) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 08:17:50.269882 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0709be9e-e63f-4c94-9c43-722dbeb76d5a None None] PATH_INFO: `/` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:17:50.270958 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.fernet_utils [None req-8a193440-9d0c-4858-a637-debe3a72d0a1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80245) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 08:17:50.271706 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 458/920] 10.222.0.47 () {54 vars in 926 bytes} [Thu Jul 21 08:17:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 21 08:17:50.276778 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bbb7206-5ecb-4fcb-83d7-0106edda6dc0 None None] REQUEST_METHOD: `POST` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:17:50.277066 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bbb7206-5ecb-4fcb-83d7-0106edda6dc0 None None] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:17:50.277349 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bbb7206-5ecb-4fcb-83d7-0106edda6dc0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:17:50.284727 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG dbcounter [-] [80244] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80244) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:17:50.298380 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.auth.core [None req-4bbb7206-5ecb-4fcb-83d7-0106edda6dc0 None None] MFA Rules not processed for user `2dce40b8d74c41ec94cc36e6ff05d412`. Rule list: `[]` (Enabled: `True`). {{(pid=80244) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 08:17:50.300798 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.fernet_utils [None req-4bbb7206-5ecb-4fcb-83d7-0106edda6dc0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80244) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 08:17:50.301651 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG dbcounter [-] [80245] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80245) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:17:50.354980 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: [pid: 80245|app: 0|req: 463/921] 10.222.0.47 () {58 vars in 1023 bytes} [Thu Jul 21 08:17:50 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 21 08:17:50.380250 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a193440-9d0c-4858-a637-debe3a72d0a1 None None] Authenticating user token {{(pid=80245) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:17:50.390904 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 459/922] 10.222.0.47 () {58 vars in 1023 bytes} [Thu Jul 21 08:17:50 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 21 08:17:50.416332 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bbb7206-5ecb-4fcb-83d7-0106edda6dc0 None None] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:17:50.504079 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a193440-9d0c-4858-a637-debe3a72d0a1 None None] Validating token access rules against request {{(pid=80245) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:17:50.538330 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bbb7206-5ecb-4fcb-83d7-0106edda6dc0 None None] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:17:50.541891 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-924d2ea5-f33d-4edb-8692-76aca1c98cb5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80245) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:17:50.542715 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-924d2ea5-f33d-4edb-8692-76aca1c98cb5 None admin] REQUEST_METHOD: `GET` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:17:50.542994 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-924d2ea5-f33d-4edb-8692-76aca1c98cb5 None admin] SCRIPT_NAME: `/identity` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:17:50.543266 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-924d2ea5-f33d-4edb-8692-76aca1c98cb5 None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:17:50.543958 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-924d2ea5-f33d-4edb-8692-76aca1c98cb5 None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:17:50.545336 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-924d2ea5-f33d-4edb-8692-76aca1c98cb5 None admin] RBAC: Authorization granted {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:17:50.559129 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application [None req-924d2ea5-f33d-4edb-8692-76aca1c98cb5 None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jul 21 08:17:50.559129 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 08:17:50.559129 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 21 08:17:50.559129 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 21 08:17:50.559129 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 21 08:17:50.559129 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 21 08:17:50.559129 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 21 08:17:50.559129 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 21 08:17:50.559129 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jul 21 08:17:50.559129 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application Jul 21 08:17:50.559129 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 21 08:17:50.559129 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application Jul 21 08:17:50.559129 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 08:17:50.559129 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 08:17:50.559129 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 08:17:50.559129 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 08:17:50.560357 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 08:17:50.560357 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 08:17:50.560357 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 08:17:50.560357 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 08:17:50.560357 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 08:17:50.560357 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 08:17:50.560357 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 08:17:50.560357 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 21 08:17:50.560357 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 21 08:17:50.560357 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 21 08:17:50.560357 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 21 08:17:50.560357 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 08:17:50.560357 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 08:17:50.560357 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 08:17:50.560357 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 08:17:50.560357 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 08:17:50.561569 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 08:17:50.561569 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 08:17:50.561569 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application with Lock( Jul 21 08:17:50.561569 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 08:17:50.561569 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application return self._enter() Jul 21 08:17:50.561569 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 08:17:50.561569 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 08:17:50.561569 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 08:17:50.561569 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application return self.creator() Jul 21 08:17:50.561569 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 08:17:50.561569 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application created_value = creator( Jul 21 08:17:50.561569 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 21 08:17:50.561569 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 21 08:17:50.561569 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jul 21 08:17:50.561569 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application Jul 21 08:17:50.561569 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: [pid: 80245|app: 0|req: 464/923] 10.222.0.47 () {56 vars in 1132 bytes} [Thu Jul 21 08:17:50 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 21 08:17:50.566170 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-924d2ea5-f33d-4edb-8692-76aca1c98cb5 None admin] Authenticating user token {{(pid=80245) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:17:50.581774 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75407a97-a6e4-430e-98eb-848abae75e42 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:17:50.582547 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75407a97-a6e4-430e-98eb-848abae75e42 None admin] REQUEST_METHOD: `GET` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:17:50.582835 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75407a97-a6e4-430e-98eb-848abae75e42 None admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:17:50.583284 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75407a97-a6e4-430e-98eb-848abae75e42 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:17:50.585663 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG dbcounter [-] [80244] Writing DB stats keystone:SELECT=100 {{(pid=80244) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:17:50.598826 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75407a97-a6e4-430e-98eb-848abae75e42 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 21 08:17:50.599964 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG dbcounter [-] [80245] Writing DB stats keystone:SELECT=100 {{(pid=80245) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:17:50.611205 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application [None req-75407a97-a6e4-430e-98eb-848abae75e42 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 21 08:17:50.611205 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 08:17:50.611205 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 08:17:50.611205 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 08:17:50.611205 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 08:17:50.611205 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 08:17:50.611205 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 08:17:50.611205 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 08:17:50.611205 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 08:17:50.611205 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 08:17:50.611205 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 08:17:50.611205 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 08:17:50.611205 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 21 08:17:50.611205 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 21 08:17:50.611205 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 21 08:17:50.611205 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 21 08:17:50.612192 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 21 08:17:50.612192 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 21 08:17:50.612192 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 21 08:17:50.612192 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 21 08:17:50.612192 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 08:17:50.612192 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 08:17:50.612192 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 08:17:50.612192 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 08:17:50.612192 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 08:17:50.612192 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 08:17:50.612192 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 08:17:50.612192 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application with Lock( Jul 21 08:17:50.612192 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 08:17:50.612192 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self._enter() Jul 21 08:17:50.612192 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 08:17:50.612192 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 08:17:50.613642 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 08:17:50.613642 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self.creator() Jul 21 08:17:50.613642 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 08:17:50.613642 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application created_value = creator( Jul 21 08:17:50.613642 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 21 08:17:50.613642 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 21 08:17:50.613642 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 21 08:17:50.613642 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 21 08:17:50.613642 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 21 08:17:50.613642 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 21 08:17:50.613642 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 21 08:17:50.613642 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application Jul 21 08:17:50.613642 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 460/924] 10.222.0.47 () {56 vars in 1120 bytes} [Thu Jul 21 08:17:50 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 08:17:50.618833 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75407a97-a6e4-430e-98eb-848abae75e42 None admin] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:17:50.695457 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-924d2ea5-f33d-4edb-8692-76aca1c98cb5 None admin] Validating token access rules against request {{(pid=80245) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:17:50.738726 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a39fa6ae-b08a-4bd7-be86-08006569d4cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80245) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:17:50.739575 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a39fa6ae-b08a-4bd7-be86-08006569d4cb None admin] REQUEST_METHOD: `GET` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:17:50.739993 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a39fa6ae-b08a-4bd7-be86-08006569d4cb None admin] SCRIPT_NAME: `/identity` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:17:50.740283 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a39fa6ae-b08a-4bd7-be86-08006569d4cb None admin] PATH_INFO: `/v3/services` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:17:50.742041 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a39fa6ae-b08a-4bd7-be86-08006569d4cb None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=80245) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 08:17:50.742371 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a39fa6ae-b08a-4bd7-be86-08006569d4cb None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:17:50.744582 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a39fa6ae-b08a-4bd7-be86-08006569d4cb None admin] RBAC: Authorization granted {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:17:50.750683 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: [pid: 80245|app: 0|req: 465/925] 10.222.0.47 () {56 vars in 1126 bytes} [Thu Jul 21 08:17:50 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 08:17:50.758127 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a39fa6ae-b08a-4bd7-be86-08006569d4cb None admin] Authenticating user token {{(pid=80245) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:17:50.762919 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75407a97-a6e4-430e-98eb-848abae75e42 None admin] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:17:50.800073 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9caeb3bd-545a-4ee1-a429-ab6c787fa468 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:17:50.800855 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9caeb3bd-545a-4ee1-a429-ab6c787fa468 None admin] REQUEST_METHOD: `GET` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:17:50.801148 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9caeb3bd-545a-4ee1-a429-ab6c787fa468 None admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:17:50.801428 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9caeb3bd-545a-4ee1-a429-ab6c787fa468 None admin] PATH_INFO: `/v3/roles` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:17:50.802192 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9caeb3bd-545a-4ee1-a429-ab6c787fa468 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80244) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 08:17:50.802530 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9caeb3bd-545a-4ee1-a429-ab6c787fa468 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:17:50.804119 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9caeb3bd-545a-4ee1-a429-ab6c787fa468 None admin] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:17:50.813122 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 461/926] 10.222.0.47 () {56 vars in 1115 bytes} [Thu Jul 21 08:17:50 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 21 08:17:50.819100 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9caeb3bd-545a-4ee1-a429-ab6c787fa468 None admin] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:17:50.856440 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG dbcounter [-] [80245] Writing DB stats keystone:SELECT=100 {{(pid=80245) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:17:50.876353 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a39fa6ae-b08a-4bd7-be86-08006569d4cb None admin] Validating token access rules against request {{(pid=80245) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:17:50.881640 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG dbcounter [-] [80244] Writing DB stats keystone:SELECT=100 {{(pid=80244) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:17:50.923919 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b051390-e28b-4eac-9b24-fd9aaed9ff74 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80245) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:17:50.925237 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b051390-e28b-4eac-9b24-fd9aaed9ff74 None admin] REQUEST_METHOD: `GET` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:17:50.925598 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b051390-e28b-4eac-9b24-fd9aaed9ff74 None admin] SCRIPT_NAME: `/identity` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:17:50.925949 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b051390-e28b-4eac-9b24-fd9aaed9ff74 None admin] PATH_INFO: `/v3/services` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:17:50.926981 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b051390-e28b-4eac-9b24-fd9aaed9ff74 None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=80245) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 08:17:50.927292 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b051390-e28b-4eac-9b24-fd9aaed9ff74 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:17:50.929518 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b051390-e28b-4eac-9b24-fd9aaed9ff74 None admin] RBAC: Authorization granted {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:17:50.936887 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: [pid: 80245|app: 0|req: 466/927] 10.222.0.47 () {56 vars in 1126 bytes} [Thu Jul 21 08:17:50 2022] GET /identity/v3/services?type=compute_legacy => generated 386 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 08:17:50.937685 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9caeb3bd-545a-4ee1-a429-ab6c787fa468 None admin] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:17:50.942362 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b051390-e28b-4eac-9b24-fd9aaed9ff74 None admin] Authenticating user token {{(pid=80245) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:17:50.978278 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64c8372e-72f5-441f-8a2e-ede0448e5807 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:17:50.979476 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64c8372e-72f5-441f-8a2e-ede0448e5807 None admin] REQUEST_METHOD: `GET` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:17:50.979982 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64c8372e-72f5-441f-8a2e-ede0448e5807 None admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:17:50.980457 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64c8372e-72f5-441f-8a2e-ede0448e5807 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:17:50.991977 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64c8372e-72f5-441f-8a2e-ede0448e5807 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:17:50.993897 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64c8372e-72f5-441f-8a2e-ede0448e5807 None admin] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:17:51.012387 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application [None req-64c8372e-72f5-441f-8a2e-ede0448e5807 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 08:17:51.012387 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 08:17:51.012387 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 08:17:51.012387 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 08:17:51.012387 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 08:17:51.012387 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 08:17:51.012387 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 08:17:51.012387 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 08:17:51.012387 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 08:17:51.012387 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 08:17:51.012387 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 08:17:51.012387 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 08:17:51.012387 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 21 08:17:51.012387 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 21 08:17:51.012387 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 21 08:17:51.013292 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 21 08:17:51.013292 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 08:17:51.013292 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 08:17:51.013292 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 08:17:51.013292 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 08:17:51.013292 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 08:17:51.013292 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 08:17:51.013292 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 08:17:51.013292 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application with Lock( Jul 21 08:17:51.013292 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 08:17:51.013292 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self._enter() Jul 21 08:17:51.013292 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 08:17:51.013292 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 08:17:51.013292 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 08:17:51.013292 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self.creator() Jul 21 08:17:51.013292 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 08:17:51.013292 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application created_value = creator( Jul 21 08:17:51.014152 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 21 08:17:51.014152 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 21 08:17:51.014152 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 08:17:51.014152 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application Jul 21 08:17:51.021567 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 462/928] 10.222.0.47 () {56 vars in 1108 bytes} [Thu Jul 21 08:17:50 2022] GET /identity/v3/domains/Default => generated 87 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 21 08:17:51.022751 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64c8372e-72f5-441f-8a2e-ede0448e5807 None admin] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:17:51.050656 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b051390-e28b-4eac-9b24-fd9aaed9ff74 None admin] Validating token access rules against request {{(pid=80245) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:17:51.089695 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a1b2fc2-8eaf-4c43-8613-05c624ba1ef5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80245) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:17:51.090368 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a1b2fc2-8eaf-4c43-8613-05c624ba1ef5 None admin] REQUEST_METHOD: `POST` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:17:51.090652 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a1b2fc2-8eaf-4c43-8613-05c624ba1ef5 None admin] SCRIPT_NAME: `/identity` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:17:51.090930 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a1b2fc2-8eaf-4c43-8613-05c624ba1ef5 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:17:51.091710 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a1b2fc2-8eaf-4c43-8613-05c624ba1ef5 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:17:51.093319 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a1b2fc2-8eaf-4c43-8613-05c624ba1ef5 None admin] RBAC: Authorization granted {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:17:51.123802 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: [pid: 80245|app: 0|req: 467/929] 10.222.0.47 () {60 vars in 1144 bytes} [Thu Jul 21 08:17:50 2022] POST /identity/v3/endpoints => generated 347 bytes in 177 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 21 08:17:51.147746 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64c8372e-72f5-441f-8a2e-ede0448e5807 None admin] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:17:51.158536 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG dbcounter [-] [80244] Writing DB stats keystone:SELECT=100 {{(pid=80244) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:17:51.194849 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8030188d-40cf-4348-a30b-761644da532e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:17:51.195568 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8030188d-40cf-4348-a30b-761644da532e None admin] REQUEST_METHOD: `GET` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:17:51.195854 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8030188d-40cf-4348-a30b-761644da532e None admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:17:51.196180 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8030188d-40cf-4348-a30b-761644da532e None admin] PATH_INFO: `/v3/domains` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:17:51.196927 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8030188d-40cf-4348-a30b-761644da532e None admin] RBAC: Adding query filter params (name=Default) {{(pid=80244) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 08:17:51.197230 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8030188d-40cf-4348-a30b-761644da532e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:17:51.198655 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8030188d-40cf-4348-a30b-761644da532e None admin] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:17:51.212735 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 463/930] 10.222.0.47 () {56 vars in 1109 bytes} [Thu Jul 21 08:17:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 08:17:51.218791 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a1b2fc2-8eaf-4c43-8613-05c624ba1ef5 None admin] Authenticating user token {{(pid=80245) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:17:51.233133 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG dbcounter [-] [80245] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80245) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:17:51.353487 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a1b2fc2-8eaf-4c43-8613-05c624ba1ef5 None admin] Validating token access rules against request {{(pid=80245) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:17:51.389968 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34a0ff28-7748-47f2-9a8d-93ea941d0636 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80245) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:17:51.390747 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34a0ff28-7748-47f2-9a8d-93ea941d0636 None admin] REQUEST_METHOD: `GET` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:17:51.391021 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34a0ff28-7748-47f2-9a8d-93ea941d0636 None admin] SCRIPT_NAME: `/identity` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:17:51.391295 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34a0ff28-7748-47f2-9a8d-93ea941d0636 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:17:51.401031 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34a0ff28-7748-47f2-9a8d-93ea941d0636 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:17:51.402492 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34a0ff28-7748-47f2-9a8d-93ea941d0636 None admin] RBAC: Authorization granted {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:17:51.418889 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application [None req-34a0ff28-7748-47f2-9a8d-93ea941d0636 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 21 08:17:51.418889 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 08:17:51.418889 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 08:17:51.418889 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 08:17:51.418889 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 08:17:51.418889 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 08:17:51.418889 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 08:17:51.418889 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 08:17:51.418889 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 08:17:51.418889 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 08:17:51.418889 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 08:17:51.418889 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 08:17:51.418889 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 21 08:17:51.418889 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 21 08:17:51.418889 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 21 08:17:51.418889 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 21 08:17:51.419933 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 08:17:51.419933 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 08:17:51.419933 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 21 08:17:51.419933 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 21 08:17:51.419933 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 21 08:17:51.419933 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 21 08:17:51.419933 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 08:17:51.419933 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 08:17:51.419933 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 08:17:51.419933 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 08:17:51.419933 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 08:17:51.419933 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application with Lock( Jul 21 08:17:51.419933 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 08:17:51.419933 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application return self._enter() Jul 21 08:17:51.419933 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 08:17:51.419933 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 08:17:51.419933 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 08:17:51.419933 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application return self.creator() Jul 21 08:17:51.420861 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 08:17:51.420861 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application created_value = creator( Jul 21 08:17:51.420861 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 21 08:17:51.420861 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 21 08:17:51.420861 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 21 08:17:51.420861 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 21 08:17:51.420861 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 21 08:17:51.420861 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 21 08:17:51.420861 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 21 08:17:51.420861 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application Jul 21 08:17:51.420861 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: [pid: 80245|app: 0|req: 468/931] 10.222.0.47 () {56 vars in 1117 bytes} [Thu Jul 21 08:17:51 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 08:17:51.426469 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8030188d-40cf-4348-a30b-761644da532e None admin] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:17:51.536790 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8030188d-40cf-4348-a30b-761644da532e None admin] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:17:51.572644 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a57c3e3-9be6-49bc-8594-1e530b7128cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:17:51.573322 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a57c3e3-9be6-49bc-8594-1e530b7128cc None admin] REQUEST_METHOD: `GET` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:17:51.573603 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a57c3e3-9be6-49bc-8594-1e530b7128cc None admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:17:51.573874 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a57c3e3-9be6-49bc-8594-1e530b7128cc None admin] PATH_INFO: `/v3/users` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:17:51.574714 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a57c3e3-9be6-49bc-8594-1e530b7128cc None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80244) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 08:17:51.575019 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a57c3e3-9be6-49bc-8594-1e530b7128cc None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:17:51.576472 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a57c3e3-9be6-49bc-8594-1e530b7128cc None admin] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:17:51.595527 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 464/932] 10.222.0.47 () {56 vars in 1149 bytes} [Thu Jul 21 08:17:51 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 08:17:51.601588 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34a0ff28-7748-47f2-9a8d-93ea941d0636 None admin] Authenticating user token {{(pid=80245) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:17:51.651712 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c81a527-f9b6-4f50-a93d-cb22d77e2571 None None] REQUEST_METHOD: `GET` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:17:51.652372 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c81a527-f9b6-4f50-a93d-cb22d77e2571 None None] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:17:51.652748 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c81a527-f9b6-4f50-a93d-cb22d77e2571 None None] PATH_INFO: `/` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:17:51.654569 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 465/933] 10.222.0.47 () {54 vars in 926 bytes} [Thu Jul 21 08:17:51 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 21 08:17:51.660460 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c704d315-83e7-4aa8-ad3f-3943c83b1352 None None] REQUEST_METHOD: `POST` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:17:51.660750 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c704d315-83e7-4aa8-ad3f-3943c83b1352 None None] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:17:51.661032 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c704d315-83e7-4aa8-ad3f-3943c83b1352 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:17:51.676323 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG dbcounter [-] [80245] Writing DB stats keystone:SELECT=100 {{(pid=80245) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:17:51.678918 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.auth.core [None req-c704d315-83e7-4aa8-ad3f-3943c83b1352 None None] MFA Rules not processed for user `2dce40b8d74c41ec94cc36e6ff05d412`. Rule list: `[]` (Enabled: `True`). {{(pid=80244) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 08:17:51.681116 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.fernet_utils [None req-c704d315-83e7-4aa8-ad3f-3943c83b1352 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80244) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 08:17:51.699582 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG dbcounter [-] [80244] Writing DB stats keystone:SELECT=100 {{(pid=80244) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:17:51.713738 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34a0ff28-7748-47f2-9a8d-93ea941d0636 None admin] Validating token access rules against request {{(pid=80245) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:17:51.750344 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9751bb73-2a40-4ce7-844e-647fd30ac03d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80245) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:17:51.751447 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9751bb73-2a40-4ce7-844e-647fd30ac03d None admin] REQUEST_METHOD: `GET` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:17:51.751892 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9751bb73-2a40-4ce7-844e-647fd30ac03d None admin] SCRIPT_NAME: `/identity` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:17:51.752173 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9751bb73-2a40-4ce7-844e-647fd30ac03d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:17:51.756467 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 466/934] 10.222.0.47 () {58 vars in 1023 bytes} [Thu Jul 21 08:17:51 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 21 08:17:51.757967 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0d935e6-a732-4b1c-936e-1fbabfc63902 None None] REQUEST_METHOD: `GET` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:17:51.758740 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0d935e6-a732-4b1c-936e-1fbabfc63902 None None] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:17:51.759146 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0d935e6-a732-4b1c-936e-1fbabfc63902 None None] PATH_INFO: `/` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:17:51.760604 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 467/935] 10.222.0.47 () {54 vars in 926 bytes} [Thu Jul 21 08:17:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 08:17:51.766057 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bf96db4-3963-4e5c-9215-3d24fab8d5a3 None None] REQUEST_METHOD: `POST` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:17:51.766530 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bf96db4-3963-4e5c-9215-3d24fab8d5a3 None None] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:17:51.766970 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bf96db4-3963-4e5c-9215-3d24fab8d5a3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:17:51.767441 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9751bb73-2a40-4ce7-844e-647fd30ac03d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:17:51.769670 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9751bb73-2a40-4ce7-844e-647fd30ac03d None admin] RBAC: Authorization granted {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:17:51.784504 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.auth.core [None req-7bf96db4-3963-4e5c-9215-3d24fab8d5a3 None None] MFA Rules not processed for user `2dce40b8d74c41ec94cc36e6ff05d412`. Rule list: `[]` (Enabled: `True`). {{(pid=80244) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 08:17:51.785988 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.fernet_utils [None req-7bf96db4-3963-4e5c-9215-3d24fab8d5a3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80244) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 08:17:51.788089 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application [None req-9751bb73-2a40-4ce7-844e-647fd30ac03d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 08:17:51.788089 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 08:17:51.788089 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 08:17:51.788089 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 08:17:51.788089 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 08:17:51.788089 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 08:17:51.788089 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 08:17:51.788089 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 08:17:51.788089 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 08:17:51.788089 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 08:17:51.788089 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 08:17:51.788089 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 08:17:51.788089 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 21 08:17:51.788089 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 21 08:17:51.788089 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 21 08:17:51.788938 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 21 08:17:51.788938 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 08:17:51.788938 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 08:17:51.788938 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 08:17:51.788938 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 08:17:51.788938 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 08:17:51.788938 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 08:17:51.788938 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 08:17:51.788938 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application with Lock( Jul 21 08:17:51.788938 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 08:17:51.788938 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application return self._enter() Jul 21 08:17:51.788938 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 08:17:51.788938 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 08:17:51.788938 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 08:17:51.788938 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application return self.creator() Jul 21 08:17:51.788938 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 08:17:51.788938 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application created_value = creator( Jul 21 08:17:51.789811 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 21 08:17:51.789811 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 21 08:17:51.789811 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 08:17:51.789811 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application Jul 21 08:17:51.789811 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: [pid: 80245|app: 0|req: 469/936] 10.222.0.47 () {56 vars in 1108 bytes} [Thu Jul 21 08:17:51 2022] GET /identity/v3/domains/Default => generated 87 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 08:17:51.790087 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9751bb73-2a40-4ce7-844e-647fd30ac03d None admin] Authenticating user token {{(pid=80245) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:17:51.863156 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 468/937] 10.222.0.47 () {58 vars in 1023 bytes} [Thu Jul 21 08:17:51 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 21 08:17:51.864205 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bf96db4-3963-4e5c-9215-3d24fab8d5a3 None None] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:17:51.901066 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9751bb73-2a40-4ce7-844e-647fd30ac03d None admin] Validating token access rules against request {{(pid=80245) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:17:51.924872 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG dbcounter [-] [80245] Writing DB stats keystone:SELECT=100 {{(pid=80245) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:17:51.941432 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-548b75e1-6637-4b86-9d18-7712d675b50e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80245) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:17:51.942252 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-548b75e1-6637-4b86-9d18-7712d675b50e None admin] REQUEST_METHOD: `GET` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:17:51.942533 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-548b75e1-6637-4b86-9d18-7712d675b50e None admin] SCRIPT_NAME: `/identity` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:17:51.942806 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-548b75e1-6637-4b86-9d18-7712d675b50e None admin] PATH_INFO: `/v3/services/object-store` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:17:51.943494 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-548b75e1-6637-4b86-9d18-7712d675b50e None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:17:51.944934 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-548b75e1-6637-4b86-9d18-7712d675b50e None admin] RBAC: Authorization granted {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:17:51.957753 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application [None req-548b75e1-6637-4b86-9d18-7712d675b50e None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 21 08:17:51.957753 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 08:17:51.957753 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 21 08:17:51.957753 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 21 08:17:51.957753 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 21 08:17:51.957753 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 21 08:17:51.957753 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 21 08:17:51.957753 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 21 08:17:51.957753 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 21 08:17:51.957753 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application Jul 21 08:17:51.957753 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 21 08:17:51.957753 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application Jul 21 08:17:51.957753 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 08:17:51.957753 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 08:17:51.957753 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 08:17:51.957753 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 08:17:51.957753 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 08:17:51.958927 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 08:17:51.958927 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 08:17:51.958927 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 08:17:51.958927 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 08:17:51.958927 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 08:17:51.958927 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 08:17:51.958927 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 21 08:17:51.958927 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 21 08:17:51.958927 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 21 08:17:51.958927 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 21 08:17:51.958927 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 08:17:51.958927 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 08:17:51.958927 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 08:17:51.958927 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 08:17:51.958927 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 08:17:51.958927 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 08:17:51.960164 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 08:17:51.960164 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application with Lock( Jul 21 08:17:51.960164 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 08:17:51.960164 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application return self._enter() Jul 21 08:17:51.960164 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 08:17:51.960164 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 08:17:51.960164 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 08:17:51.960164 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application return self.creator() Jul 21 08:17:51.960164 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 08:17:51.960164 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application created_value = creator( Jul 21 08:17:51.960164 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 21 08:17:51.960164 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 21 08:17:51.960164 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 21 08:17:51.960164 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application Jul 21 08:17:51.960164 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: [pid: 80245|app: 0|req: 470/938] 10.222.0.47 () {56 vars in 1126 bytes} [Thu Jul 21 08:17:51 2022] GET /identity/v3/services/object-store => generated 93 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 08:17:51.960962 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-548b75e1-6637-4b86-9d18-7712d675b50e None admin] Authenticating user token {{(pid=80245) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:17:51.976031 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG dbcounter [-] [80244] Writing DB stats keystone:SELECT=100 {{(pid=80244) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:17:51.997935 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bf96db4-3963-4e5c-9215-3d24fab8d5a3 None None] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:17:52.037293 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36b21e82-ff71-40be-ace9-04d3bb9fc037 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:17:52.038293 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36b21e82-ff71-40be-ace9-04d3bb9fc037 None admin] REQUEST_METHOD: `GET` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:17:52.038755 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36b21e82-ff71-40be-ace9-04d3bb9fc037 None admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:17:52.039205 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36b21e82-ff71-40be-ace9-04d3bb9fc037 None admin] PATH_INFO: `/v3/domains` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:17:52.040131 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36b21e82-ff71-40be-ace9-04d3bb9fc037 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80244) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 08:17:52.040631 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36b21e82-ff71-40be-ace9-04d3bb9fc037 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:17:52.042263 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36b21e82-ff71-40be-ace9-04d3bb9fc037 None admin] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:17:52.055253 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 469/939] 10.222.0.47 () {56 vars in 1109 bytes} [Thu Jul 21 08:17:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 08:17:52.056460 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36b21e82-ff71-40be-ace9-04d3bb9fc037 None admin] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:17:52.071902 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-548b75e1-6637-4b86-9d18-7712d675b50e None admin] Validating token access rules against request {{(pid=80245) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:17:52.114308 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3dced14-cd9b-4bbe-84c2-5541bba29e67 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80245) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:17:52.114982 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3dced14-cd9b-4bbe-84c2-5541bba29e67 None admin] REQUEST_METHOD: `POST` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:17:52.115266 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3dced14-cd9b-4bbe-84c2-5541bba29e67 None admin] SCRIPT_NAME: `/identity` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:17:52.115570 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3dced14-cd9b-4bbe-84c2-5541bba29e67 None admin] PATH_INFO: `/v3/projects` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:17:52.116270 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3dced14-cd9b-4bbe-84c2-5541bba29e67 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:17:52.117725 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3dced14-cd9b-4bbe-84c2-5541bba29e67 None admin] RBAC: Authorization granted {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:17:52.119099 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: WARNING keystone.server.flask.common [None req-e3dced14-cd9b-4bbe-84c2-5541bba29e67 None admin] Deprecated: Not specifying a domain during a create user, group or project call, and relying on falling back to the default domain, is deprecated as of Liberty. There is no plan to remove this compatibility, however, future API versions may remove this, so please specify the domain explicitly or use a domain-scoped token. Jul 21 08:17:52.137750 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.sql.core [None req-e3dced14-cd9b-4bbe-84c2-5541bba29e67 None admin] Conflict project: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Jul 21 08:17:52.137750 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] Jul 21 08:17:52.137750 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: [parameters: {'id': 'bc61fca2106f4a7e8b2fe9e3593c0d2b', 'name': 'service', 'domain_id': 'default', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': 'default', 'is_domain': 0}] Jul 21 08:17:52.137750 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=80245) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Jul 21 08:17:52.156354 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application [None req-e3dced14-cd9b-4bbe-84c2-5541bba29e67 None admin] Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id bc61fca2106f4a7e8b2fe9e3593c0d2b.: keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id bc61fca2106f4a7e8b2fe9e3593c0d2b. Jul 21 08:17:52.156354 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 08:17:52.156354 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Jul 21 08:17:52.156354 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application self.dialect.do_execute( Jul 21 08:17:52.156354 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Jul 21 08:17:52.156354 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Jul 21 08:17:52.156354 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Jul 21 08:17:52.156354 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application result = self._query(query) Jul 21 08:17:52.156354 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Jul 21 08:17:52.156354 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application conn.query(q) Jul 21 08:17:52.156354 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Jul 21 08:17:52.156354 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Jul 21 08:17:52.156354 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Jul 21 08:17:52.157369 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application result.read() Jul 21 08:17:52.157369 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Jul 21 08:17:52.157369 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Jul 21 08:17:52.157369 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Jul 21 08:17:52.157369 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application packet.raise_for_error() Jul 21 08:17:52.157369 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Jul 21 08:17:52.157369 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Jul 21 08:17:52.157369 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Jul 21 08:17:52.157369 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Jul 21 08:17:52.157369 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Jul 21 08:17:52.157369 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application Jul 21 08:17:52.157369 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Jul 21 08:17:52.157369 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application Jul 21 08:17:52.157369 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 08:17:52.157369 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Jul 21 08:17:52.157369 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application return method(*args, **kwargs) Jul 21 08:17:52.157369 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 241, in create_project Jul 21 08:17:52.157369 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application return project_ref.to_dict() Jul 21 08:17:52.158373 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Jul 21 08:17:52.158373 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application next(self.gen) Jul 21 08:17:52.158373 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Jul 21 08:17:52.158373 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application yield resource Jul 21 08:17:52.158373 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Jul 21 08:17:52.158373 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application next(self.gen) Jul 21 08:17:52.158373 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Jul 21 08:17:52.158373 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application self.session.rollback() Jul 21 08:17:52.158373 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Jul 21 08:17:52.158373 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application self.force_reraise() Jul 21 08:17:52.158373 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Jul 21 08:17:52.158373 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application raise self.value Jul 21 08:17:52.158373 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Jul 21 08:17:52.158373 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Jul 21 08:17:52.158373 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Jul 21 08:17:52.158373 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application session.commit() Jul 21 08:17:52.158373 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Jul 21 08:17:52.159252 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Jul 21 08:17:52.159252 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Jul 21 08:17:52.159252 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application self._prepare_impl() Jul 21 08:17:52.159252 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Jul 21 08:17:52.159252 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application self.session.flush() Jul 21 08:17:52.159252 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Jul 21 08:17:52.159252 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application self._flush(objects) Jul 21 08:17:52.159252 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Jul 21 08:17:52.159252 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Jul 21 08:17:52.159252 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Jul 21 08:17:52.159252 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application compat.raise_( Jul 21 08:17:52.159252 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Jul 21 08:17:52.159252 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application raise exception Jul 21 08:17:52.159252 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Jul 21 08:17:52.159252 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application flush_context.execute() Jul 21 08:17:52.159252 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Jul 21 08:17:52.159252 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application rec.execute(self) Jul 21 08:17:52.160213 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Jul 21 08:17:52.160213 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Jul 21 08:17:52.160213 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Jul 21 08:17:52.160213 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application _emit_insert_statements( Jul 21 08:17:52.160213 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1097, in _emit_insert_statements Jul 21 08:17:52.160213 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application c = connection._execute_20( Jul 21 08:17:52.160213 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1631, in _execute_20 Jul 21 08:17:52.160213 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Jul 21 08:17:52.160213 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 332, in _execute_on_connection Jul 21 08:17:52.160213 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Jul 21 08:17:52.160213 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1498, in _execute_clauseelement Jul 21 08:17:52.160213 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application ret = self._execute_context( Jul 21 08:17:52.160213 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1862, in _execute_context Jul 21 08:17:52.160213 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Jul 21 08:17:52.160213 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2041, in _handle_dbapi_exception Jul 21 08:17:52.160999 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Jul 21 08:17:52.160999 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Jul 21 08:17:52.160999 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application raise exception Jul 21 08:17:52.160999 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Jul 21 08:17:52.160999 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application self.dialect.do_execute( Jul 21 08:17:52.160999 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Jul 21 08:17:52.160999 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Jul 21 08:17:52.160999 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Jul 21 08:17:52.160999 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application result = self._query(query) Jul 21 08:17:52.160999 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Jul 21 08:17:52.160999 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application conn.query(q) Jul 21 08:17:52.160999 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Jul 21 08:17:52.160999 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Jul 21 08:17:52.160999 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Jul 21 08:17:52.160999 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application result.read() Jul 21 08:17:52.160999 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Jul 21 08:17:52.160999 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Jul 21 08:17:52.161882 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Jul 21 08:17:52.161882 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application packet.raise_for_error() Jul 21 08:17:52.161882 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Jul 21 08:17:52.161882 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Jul 21 08:17:52.161882 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Jul 21 08:17:52.161882 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Jul 21 08:17:52.161882 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Jul 21 08:17:52.161882 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] Jul 21 08:17:52.161882 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application [parameters: {'id': 'bc61fca2106f4a7e8b2fe9e3593c0d2b', 'name': 'service', 'domain_id': 'default', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': 'default', 'is_domain': 0}] Jul 21 08:17:52.161882 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Jul 21 08:17:52.161882 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application Jul 21 08:17:52.161882 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 21 08:17:52.161882 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application Jul 21 08:17:52.161882 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 08:17:52.161882 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 216, in create_project Jul 21 08:17:52.162667 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application ret = self.driver.create_project(project_id, project) Jul 21 08:17:52.162667 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Jul 21 08:17:52.162667 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Jul 21 08:17:52.162667 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - Duplicate entry found with name service at domain ID default. Jul 21 08:17:52.162667 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application Jul 21 08:17:52.162667 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 21 08:17:52.162667 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application Jul 21 08:17:52.162667 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 08:17:52.162667 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 08:17:52.162667 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 08:17:52.162667 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 08:17:52.162667 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 08:17:52.162667 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 08:17:52.162667 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 08:17:52.162667 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 08:17:52.162667 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 08:17:52.162667 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 08:17:52.163580 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 08:17:52.163580 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 176, in post Jul 21 08:17:52.163580 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application ref = PROVIDERS.resource_api.create_project( Jul 21 08:17:52.163580 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 08:17:52.163580 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 08:17:52.163580 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 218, in create_project Jul 21 08:17:52.163580 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application raise exception.Conflict( Jul 21 08:17:52.163580 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id bc61fca2106f4a7e8b2fe9e3593c0d2b. Jul 21 08:17:52.163580 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application Jul 21 08:17:52.163580 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: [pid: 80245|app: 0|req: 471/940] 10.222.0.47 () {60 vars in 1140 bytes} [Thu Jul 21 08:17:51 2022] POST /identity/v3/projects => generated 262 bytes in 198 msecs (HTTP/1.1 409) 5 headers in 183 bytes (1 switches on core 0) Jul 21 08:17:52.163580 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3dced14-cd9b-4bbe-84c2-5541bba29e67 None admin] Authenticating user token {{(pid=80245) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:17:52.192975 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36b21e82-ff71-40be-ace9-04d3bb9fc037 None admin] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:17:52.207478 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG dbcounter [-] [80245] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80245) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:17:52.229290 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-559d20e0-9791-40ad-b574-9a3f4cc0f143 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:17:52.229966 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-559d20e0-9791-40ad-b574-9a3f4cc0f143 None admin] REQUEST_METHOD: `GET` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:17:52.230254 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-559d20e0-9791-40ad-b574-9a3f4cc0f143 None admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:17:52.230532 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-559d20e0-9791-40ad-b574-9a3f4cc0f143 None admin] PATH_INFO: `/v3/services` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:17:52.231242 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-559d20e0-9791-40ad-b574-9a3f4cc0f143 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=80244) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 08:17:52.231698 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-559d20e0-9791-40ad-b574-9a3f4cc0f143 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:17:52.232954 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-559d20e0-9791-40ad-b574-9a3f4cc0f143 None admin] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:17:52.238898 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 470/941] 10.222.0.47 () {56 vars in 1122 bytes} [Thu Jul 21 08:17:52 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 08:17:52.239546 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-559d20e0-9791-40ad-b574-9a3f4cc0f143 None admin] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:17:52.254776 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG dbcounter [-] [80244] Writing DB stats keystone:SELECT=100 {{(pid=80244) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:17:52.278977 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3dced14-cd9b-4bbe-84c2-5541bba29e67 None admin] Validating token access rules against request {{(pid=80245) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:17:52.313044 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfec7faf-ec42-4138-b5d9-6de3b1c94a73 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80245) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:17:52.313844 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfec7faf-ec42-4138-b5d9-6de3b1c94a73 None admin] REQUEST_METHOD: `GET` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:17:52.314125 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfec7faf-ec42-4138-b5d9-6de3b1c94a73 None admin] SCRIPT_NAME: `/identity` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:17:52.314555 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfec7faf-ec42-4138-b5d9-6de3b1c94a73 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:17:52.321750 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfec7faf-ec42-4138-b5d9-6de3b1c94a73 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:17:52.323299 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfec7faf-ec42-4138-b5d9-6de3b1c94a73 None admin] RBAC: Authorization granted {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:17:52.337374 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application [None req-cfec7faf-ec42-4138-b5d9-6de3b1c94a73 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 21 08:17:52.337374 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 08:17:52.337374 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 08:17:52.337374 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 08:17:52.337374 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 08:17:52.337374 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 08:17:52.337374 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 08:17:52.337374 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 08:17:52.337374 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 08:17:52.337374 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 08:17:52.337374 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 08:17:52.337374 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 08:17:52.337374 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 21 08:17:52.337374 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 21 08:17:52.337374 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 21 08:17:52.338443 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 21 08:17:52.338443 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 08:17:52.338443 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 08:17:52.338443 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 08:17:52.338443 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 08:17:52.338443 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 08:17:52.338443 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 08:17:52.338443 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 08:17:52.338443 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application with Lock( Jul 21 08:17:52.338443 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 08:17:52.338443 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application return self._enter() Jul 21 08:17:52.338443 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 08:17:52.338443 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 08:17:52.338443 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 08:17:52.338443 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application return self.creator() Jul 21 08:17:52.338443 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 08:17:52.338443 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application created_value = creator( Jul 21 08:17:52.339364 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 21 08:17:52.339364 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 21 08:17:52.339364 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 21 08:17:52.339364 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 21 08:17:52.339364 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 21 08:17:52.339364 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 21 08:17:52.339364 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 21 08:17:52.339364 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application Jul 21 08:17:52.339364 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: [pid: 80245|app: 0|req: 472/942] 10.222.0.47 () {56 vars in 1111 bytes} [Thu Jul 21 08:17:52 2022] GET /identity/v3/projects/service => generated 88 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 08:17:52.339844 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfec7faf-ec42-4138-b5d9-6de3b1c94a73 None admin] Authenticating user token {{(pid=80245) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:17:52.368418 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-559d20e0-9791-40ad-b574-9a3f4cc0f143 None admin] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:17:52.407501 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab5cf778-87ff-48a1-82ff-6eaf772d1d40 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:17:52.408363 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab5cf778-87ff-48a1-82ff-6eaf772d1d40 None admin] REQUEST_METHOD: `GET` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:17:52.408653 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab5cf778-87ff-48a1-82ff-6eaf772d1d40 None admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:17:52.408928 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab5cf778-87ff-48a1-82ff-6eaf772d1d40 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:17:52.416801 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab5cf778-87ff-48a1-82ff-6eaf772d1d40 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:17:52.418392 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab5cf778-87ff-48a1-82ff-6eaf772d1d40 None admin] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:17:52.434780 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application [None req-ab5cf778-87ff-48a1-82ff-6eaf772d1d40 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 21 08:17:52.434780 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 08:17:52.434780 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 08:17:52.434780 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 08:17:52.434780 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 08:17:52.434780 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 08:17:52.434780 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 08:17:52.434780 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 08:17:52.434780 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 08:17:52.434780 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 08:17:52.434780 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 08:17:52.434780 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 08:17:52.434780 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 21 08:17:52.434780 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 21 08:17:52.434780 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 21 08:17:52.436381 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 21 08:17:52.436381 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 08:17:52.436381 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 08:17:52.436381 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 08:17:52.436381 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 08:17:52.436381 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 08:17:52.436381 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 08:17:52.436381 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 08:17:52.436381 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application with Lock( Jul 21 08:17:52.436381 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 08:17:52.436381 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self._enter() Jul 21 08:17:52.436381 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 08:17:52.436381 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 08:17:52.436381 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 08:17:52.436381 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self.creator() Jul 21 08:17:52.436381 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 08:17:52.436381 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application created_value = creator( Jul 21 08:17:52.437333 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 21 08:17:52.437333 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 21 08:17:52.437333 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 21 08:17:52.437333 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 21 08:17:52.437333 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 21 08:17:52.437333 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 21 08:17:52.437333 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 21 08:17:52.437333 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application Jul 21 08:17:52.439662 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 471/943] 10.222.0.47 () {56 vars in 1111 bytes} [Thu Jul 21 08:17:52 2022] GET /identity/v3/projects/service => generated 88 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 08:17:52.441366 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab5cf778-87ff-48a1-82ff-6eaf772d1d40 None admin] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:17:52.452231 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG dbcounter [-] [80245] Writing DB stats keystone:SELECT=100 {{(pid=80245) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:17:52.455749 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfec7faf-ec42-4138-b5d9-6de3b1c94a73 None admin] Validating token access rules against request {{(pid=80245) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:17:52.499790 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc376041-9b26-4aba-9146-8f27a2f52064 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80245) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:17:52.500638 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc376041-9b26-4aba-9146-8f27a2f52064 None admin] REQUEST_METHOD: `GET` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:17:52.500923 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc376041-9b26-4aba-9146-8f27a2f52064 None admin] SCRIPT_NAME: `/identity` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:17:52.501195 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc376041-9b26-4aba-9146-8f27a2f52064 None admin] PATH_INFO: `/v3/services` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:17:52.501899 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc376041-9b26-4aba-9146-8f27a2f52064 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=80245) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 08:17:52.502202 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc376041-9b26-4aba-9146-8f27a2f52064 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:17:52.503620 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc376041-9b26-4aba-9146-8f27a2f52064 None admin] RBAC: Authorization granted {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:17:52.511798 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: [pid: 80245|app: 0|req: 473/944] 10.222.0.47 () {56 vars in 1122 bytes} [Thu Jul 21 08:17:52 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 08:17:52.512418 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc376041-9b26-4aba-9146-8f27a2f52064 None admin] Authenticating user token {{(pid=80245) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:17:52.525629 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG dbcounter [-] [80244] Writing DB stats keystone:SELECT=100 {{(pid=80244) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:17:52.563918 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab5cf778-87ff-48a1-82ff-6eaf772d1d40 None admin] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:17:52.600206 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0669bf8a-c764-449f-bad7-7ab7a6ab03b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:17:52.600872 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0669bf8a-c764-449f-bad7-7ab7a6ab03b0 None admin] REQUEST_METHOD: `GET` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:17:52.601162 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0669bf8a-c764-449f-bad7-7ab7a6ab03b0 None admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:17:52.601428 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0669bf8a-c764-449f-bad7-7ab7a6ab03b0 None admin] PATH_INFO: `/v3/projects` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:17:52.602201 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0669bf8a-c764-449f-bad7-7ab7a6ab03b0 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80244) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 08:17:52.602496 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0669bf8a-c764-449f-bad7-7ab7a6ab03b0 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:17:52.603983 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0669bf8a-c764-449f-bad7-7ab7a6ab03b0 None admin] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:17:52.616539 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 472/945] 10.222.0.47 () {56 vars in 1148 bytes} [Thu Jul 21 08:17:52 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 08:17:52.617162 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0669bf8a-c764-449f-bad7-7ab7a6ab03b0 None admin] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:17:52.623840 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc376041-9b26-4aba-9146-8f27a2f52064 None admin] Validating token access rules against request {{(pid=80245) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:17:52.660414 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0552bb6-feb9-4c06-99e1-fe4912d597e8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80245) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:17:52.661083 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0552bb6-feb9-4c06-99e1-fe4912d597e8 None admin] REQUEST_METHOD: `GET` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:17:52.661370 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0552bb6-feb9-4c06-99e1-fe4912d597e8 None admin] SCRIPT_NAME: `/identity` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:17:52.661655 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0552bb6-feb9-4c06-99e1-fe4912d597e8 None admin] PATH_INFO: `/v3/projects` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:17:52.662407 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0552bb6-feb9-4c06-99e1-fe4912d597e8 None admin] RBAC: Adding query filter params (name=service) {{(pid=80245) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 08:17:52.662717 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0552bb6-feb9-4c06-99e1-fe4912d597e8 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:17:52.664247 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0552bb6-feb9-4c06-99e1-fe4912d597e8 None admin] RBAC: Authorization granted {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:17:52.689078 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: [pid: 80245|app: 0|req: 474/946] 10.222.0.47 () {56 vars in 1112 bytes} [Thu Jul 21 08:17:52 2022] GET /identity/v3/projects?name=service => generated 411 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 08:17:52.689078 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0552bb6-feb9-4c06-99e1-fe4912d597e8 None admin] Authenticating user token {{(pid=80245) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:17:52.713207 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG dbcounter [-] [80245] Writing DB stats keystone:SELECT=100 {{(pid=80245) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:17:52.748645 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0669bf8a-c764-449f-bad7-7ab7a6ab03b0 None admin] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:17:52.789160 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG dbcounter [-] [80244] Writing DB stats keystone:SELECT=100 {{(pid=80244) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:17:52.801629 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a39e32c-b76c-4f1d-8589-e2f56848677f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:17:52.802004 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0552bb6-feb9-4c06-99e1-fe4912d597e8 None admin] Validating token access rules against request {{(pid=80245) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:17:52.802477 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a39e32c-b76c-4f1d-8589-e2f56848677f None admin] REQUEST_METHOD: `GET` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:17:52.802836 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a39e32c-b76c-4f1d-8589-e2f56848677f None admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:17:52.803223 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a39e32c-b76c-4f1d-8589-e2f56848677f None admin] PATH_INFO: `/v3/endpoints` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:17:52.804489 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a39e32c-b76c-4f1d-8589-e2f56848677f None admin] RBAC: Adding query filter params (interface=public, service_id=53d339aeb0184b07a50e71eed278f656, region_id=RegionOne) {{(pid=80244) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 08:17:52.805878 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a39e32c-b76c-4f1d-8589-e2f56848677f None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:17:52.807401 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a39e32c-b76c-4f1d-8589-e2f56848677f None admin] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:17:52.821657 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 473/947] 10.222.0.47 () {56 vars in 1251 bytes} [Thu Jul 21 08:17:52 2022] GET /identity/v3/endpoints?service_id=53d339aeb0184b07a50e71eed278f656&interface=public®ion_id=RegionOne => generated 194 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 08:17:52.828403 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a39e32c-b76c-4f1d-8589-e2f56848677f None admin] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:17:52.850973 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c091045f-36ae-4667-b1cd-c1276c255487 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80245) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:17:52.852533 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c091045f-36ae-4667-b1cd-c1276c255487 None admin] REQUEST_METHOD: `PUT` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:17:52.852972 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c091045f-36ae-4667-b1cd-c1276c255487 None admin] SCRIPT_NAME: `/identity` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:17:52.853396 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c091045f-36ae-4667-b1cd-c1276c255487 None admin] PATH_INFO: `/v3/projects/4faef13aa50e4e36b6c265fb1398fa14/users/2ef7f1e91e554a1f88eb6a63e60a475f/roles/4b8dff36c32b49618b303451de1d1f69` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:17:52.856958 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c091045f-36ae-4667-b1cd-c1276c255487 None admin] RBAC: Authorizing `identity:create_grant(project_id=4faef13aa50e4e36b6c265fb1398fa14, user_id=2ef7f1e91e554a1f88eb6a63e60a475f, role_id=4b8dff36c32b49618b303451de1d1f69)` {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:17:52.859576 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c091045f-36ae-4667-b1cd-c1276c255487 None admin] RBAC: Authorization granted {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:17:52.882626 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: [pid: 80245|app: 0|req: 475/948] 10.222.0.47 () {58 vars in 1439 bytes} [Thu Jul 21 08:17:52 2022] PUT /identity/v3/projects/4faef13aa50e4e36b6c265fb1398fa14/users/2ef7f1e91e554a1f88eb6a63e60a475f/roles/4b8dff36c32b49618b303451de1d1f69 => generated 0 bytes in 200 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 08:17:52.958454 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a39e32c-b76c-4f1d-8589-e2f56848677f None admin] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:17:52.998148 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27cbf5f4-83ef-43fc-85c5-0acfb7a5ceeb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:17:52.999090 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27cbf5f4-83ef-43fc-85c5-0acfb7a5ceeb None admin] REQUEST_METHOD: `GET` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:17:52.999571 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27cbf5f4-83ef-43fc-85c5-0acfb7a5ceeb None admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:17:53.000033 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27cbf5f4-83ef-43fc-85c5-0acfb7a5ceeb None admin] PATH_INFO: `/v3/services` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:17:53.000902 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27cbf5f4-83ef-43fc-85c5-0acfb7a5ceeb None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:17:53.002469 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27cbf5f4-83ef-43fc-85c5-0acfb7a5ceeb None admin] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:17:53.010391 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 474/949] 10.222.0.47 () {56 vars in 1087 bytes} [Thu Jul 21 08:17:52 2022] GET /identity/v3/services => generated 1497 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 08:17:53.354197 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3572925b-0775-46cf-9065-f07643e88432 None None] REQUEST_METHOD: `GET` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:17:53.354450 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3572925b-0775-46cf-9065-f07643e88432 None None] SCRIPT_NAME: `/identity` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:17:53.354680 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3572925b-0775-46cf-9065-f07643e88432 None None] PATH_INFO: `/` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:17:53.356354 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: [pid: 80245|app: 0|req: 476/950] 10.222.0.47 () {54 vars in 926 bytes} [Thu Jul 21 08:17:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 08:17:53.362006 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32ebce05-299f-4007-8ddf-6d573e480fa5 None None] REQUEST_METHOD: `POST` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:17:53.362347 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32ebce05-299f-4007-8ddf-6d573e480fa5 None None] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:17:53.362634 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32ebce05-299f-4007-8ddf-6d573e480fa5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:17:53.383581 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.auth.core [None req-32ebce05-299f-4007-8ddf-6d573e480fa5 None None] MFA Rules not processed for user `2dce40b8d74c41ec94cc36e6ff05d412`. Rule list: `[]` (Enabled: `True`). {{(pid=80244) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 08:17:53.384867 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.fernet_utils [None req-32ebce05-299f-4007-8ddf-6d573e480fa5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80244) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 08:17:53.438274 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG dbcounter [-] [80244] Writing DB stats keystone:SELECT=100 {{(pid=80244) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:17:53.459288 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 475/951] 10.222.0.47 () {58 vars in 1023 bytes} [Thu Jul 21 08:17:53 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 21 08:17:53.480974 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3572925b-0775-46cf-9065-f07643e88432 None None] Authenticating user token {{(pid=80245) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:17:53.573062 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG dbcounter [-] [80245] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80245) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:17:53.592863 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3572925b-0775-46cf-9065-f07643e88432 None None] Validating token access rules against request {{(pid=80245) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:17:53.629254 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93a76af7-357c-4d4e-8973-2abe85f469fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80245) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:17:53.630020 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93a76af7-357c-4d4e-8973-2abe85f469fe None admin] REQUEST_METHOD: `GET` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:17:53.630312 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93a76af7-357c-4d4e-8973-2abe85f469fe None admin] SCRIPT_NAME: `/identity` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:17:53.630587 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93a76af7-357c-4d4e-8973-2abe85f469fe None admin] PATH_INFO: `/v3/services/compute` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:17:53.631235 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93a76af7-357c-4d4e-8973-2abe85f469fe None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:17:53.632617 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93a76af7-357c-4d4e-8973-2abe85f469fe None admin] RBAC: Authorization granted {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:17:53.645225 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application [None req-93a76af7-357c-4d4e-8973-2abe85f469fe None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Jul 21 08:17:53.645225 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 08:17:53.645225 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 21 08:17:53.645225 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 21 08:17:53.645225 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 21 08:17:53.645225 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 21 08:17:53.645225 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 21 08:17:53.645225 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 21 08:17:53.645225 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Jul 21 08:17:53.645225 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application Jul 21 08:17:53.645225 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 21 08:17:53.645225 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application Jul 21 08:17:53.645225 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 08:17:53.645225 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 08:17:53.645225 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 08:17:53.645225 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 08:17:53.645225 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 08:17:53.646236 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 08:17:53.646236 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 08:17:53.646236 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 08:17:53.646236 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 08:17:53.646236 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 08:17:53.646236 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 08:17:53.646236 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 21 08:17:53.646236 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 21 08:17:53.646236 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 21 08:17:53.646236 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 21 08:17:53.646236 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 08:17:53.646236 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 08:17:53.646236 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 08:17:53.646236 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 08:17:53.646236 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 08:17:53.646236 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 08:17:53.647215 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 08:17:53.647215 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application with Lock( Jul 21 08:17:53.647215 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 08:17:53.647215 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application return self._enter() Jul 21 08:17:53.647215 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 08:17:53.647215 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 08:17:53.647215 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 08:17:53.647215 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application return self.creator() Jul 21 08:17:53.647215 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 08:17:53.647215 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application created_value = creator( Jul 21 08:17:53.647215 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 21 08:17:53.647215 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 21 08:17:53.647215 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Jul 21 08:17:53.647215 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application Jul 21 08:17:53.647215 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: [pid: 80245|app: 0|req: 477/952] 10.222.0.47 () {56 vars in 1111 bytes} [Thu Jul 21 08:17:53 2022] GET /identity/v3/services/compute => generated 88 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 08:17:53.652501 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32ebce05-299f-4007-8ddf-6d573e480fa5 None None] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:17:53.760673 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32ebce05-299f-4007-8ddf-6d573e480fa5 None None] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:17:53.795431 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-752d34d9-a308-4d6c-bf59-76352f81e0d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:17:53.796399 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-752d34d9-a308-4d6c-bf59-76352f81e0d5 None admin] REQUEST_METHOD: `GET` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:17:53.796898 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-752d34d9-a308-4d6c-bf59-76352f81e0d5 None admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:17:53.797365 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-752d34d9-a308-4d6c-bf59-76352f81e0d5 None admin] PATH_INFO: `/v3/services` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:17:53.798253 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-752d34d9-a308-4d6c-bf59-76352f81e0d5 None admin] RBAC: Adding query filter params (name=compute) {{(pid=80244) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 08:17:53.798740 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-752d34d9-a308-4d6c-bf59-76352f81e0d5 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:17:53.800848 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-752d34d9-a308-4d6c-bf59-76352f81e0d5 None admin] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:17:53.806406 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 476/953] 10.222.0.47 () {56 vars in 1112 bytes} [Thu Jul 21 08:17:53 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 08:17:53.811719 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93a76af7-357c-4d4e-8973-2abe85f469fe None admin] Authenticating user token {{(pid=80245) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:17:53.917705 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93a76af7-357c-4d4e-8973-2abe85f469fe None admin] Validating token access rules against request {{(pid=80245) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:17:53.952586 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b0f111f-25eb-4f49-bdc9-de5476480746 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80245) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:17:53.953252 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b0f111f-25eb-4f49-bdc9-de5476480746 None admin] REQUEST_METHOD: `GET` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:17:53.953529 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b0f111f-25eb-4f49-bdc9-de5476480746 None admin] SCRIPT_NAME: `/identity` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:17:53.953800 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b0f111f-25eb-4f49-bdc9-de5476480746 None admin] PATH_INFO: `/v3/services` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:17:53.954503 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b0f111f-25eb-4f49-bdc9-de5476480746 None admin] RBAC: Adding query filter params (type=compute) {{(pid=80245) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 08:17:53.954805 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b0f111f-25eb-4f49-bdc9-de5476480746 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:17:53.956291 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b0f111f-25eb-4f49-bdc9-de5476480746 None admin] RBAC: Authorization granted {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:17:53.961493 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: [pid: 80245|app: 0|req: 478/954] 10.222.0.47 () {56 vars in 1112 bytes} [Thu Jul 21 08:17:53 2022] GET /identity/v3/services?type=compute => generated 124 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 08:17:54.974528 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de33a3f0-1a30-40fe-895a-cec0f92e2fc6 None None] REQUEST_METHOD: `GET` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:17:54.975051 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de33a3f0-1a30-40fe-895a-cec0f92e2fc6 None None] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:17:54.975392 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de33a3f0-1a30-40fe-895a-cec0f92e2fc6 None None] PATH_INFO: `/` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:17:54.977109 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 477/955] 10.222.0.47 () {54 vars in 926 bytes} [Thu Jul 21 08:17:54 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 08:17:54.982318 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8a7c63a-b835-4ebe-95fe-80f35d07e2ca None None] REQUEST_METHOD: `POST` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:17:54.982729 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8a7c63a-b835-4ebe-95fe-80f35d07e2ca None None] SCRIPT_NAME: `/identity` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:17:54.983054 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8a7c63a-b835-4ebe-95fe-80f35d07e2ca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:17:55.001513 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.auth.core [None req-c8a7c63a-b835-4ebe-95fe-80f35d07e2ca None None] MFA Rules not processed for user `2dce40b8d74c41ec94cc36e6ff05d412`. Rule list: `[]` (Enabled: `True`). {{(pid=80245) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 08:17:55.003589 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.fernet_utils [None req-c8a7c63a-b835-4ebe-95fe-80f35d07e2ca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80245) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 08:17:55.011861 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG dbcounter [-] [80245] Writing DB stats keystone:SELECT=100 {{(pid=80245) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:17:55.047600 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00720cbd-6022-49ee-bb68-8ae6ee5f62cc None None] REQUEST_METHOD: `GET` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:17:55.047946 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00720cbd-6022-49ee-bb68-8ae6ee5f62cc None None] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:17:55.048167 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00720cbd-6022-49ee-bb68-8ae6ee5f62cc None None] PATH_INFO: `/` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:17:55.050838 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 478/956] 10.222.0.47 () {54 vars in 926 bytes} [Thu Jul 21 08:17:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 08:17:55.054993 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd795c36-d9a7-4cc5-b11e-5b4b59e24313 None None] REQUEST_METHOD: `POST` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:17:55.055286 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd795c36-d9a7-4cc5-b11e-5b4b59e24313 None None] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:17:55.055629 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd795c36-d9a7-4cc5-b11e-5b4b59e24313 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:17:55.073232 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.auth.core [None req-bd795c36-d9a7-4cc5-b11e-5b4b59e24313 None None] MFA Rules not processed for user `2dce40b8d74c41ec94cc36e6ff05d412`. Rule list: `[]` (Enabled: `True`). {{(pid=80244) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 08:17:55.074425 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.fernet_utils [None req-bd795c36-d9a7-4cc5-b11e-5b4b59e24313 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80244) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 08:17:55.076652 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: [pid: 80245|app: 0|req: 479/957] 10.222.0.47 () {58 vars in 1023 bytes} [Thu Jul 21 08:17:54 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 21 08:17:55.103103 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8a7c63a-b835-4ebe-95fe-80f35d07e2ca None None] Authenticating user token {{(pid=80245) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:17:55.124503 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG dbcounter [-] [80244] Writing DB stats keystone:SELECT=100 {{(pid=80244) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:17:55.157352 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 479/958] 10.222.0.47 () {58 vars in 1023 bytes} [Thu Jul 21 08:17:55 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 21 08:17:55.160699 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7992e830-192f-467f-8c58-c8a1ed5d3fd6 None None] REQUEST_METHOD: `GET` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:17:55.161024 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7992e830-192f-467f-8c58-c8a1ed5d3fd6 None None] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:17:55.161253 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7992e830-192f-467f-8c58-c8a1ed5d3fd6 None None] PATH_INFO: `/` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:17:55.166684 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 480/959] 10.222.0.47 () {54 vars in 926 bytes} [Thu Jul 21 08:17:55 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 08:17:55.168173 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eb7fca1-42a3-4c37-a938-061ee2df03d6 None None] REQUEST_METHOD: `POST` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:17:55.168459 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eb7fca1-42a3-4c37-a938-061ee2df03d6 None None] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:17:55.168733 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eb7fca1-42a3-4c37-a938-061ee2df03d6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:17:55.191720 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.auth.core [None req-6eb7fca1-42a3-4c37-a938-061ee2df03d6 None None] MFA Rules not processed for user `2dce40b8d74c41ec94cc36e6ff05d412`. Rule list: `[]` (Enabled: `True`). {{(pid=80244) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 08:17:55.192913 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.fernet_utils [None req-6eb7fca1-42a3-4c37-a938-061ee2df03d6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80244) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 08:17:55.223528 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8a7c63a-b835-4ebe-95fe-80f35d07e2ca None None] Validating token access rules against request {{(pid=80245) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:17:55.262548 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d22b398-86ab-4a36-bdee-18f14943103e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80245) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:17:55.263142 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d22b398-86ab-4a36-bdee-18f14943103e None admin] REQUEST_METHOD: `POST` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:17:55.263456 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d22b398-86ab-4a36-bdee-18f14943103e None admin] SCRIPT_NAME: `/identity` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:17:55.263736 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d22b398-86ab-4a36-bdee-18f14943103e None admin] PATH_INFO: `/v3/users` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:17:55.275646 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 481/960] 10.222.0.47 () {58 vars in 1023 bytes} [Thu Jul 21 08:17:55 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 21 08:17:55.275805 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d22b398-86ab-4a36-bdee-18f14943103e None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:17:55.275901 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eb7fca1-42a3-4c37-a938-061ee2df03d6 None None] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:17:55.276001 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d22b398-86ab-4a36-bdee-18f14943103e None admin] RBAC: Authorization granted {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:17:55.288420 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.sql.core [None req-6d22b398-86ab-4a36-bdee-18f14943103e None admin] Conflict user: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Jul 21 08:17:55.288420 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: [SQL: INSERT INTO local_user (user_id, domain_id, name, failed_auth_count, failed_auth_at) VALUES (%(user_id)s, %(domain_id)s, %(name)s, %(failed_auth_count)s, %(failed_auth_at)s)] Jul 21 08:17:55.288420 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: [parameters: {'user_id': '165a6d8227a445c78ffac50dab6b5433', 'domain_id': 'default', 'name': 'cinder', 'failed_auth_count': 0, 'failed_auth_at': None}] Jul 21 08:17:55.288420 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=80245) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Jul 21 08:17:55.296563 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application [None req-6d22b398-86ab-4a36-bdee-18f14943103e None admin] Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default.: keystone.exception.Conflict: Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default. Jul 21 08:17:55.296563 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 08:17:55.296563 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Jul 21 08:17:55.296563 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application self.dialect.do_execute( Jul 21 08:17:55.296563 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Jul 21 08:17:55.296563 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Jul 21 08:17:55.296563 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Jul 21 08:17:55.296563 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application result = self._query(query) Jul 21 08:17:55.296563 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Jul 21 08:17:55.296563 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application conn.query(q) Jul 21 08:17:55.296563 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Jul 21 08:17:55.296563 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Jul 21 08:17:55.296563 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Jul 21 08:17:55.296563 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application result.read() Jul 21 08:17:55.296563 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Jul 21 08:17:55.297442 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Jul 21 08:17:55.297442 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Jul 21 08:17:55.297442 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application packet.raise_for_error() Jul 21 08:17:55.297442 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Jul 21 08:17:55.297442 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Jul 21 08:17:55.297442 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Jul 21 08:17:55.297442 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Jul 21 08:17:55.297442 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Jul 21 08:17:55.297442 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application Jul 21 08:17:55.297442 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Jul 21 08:17:55.297442 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application Jul 21 08:17:55.297442 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 08:17:55.297442 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Jul 21 08:17:55.297442 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application return method(*args, **kwargs) Jul 21 08:17:55.297442 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 141, in create_user Jul 21 08:17:55.297442 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application return base.filter_user(user_ref.to_dict()) Jul 21 08:17:55.297442 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Jul 21 08:17:55.297442 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application next(self.gen) Jul 21 08:17:55.298490 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Jul 21 08:17:55.298490 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application yield resource Jul 21 08:17:55.298490 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Jul 21 08:17:55.298490 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application next(self.gen) Jul 21 08:17:55.298490 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Jul 21 08:17:55.298490 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application self.session.rollback() Jul 21 08:17:55.298490 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Jul 21 08:17:55.298490 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application self.force_reraise() Jul 21 08:17:55.298490 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Jul 21 08:17:55.298490 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application raise self.value Jul 21 08:17:55.298490 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Jul 21 08:17:55.298490 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Jul 21 08:17:55.298490 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Jul 21 08:17:55.298490 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application session.commit() Jul 21 08:17:55.298490 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Jul 21 08:17:55.298490 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Jul 21 08:17:55.298490 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Jul 21 08:17:55.299427 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application self._prepare_impl() Jul 21 08:17:55.299427 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Jul 21 08:17:55.299427 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application self.session.flush() Jul 21 08:17:55.299427 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Jul 21 08:17:55.299427 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application self._flush(objects) Jul 21 08:17:55.299427 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Jul 21 08:17:55.299427 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Jul 21 08:17:55.299427 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Jul 21 08:17:55.299427 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application compat.raise_( Jul 21 08:17:55.299427 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Jul 21 08:17:55.299427 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application raise exception Jul 21 08:17:55.299427 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Jul 21 08:17:55.299427 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application flush_context.execute() Jul 21 08:17:55.299427 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Jul 21 08:17:55.299427 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application rec.execute(self) Jul 21 08:17:55.299427 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Jul 21 08:17:55.299427 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Jul 21 08:17:55.301633 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Jul 21 08:17:55.301633 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application _emit_insert_statements( Jul 21 08:17:55.301633 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1238, in _emit_insert_statements Jul 21 08:17:55.301633 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application result = connection._execute_20( Jul 21 08:17:55.301633 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1631, in _execute_20 Jul 21 08:17:55.301633 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Jul 21 08:17:55.301633 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 332, in _execute_on_connection Jul 21 08:17:55.301633 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Jul 21 08:17:55.301633 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1498, in _execute_clauseelement Jul 21 08:17:55.301633 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application ret = self._execute_context( Jul 21 08:17:55.301633 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1862, in _execute_context Jul 21 08:17:55.301633 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Jul 21 08:17:55.301633 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2041, in _handle_dbapi_exception Jul 21 08:17:55.301633 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Jul 21 08:17:55.301633 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Jul 21 08:17:55.301633 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application raise exception Jul 21 08:17:55.302506 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Jul 21 08:17:55.302506 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application self.dialect.do_execute( Jul 21 08:17:55.302506 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Jul 21 08:17:55.302506 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Jul 21 08:17:55.302506 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Jul 21 08:17:55.302506 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application result = self._query(query) Jul 21 08:17:55.302506 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Jul 21 08:17:55.302506 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application conn.query(q) Jul 21 08:17:55.302506 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Jul 21 08:17:55.302506 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Jul 21 08:17:55.302506 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Jul 21 08:17:55.302506 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application result.read() Jul 21 08:17:55.302506 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Jul 21 08:17:55.302506 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Jul 21 08:17:55.302506 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Jul 21 08:17:55.302506 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application packet.raise_for_error() Jul 21 08:17:55.303667 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Jul 21 08:17:55.303667 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Jul 21 08:17:55.303667 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Jul 21 08:17:55.303667 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Jul 21 08:17:55.303667 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Jul 21 08:17:55.303667 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application [SQL: INSERT INTO local_user (user_id, domain_id, name, failed_auth_count, failed_auth_at) VALUES (%(user_id)s, %(domain_id)s, %(name)s, %(failed_auth_count)s, %(failed_auth_at)s)] Jul 21 08:17:55.303667 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application [parameters: {'user_id': '165a6d8227a445c78ffac50dab6b5433', 'domain_id': 'default', 'name': 'cinder', 'failed_auth_count': 0, 'failed_auth_at': None}] Jul 21 08:17:55.303667 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Jul 21 08:17:55.303667 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application Jul 21 08:17:55.303667 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 21 08:17:55.303667 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application Jul 21 08:17:55.303667 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 08:17:55.303667 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 08:17:55.303667 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 08:17:55.303667 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 08:17:55.304976 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 08:17:55.304976 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 08:17:55.304976 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 08:17:55.304976 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 08:17:55.304976 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 08:17:55.304976 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 08:17:55.304976 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 08:17:55.304976 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 244, in post Jul 21 08:17:55.304976 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.create_user( Jul 21 08:17:55.304976 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 08:17:55.304976 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 08:17:55.304976 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 21 08:17:55.304976 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 21 08:17:55.304976 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 21 08:17:55.304976 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 21 08:17:55.304976 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1000, in create_user Jul 21 08:17:55.304976 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application ref = self._create_user_with_federated_objects(user, driver) Jul 21 08:17:55.310051 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 966, in _create_user_with_federated_objects Jul 21 08:17:55.310051 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application user = driver.create_user(user['id'], user) Jul 21 08:17:55.310051 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Jul 21 08:17:55.310051 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Jul 21 08:17:55.310051 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default. Jul 21 08:17:55.310051 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application Jul 21 08:17:55.310051 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: [pid: 80245|app: 0|req: 480/961] 10.222.0.47 () {60 vars in 1131 bytes} [Thu Jul 21 08:17:55 2022] POST /identity/v3/users => generated 160 bytes in 196 msecs (HTTP/1.1 409) 5 headers in 183 bytes (2 switches on core 0) Jul 21 08:17:55.310051 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d22b398-86ab-4a36-bdee-18f14943103e None admin] Authenticating user token {{(pid=80245) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:17:55.310051 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG dbcounter [-] [80245] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80245) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:17:55.382824 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eb7fca1-42a3-4c37-a938-061ee2df03d6 None None] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:17:55.392623 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG dbcounter [-] [80244] Writing DB stats keystone:SELECT=100 {{(pid=80244) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:17:55.405570 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d22b398-86ab-4a36-bdee-18f14943103e None admin] Validating token access rules against request {{(pid=80245) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:17:55.420783 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87bbff54-4406-4729-a199-bd8143833ff5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:17:55.421603 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87bbff54-4406-4729-a199-bd8143833ff5 None admin] REQUEST_METHOD: `GET` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:17:55.421883 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87bbff54-4406-4729-a199-bd8143833ff5 None admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:17:55.422271 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87bbff54-4406-4729-a199-bd8143833ff5 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:17:55.433818 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87bbff54-4406-4729-a199-bd8143833ff5 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 21 08:17:55.441171 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f91c0c7d-870c-45b5-858e-b21a7014995a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80245) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:17:55.442036 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f91c0c7d-870c-45b5-858e-b21a7014995a None admin] REQUEST_METHOD: `GET` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:17:55.442317 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f91c0c7d-870c-45b5-858e-b21a7014995a None admin] SCRIPT_NAME: `/identity` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:17:55.442504 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application [None req-87bbff54-4406-4729-a199-bd8143833ff5 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 21 08:17:55.442504 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 08:17:55.442504 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 08:17:55.442504 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 08:17:55.442504 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 08:17:55.442504 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 08:17:55.442504 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 08:17:55.442504 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 08:17:55.442504 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 08:17:55.442504 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 08:17:55.442504 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 08:17:55.442504 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 08:17:55.442504 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 21 08:17:55.442504 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 21 08:17:55.442504 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 21 08:17:55.442504 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 21 08:17:55.443636 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 21 08:17:55.443636 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 21 08:17:55.443636 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 21 08:17:55.443636 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 21 08:17:55.443636 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 08:17:55.443636 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 08:17:55.443636 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 08:17:55.443636 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 08:17:55.443636 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 08:17:55.443636 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 08:17:55.443636 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 08:17:55.443636 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application with Lock( Jul 21 08:17:55.443636 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 08:17:55.443636 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self._enter() Jul 21 08:17:55.443636 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 08:17:55.443636 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 08:17:55.449486 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 08:17:55.449486 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self.creator() Jul 21 08:17:55.449486 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 08:17:55.449486 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application created_value = creator( Jul 21 08:17:55.449486 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 21 08:17:55.449486 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 21 08:17:55.449486 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 21 08:17:55.449486 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 21 08:17:55.449486 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 21 08:17:55.449486 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 21 08:17:55.449486 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 21 08:17:55.449486 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application Jul 21 08:17:55.450146 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f91c0c7d-870c-45b5-858e-b21a7014995a None admin] PATH_INFO: `/v3/services/object-store` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:17:55.450146 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f91c0c7d-870c-45b5-858e-b21a7014995a None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:17:55.450281 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 482/962] 10.222.0.47 () {56 vars in 1120 bytes} [Thu Jul 21 08:17:55 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 08:17:55.450281 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87bbff54-4406-4729-a199-bd8143833ff5 None admin] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:17:55.450412 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f91c0c7d-870c-45b5-858e-b21a7014995a None admin] RBAC: Authorization granted {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:17:55.463172 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application [None req-f91c0c7d-870c-45b5-858e-b21a7014995a None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 21 08:17:55.463172 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 08:17:55.463172 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 21 08:17:55.463172 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 21 08:17:55.463172 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 21 08:17:55.463172 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 21 08:17:55.463172 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 21 08:17:55.463172 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 21 08:17:55.463172 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 21 08:17:55.463172 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application Jul 21 08:17:55.463172 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 21 08:17:55.463172 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application Jul 21 08:17:55.463172 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 08:17:55.463172 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 08:17:55.463172 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 08:17:55.463172 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 08:17:55.463172 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 08:17:55.464345 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 08:17:55.464345 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 08:17:55.464345 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 08:17:55.464345 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 08:17:55.464345 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 08:17:55.464345 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 08:17:55.464345 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 21 08:17:55.464345 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 21 08:17:55.464345 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 21 08:17:55.464345 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 21 08:17:55.464345 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 08:17:55.464345 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 08:17:55.464345 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 08:17:55.464345 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 08:17:55.464345 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 08:17:55.464345 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 08:17:55.465935 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 08:17:55.465935 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application with Lock( Jul 21 08:17:55.465935 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 08:17:55.465935 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application return self._enter() Jul 21 08:17:55.465935 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 08:17:55.465935 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 08:17:55.465935 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 08:17:55.465935 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application return self.creator() Jul 21 08:17:55.465935 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 08:17:55.465935 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application created_value = creator( Jul 21 08:17:55.465935 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 21 08:17:55.465935 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 21 08:17:55.465935 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 21 08:17:55.465935 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application Jul 21 08:17:55.465935 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: [pid: 80245|app: 0|req: 481/963] 10.222.0.47 () {56 vars in 1126 bytes} [Thu Jul 21 08:17:55 2022] GET /identity/v3/services/object-store => generated 93 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 08:17:55.468008 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f91c0c7d-870c-45b5-858e-b21a7014995a None admin] Authenticating user token {{(pid=80245) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:17:55.544534 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG dbcounter [-] [80245] Writing DB stats keystone:SELECT=100 {{(pid=80245) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:17:55.569655 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87bbff54-4406-4729-a199-bd8143833ff5 None admin] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:17:55.580301 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f91c0c7d-870c-45b5-858e-b21a7014995a None admin] Validating token access rules against request {{(pid=80245) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:17:55.609896 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b98fba1a-d5e2-4e6d-a5ed-7188133e5d45 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:17:55.610697 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b98fba1a-d5e2-4e6d-a5ed-7188133e5d45 None admin] REQUEST_METHOD: `GET` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:17:55.610979 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b98fba1a-d5e2-4e6d-a5ed-7188133e5d45 None admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:17:55.611457 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b98fba1a-d5e2-4e6d-a5ed-7188133e5d45 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:17:55.616739 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-247c66a6-eb87-4bfc-ac91-7d7dac12bebd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80245) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:17:55.617416 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-247c66a6-eb87-4bfc-ac91-7d7dac12bebd None admin] REQUEST_METHOD: `GET` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:17:55.617707 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-247c66a6-eb87-4bfc-ac91-7d7dac12bebd None admin] SCRIPT_NAME: `/identity` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:17:55.617982 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-247c66a6-eb87-4bfc-ac91-7d7dac12bebd None admin] PATH_INFO: `/v3/roles` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:17:55.619281 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-247c66a6-eb87-4bfc-ac91-7d7dac12bebd None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80245) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 08:17:55.619281 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-247c66a6-eb87-4bfc-ac91-7d7dac12bebd None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:17:55.620425 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-247c66a6-eb87-4bfc-ac91-7d7dac12bebd None admin] RBAC: Authorization granted {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:17:55.620643 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b98fba1a-d5e2-4e6d-a5ed-7188133e5d45 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:17:55.622080 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b98fba1a-d5e2-4e6d-a5ed-7188133e5d45 None admin] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:17:55.632768 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: [pid: 80245|app: 0|req: 482/964] 10.222.0.47 () {56 vars in 1115 bytes} [Thu Jul 21 08:17:55 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 08:17:55.634814 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-247c66a6-eb87-4bfc-ac91-7d7dac12bebd None admin] Authenticating user token {{(pid=80245) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:17:55.637561 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application [None req-b98fba1a-d5e2-4e6d-a5ed-7188133e5d45 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Jul 21 08:17:55.637561 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 08:17:55.637561 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 08:17:55.637561 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 08:17:55.637561 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 08:17:55.637561 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 08:17:55.637561 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 08:17:55.637561 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 08:17:55.637561 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 08:17:55.637561 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 08:17:55.637561 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 08:17:55.637561 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 08:17:55.637561 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 21 08:17:55.637561 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 21 08:17:55.637561 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 21 08:17:55.637561 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 21 08:17:55.638477 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 08:17:55.638477 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 08:17:55.638477 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 21 08:17:55.638477 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 21 08:17:55.638477 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 21 08:17:55.638477 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 21 08:17:55.638477 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 08:17:55.638477 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 08:17:55.638477 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 08:17:55.638477 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 08:17:55.638477 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 08:17:55.638477 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application with Lock( Jul 21 08:17:55.638477 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 08:17:55.638477 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self._enter() Jul 21 08:17:55.638477 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 08:17:55.638477 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 08:17:55.638477 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 08:17:55.638477 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self.creator() Jul 21 08:17:55.639865 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 08:17:55.639865 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application created_value = creator( Jul 21 08:17:55.639865 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 21 08:17:55.639865 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 21 08:17:55.639865 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 21 08:17:55.639865 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 21 08:17:55.639865 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 21 08:17:55.639865 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 21 08:17:55.639865 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Jul 21 08:17:55.639865 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application Jul 21 08:17:55.639865 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 483/965] 10.222.0.47 () {56 vars in 1099 bytes} [Thu Jul 21 08:17:55 2022] GET /identity/v3/users/cinder => generated 84 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 08:17:55.639865 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b98fba1a-d5e2-4e6d-a5ed-7188133e5d45 None admin] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:17:55.670743 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG dbcounter [-] [80244] Writing DB stats keystone:SELECT=100 {{(pid=80244) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:17:55.748963 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-247c66a6-eb87-4bfc-ac91-7d7dac12bebd None admin] Validating token access rules against request {{(pid=80245) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:17:55.753067 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b98fba1a-d5e2-4e6d-a5ed-7188133e5d45 None admin] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:17:55.778179 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG dbcounter [-] [80245] Writing DB stats keystone:SELECT=100 {{(pid=80245) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:17:55.786804 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb6af089-543f-4afb-87a3-234defa41fbd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80245) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:17:55.787508 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb6af089-543f-4afb-87a3-234defa41fbd None admin] REQUEST_METHOD: `GET` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:17:55.787759 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb6af089-543f-4afb-87a3-234defa41fbd None admin] SCRIPT_NAME: `/identity` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:17:55.788061 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb6af089-543f-4afb-87a3-234defa41fbd None admin] PATH_INFO: `/v3/services` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:17:55.788769 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb6af089-543f-4afb-87a3-234defa41fbd None admin] RBAC: Adding query filter params (name=object-store) {{(pid=80245) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 08:17:55.789074 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb6af089-543f-4afb-87a3-234defa41fbd None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:17:55.790433 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb6af089-543f-4afb-87a3-234defa41fbd None admin] RBAC: Authorization granted {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:17:55.790544 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b22ba5a4-23e8-47ad-a676-1ce202703971 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:17:55.791349 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b22ba5a4-23e8-47ad-a676-1ce202703971 None admin] REQUEST_METHOD: `GET` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:17:55.791813 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b22ba5a4-23e8-47ad-a676-1ce202703971 None admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:17:55.792096 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b22ba5a4-23e8-47ad-a676-1ce202703971 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:17:55.795489 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: [pid: 80245|app: 0|req: 483/966] 10.222.0.47 () {56 vars in 1122 bytes} [Thu Jul 21 08:17:55 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 08:17:55.796460 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb6af089-543f-4afb-87a3-234defa41fbd None admin] Authenticating user token {{(pid=80245) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:17:55.809452 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b22ba5a4-23e8-47ad-a676-1ce202703971 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:17:55.810966 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b22ba5a4-23e8-47ad-a676-1ce202703971 None admin] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:17:55.828187 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application [None req-b22ba5a4-23e8-47ad-a676-1ce202703971 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 08:17:55.828187 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 08:17:55.828187 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 08:17:55.828187 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 08:17:55.828187 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 08:17:55.828187 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 08:17:55.828187 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 08:17:55.828187 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 08:17:55.828187 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 08:17:55.828187 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 08:17:55.828187 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 08:17:55.828187 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 08:17:55.828187 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 21 08:17:55.828187 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 21 08:17:55.828187 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 21 08:17:55.829252 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 21 08:17:55.829252 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 08:17:55.829252 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 08:17:55.829252 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 08:17:55.829252 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 08:17:55.829252 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 08:17:55.829252 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 08:17:55.829252 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 08:17:55.829252 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application with Lock( Jul 21 08:17:55.829252 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 08:17:55.829252 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self._enter() Jul 21 08:17:55.829252 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 08:17:55.829252 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 08:17:55.829252 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 08:17:55.829252 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self.creator() Jul 21 08:17:55.829252 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 08:17:55.829252 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application created_value = creator( Jul 21 08:17:55.830161 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 21 08:17:55.830161 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 21 08:17:55.830161 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 08:17:55.830161 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application Jul 21 08:17:55.830161 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 484/967] 10.222.0.47 () {56 vars in 1108 bytes} [Thu Jul 21 08:17:55 2022] GET /identity/v3/domains/Default => generated 87 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 08:17:55.830539 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b22ba5a4-23e8-47ad-a676-1ce202703971 None admin] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:17:55.911948 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb6af089-543f-4afb-87a3-234defa41fbd None admin] Validating token access rules against request {{(pid=80245) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:17:55.916605 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG dbcounter [-] [80244] Writing DB stats keystone:SELECT=100 {{(pid=80244) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:17:55.939955 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b22ba5a4-23e8-47ad-a676-1ce202703971 None admin] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:17:55.951923 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59b9c069-ec63-4355-8a9c-12ef33f52893 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80245) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:17:55.952642 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59b9c069-ec63-4355-8a9c-12ef33f52893 None admin] REQUEST_METHOD: `GET` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:17:55.952929 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59b9c069-ec63-4355-8a9c-12ef33f52893 None admin] SCRIPT_NAME: `/identity` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:17:55.953208 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59b9c069-ec63-4355-8a9c-12ef33f52893 None admin] PATH_INFO: `/v3/users` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:17:55.954016 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59b9c069-ec63-4355-8a9c-12ef33f52893 None admin] RBAC: Adding query filter params (name=cinder) {{(pid=80245) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 08:17:55.954326 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59b9c069-ec63-4355-8a9c-12ef33f52893 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:17:55.958035 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59b9c069-ec63-4355-8a9c-12ef33f52893 None admin] RBAC: Authorization granted {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:17:55.979841 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b5870a3-6e1e-4bae-8d75-880c5796bd31 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:17:55.980530 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b5870a3-6e1e-4bae-8d75-880c5796bd31 None admin] REQUEST_METHOD: `GET` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:17:55.980786 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b5870a3-6e1e-4bae-8d75-880c5796bd31 None admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:17:55.981060 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b5870a3-6e1e-4bae-8d75-880c5796bd31 None admin] PATH_INFO: `/v3/services` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:17:55.981781 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b5870a3-6e1e-4bae-8d75-880c5796bd31 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=80244) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 08:17:55.982096 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b5870a3-6e1e-4bae-8d75-880c5796bd31 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:17:55.983541 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b5870a3-6e1e-4bae-8d75-880c5796bd31 None admin] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:17:55.988705 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 485/968] 10.222.0.47 () {56 vars in 1122 bytes} [Thu Jul 21 08:17:55 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 08:17:55.993526 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b5870a3-6e1e-4bae-8d75-880c5796bd31 None admin] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:17:55.998619 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: [pid: 80245|app: 0|req: 484/969] 10.222.0.47 () {56 vars in 1101 bytes} [Thu Jul 21 08:17:55 2022] GET /identity/v3/users?name=cinder => generated 354 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 08:17:56.000064 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59b9c069-ec63-4355-8a9c-12ef33f52893 None admin] Authenticating user token {{(pid=80245) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:17:56.075801 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG dbcounter [-] [80245] Writing DB stats keystone:SELECT=100 {{(pid=80245) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:17:56.110829 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b5870a3-6e1e-4bae-8d75-880c5796bd31 None admin] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:17:56.133688 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59b9c069-ec63-4355-8a9c-12ef33f52893 None admin] Validating token access rules against request {{(pid=80245) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:17:56.151038 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-062f421e-f6e4-4e01-b883-2d3615bdea0e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:17:56.151751 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-062f421e-f6e4-4e01-b883-2d3615bdea0e None admin] REQUEST_METHOD: `GET` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:17:56.152030 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-062f421e-f6e4-4e01-b883-2d3615bdea0e None admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:17:56.152307 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-062f421e-f6e4-4e01-b883-2d3615bdea0e None admin] PATH_INFO: `/v3/domains` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:17:56.153038 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-062f421e-f6e4-4e01-b883-2d3615bdea0e None admin] RBAC: Adding query filter params (name=Default) {{(pid=80244) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 08:17:56.153345 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-062f421e-f6e4-4e01-b883-2d3615bdea0e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:17:56.154730 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-062f421e-f6e4-4e01-b883-2d3615bdea0e None admin] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:17:56.171364 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 486/970] 10.222.0.47 () {56 vars in 1109 bytes} [Thu Jul 21 08:17:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 08:17:56.171364 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-370e0ec9-1c15-44d2-a50e-293ea05e53df None None] REQUEST_METHOD: `GET` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:17:56.171364 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-370e0ec9-1c15-44d2-a50e-293ea05e53df None None] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:17:56.171364 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-370e0ec9-1c15-44d2-a50e-293ea05e53df None None] PATH_INFO: `/` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:17:56.172262 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 487/971] 10.222.0.47 () {54 vars in 926 bytes} [Thu Jul 21 08:17:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 08:17:56.175675 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d444219-85ea-4fc8-a253-442ef073443f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80245) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:17:56.176318 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-370e0ec9-1c15-44d2-a50e-293ea05e53df None None] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:17:56.176602 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d444219-85ea-4fc8-a253-442ef073443f None admin] REQUEST_METHOD: `POST` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:17:56.176962 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d444219-85ea-4fc8-a253-442ef073443f None admin] SCRIPT_NAME: `/identity` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:17:56.177833 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d444219-85ea-4fc8-a253-442ef073443f None admin] PATH_INFO: `/v3/endpoints` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:17:56.178119 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d444219-85ea-4fc8-a253-442ef073443f None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:17:56.179592 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d444219-85ea-4fc8-a253-442ef073443f None admin] RBAC: Authorization granted {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:17:56.180679 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG dbcounter [-] [80244] Writing DB stats keystone:SELECT=100 {{(pid=80244) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:17:56.202297 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: [pid: 80245|app: 0|req: 485/972] 10.222.0.47 () {60 vars in 1144 bytes} [Thu Jul 21 08:17:55 2022] POST /identity/v3/endpoints => generated 349 bytes in 202 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 08:17:56.204141 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a40ee94-7582-4598-a153-fc0f84dbe25b None None] REQUEST_METHOD: `POST` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:17:56.204434 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a40ee94-7582-4598-a153-fc0f84dbe25b None None] SCRIPT_NAME: `/identity` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:17:56.204715 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a40ee94-7582-4598-a153-fc0f84dbe25b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:17:56.226375 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.auth.core [None req-4a40ee94-7582-4598-a153-fc0f84dbe25b None None] MFA Rules not processed for user `2dce40b8d74c41ec94cc36e6ff05d412`. Rule list: `[]` (Enabled: `True`). {{(pid=80245) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 08:17:56.228038 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.fernet_utils [None req-4a40ee94-7582-4598-a153-fc0f84dbe25b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80245) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 08:17:56.325475 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-370e0ec9-1c15-44d2-a50e-293ea05e53df None None] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:17:56.335635 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: [pid: 80245|app: 0|req: 486/973] 10.222.0.47 () {58 vars in 1023 bytes} [Thu Jul 21 08:17:56 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 21 08:17:56.354157 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a40ee94-7582-4598-a153-fc0f84dbe25b None None] Authenticating user token {{(pid=80245) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:17:56.374980 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01e7ec54-dbc4-434e-9790-1b366d123859 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:17:56.375904 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01e7ec54-dbc4-434e-9790-1b366d123859 None admin] REQUEST_METHOD: `GET` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:17:56.376193 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01e7ec54-dbc4-434e-9790-1b366d123859 None admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:17:56.376475 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01e7ec54-dbc4-434e-9790-1b366d123859 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:17:56.385741 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01e7ec54-dbc4-434e-9790-1b366d123859 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:17:56.387185 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01e7ec54-dbc4-434e-9790-1b366d123859 None admin] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:17:56.403806 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG dbcounter [-] [80245] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80245) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:17:56.404693 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application [None req-01e7ec54-dbc4-434e-9790-1b366d123859 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 21 08:17:56.404693 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 08:17:56.404693 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 08:17:56.404693 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 08:17:56.404693 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 08:17:56.404693 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 08:17:56.404693 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 08:17:56.404693 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 08:17:56.404693 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 08:17:56.404693 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 08:17:56.404693 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 08:17:56.404693 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 08:17:56.404693 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 21 08:17:56.404693 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 21 08:17:56.404693 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 21 08:17:56.404693 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 21 08:17:56.405691 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 08:17:56.405691 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 08:17:56.405691 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 21 08:17:56.405691 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 21 08:17:56.405691 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 21 08:17:56.405691 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 21 08:17:56.405691 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 08:17:56.405691 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 08:17:56.405691 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 08:17:56.405691 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 08:17:56.405691 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 08:17:56.405691 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application with Lock( Jul 21 08:17:56.405691 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 08:17:56.405691 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self._enter() Jul 21 08:17:56.405691 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 08:17:56.405691 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 08:17:56.405691 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 08:17:56.405691 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self.creator() Jul 21 08:17:56.406770 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 08:17:56.406770 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application created_value = creator( Jul 21 08:17:56.406770 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 21 08:17:56.406770 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 21 08:17:56.406770 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 21 08:17:56.406770 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 21 08:17:56.406770 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 21 08:17:56.406770 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 21 08:17:56.406770 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 21 08:17:56.406770 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application Jul 21 08:17:56.411669 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 488/974] 10.222.0.47 () {56 vars in 1117 bytes} [Thu Jul 21 08:17:56 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 231 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 08:17:56.413400 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01e7ec54-dbc4-434e-9790-1b366d123859 None admin] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:17:56.473268 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a40ee94-7582-4598-a153-fc0f84dbe25b None None] Validating token access rules against request {{(pid=80245) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:17:56.484286 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG dbcounter [-] [80244] Writing DB stats keystone:SELECT=100 {{(pid=80244) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:17:56.514475 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fc6f815-7db2-4ba6-a2f2-472e2d6a619b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80245) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:17:56.515266 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fc6f815-7db2-4ba6-a2f2-472e2d6a619b None admin] REQUEST_METHOD: `POST` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:17:56.515759 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fc6f815-7db2-4ba6-a2f2-472e2d6a619b None admin] SCRIPT_NAME: `/identity` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:17:56.516106 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fc6f815-7db2-4ba6-a2f2-472e2d6a619b None admin] PATH_INFO: `/v3/services` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:17:56.516997 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fc6f815-7db2-4ba6-a2f2-472e2d6a619b None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:17:56.518372 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fc6f815-7db2-4ba6-a2f2-472e2d6a619b None admin] RBAC: Authorization granted {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:17:56.532567 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01e7ec54-dbc4-434e-9790-1b366d123859 None admin] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:17:56.543359 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: [pid: 80245|app: 0|req: 487/975] 10.222.0.47 () {60 vars in 1141 bytes} [Thu Jul 21 08:17:56 2022] POST /identity/v3/services => generated 242 bytes in 179 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 21 08:17:56.590743 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf58b91a-08b0-4252-b36e-dd1c2afe1908 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:17:56.591430 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf58b91a-08b0-4252-b36e-dd1c2afe1908 None admin] REQUEST_METHOD: `GET` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:17:56.591749 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf58b91a-08b0-4252-b36e-dd1c2afe1908 None admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:17:56.592010 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf58b91a-08b0-4252-b36e-dd1c2afe1908 None admin] PATH_INFO: `/v3/users` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:17:56.592893 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf58b91a-08b0-4252-b36e-dd1c2afe1908 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80244) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 08:17:56.593204 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf58b91a-08b0-4252-b36e-dd1c2afe1908 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:17:56.594706 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf58b91a-08b0-4252-b36e-dd1c2afe1908 None admin] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:17:56.619493 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 489/976] 10.222.0.47 () {56 vars in 1149 bytes} [Thu Jul 21 08:17:56 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 21 08:17:56.622994 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fc6f815-7db2-4ba6-a2f2-472e2d6a619b None admin] Authenticating user token {{(pid=80245) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:17:56.729100 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG dbcounter [-] [80245] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80245) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:17:56.734740 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fc6f815-7db2-4ba6-a2f2-472e2d6a619b None admin] Validating token access rules against request {{(pid=80245) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:17:56.772067 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b25b2ceb-0d36-4ae2-8a83-87aa1c376f88 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80245) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:17:56.772732 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b25b2ceb-0d36-4ae2-8a83-87aa1c376f88 None admin] REQUEST_METHOD: `GET` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:17:56.773017 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b25b2ceb-0d36-4ae2-8a83-87aa1c376f88 None admin] SCRIPT_NAME: `/identity` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:17:56.773287 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b25b2ceb-0d36-4ae2-8a83-87aa1c376f88 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:17:56.840889 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b25b2ceb-0d36-4ae2-8a83-87aa1c376f88 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:17:56.842312 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b25b2ceb-0d36-4ae2-8a83-87aa1c376f88 None admin] RBAC: Authorization granted {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:17:56.939269 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG dbcounter [-] [80245] Writing DB stats keystone:SELECT=100 {{(pid=80245) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:17:56.949890 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: [pid: 80245|app: 0|req: 488/977] 10.222.0.47 () {58 vars in 1282 bytes} [Thu Jul 21 08:17:56 2022] GET /identity/v3/auth/tokens => generated 1560 bytes in 323 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 21 08:17:56.951544 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf58b91a-08b0-4252-b36e-dd1c2afe1908 None admin] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:17:57.057458 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf58b91a-08b0-4252-b36e-dd1c2afe1908 None admin] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:17:57.072386 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG dbcounter [-] [80244] Writing DB stats keystone:SELECT=100 {{(pid=80244) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:17:57.094645 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40e364bc-b321-4873-947b-1cc9149effa7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:17:57.095399 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40e364bc-b321-4873-947b-1cc9149effa7 None admin] REQUEST_METHOD: `GET` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:17:57.095687 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40e364bc-b321-4873-947b-1cc9149effa7 None admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:17:57.095961 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40e364bc-b321-4873-947b-1cc9149effa7 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:17:57.106219 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40e364bc-b321-4873-947b-1cc9149effa7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:17:57.107705 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40e364bc-b321-4873-947b-1cc9149effa7 None admin] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:17:57.125317 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application [None req-40e364bc-b321-4873-947b-1cc9149effa7 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 08:17:57.125317 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 08:17:57.125317 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 08:17:57.125317 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 08:17:57.125317 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 08:17:57.125317 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 08:17:57.125317 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 08:17:57.125317 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 08:17:57.125317 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 08:17:57.125317 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 08:17:57.125317 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 08:17:57.125317 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 08:17:57.125317 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 21 08:17:57.125317 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 21 08:17:57.125317 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 21 08:17:57.126279 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 21 08:17:57.126279 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 08:17:57.126279 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 08:17:57.126279 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 08:17:57.126279 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 08:17:57.126279 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 08:17:57.126279 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 08:17:57.126279 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 08:17:57.126279 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application with Lock( Jul 21 08:17:57.126279 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 08:17:57.126279 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self._enter() Jul 21 08:17:57.126279 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 08:17:57.126279 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 08:17:57.126279 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 08:17:57.126279 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self.creator() Jul 21 08:17:57.126279 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 08:17:57.126279 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application created_value = creator( Jul 21 08:17:57.132312 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 21 08:17:57.132312 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 21 08:17:57.132312 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 08:17:57.132312 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application Jul 21 08:17:57.132312 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 490/978] 10.222.0.47 () {56 vars in 1108 bytes} [Thu Jul 21 08:17:56 2022] GET /identity/v3/domains/Default => generated 87 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 21 08:17:57.134379 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b25b2ceb-0d36-4ae2-8a83-87aa1c376f88 None admin] Authenticating user token {{(pid=80245) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:17:57.234361 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b25b2ceb-0d36-4ae2-8a83-87aa1c376f88 None admin] Validating token access rules against request {{(pid=80245) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:17:57.270145 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fa58fb5-fee8-4c1c-b33d-e4d447c05ba7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80245) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:17:57.270830 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fa58fb5-fee8-4c1c-b33d-e4d447c05ba7 None admin] REQUEST_METHOD: `GET` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:17:57.271110 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fa58fb5-fee8-4c1c-b33d-e4d447c05ba7 None admin] SCRIPT_NAME: `/identity` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:17:57.271408 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fa58fb5-fee8-4c1c-b33d-e4d447c05ba7 None admin] PATH_INFO: `/v3/domains` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:17:57.272130 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fa58fb5-fee8-4c1c-b33d-e4d447c05ba7 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80245) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 08:17:57.272430 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fa58fb5-fee8-4c1c-b33d-e4d447c05ba7 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:17:57.273785 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fa58fb5-fee8-4c1c-b33d-e4d447c05ba7 None admin] RBAC: Authorization granted {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:17:57.285639 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: [pid: 80245|app: 0|req: 489/979] 10.222.0.47 () {56 vars in 1109 bytes} [Thu Jul 21 08:17:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 08:17:57.291244 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40e364bc-b321-4873-947b-1cc9149effa7 None admin] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:17:57.397075 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40e364bc-b321-4873-947b-1cc9149effa7 None admin] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:17:57.431123 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5083af9a-b37c-4743-9421-9356367b695a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:17:57.431946 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5083af9a-b37c-4743-9421-9356367b695a None admin] REQUEST_METHOD: `GET` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:17:57.432223 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5083af9a-b37c-4743-9421-9356367b695a None admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:17:57.432490 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5083af9a-b37c-4743-9421-9356367b695a None admin] PATH_INFO: `/v3/projects/service` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:17:57.439130 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5083af9a-b37c-4743-9421-9356367b695a None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:17:57.441236 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5083af9a-b37c-4743-9421-9356367b695a None admin] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:17:57.454676 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application [None req-5083af9a-b37c-4743-9421-9356367b695a None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 21 08:17:57.454676 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 08:17:57.454676 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 08:17:57.454676 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 08:17:57.454676 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 08:17:57.454676 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 08:17:57.454676 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 08:17:57.454676 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 08:17:57.454676 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 08:17:57.454676 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 08:17:57.454676 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 08:17:57.454676 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 08:17:57.454676 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 21 08:17:57.454676 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 21 08:17:57.454676 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 21 08:17:57.455911 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 21 08:17:57.455911 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 08:17:57.455911 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 08:17:57.455911 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 08:17:57.455911 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 08:17:57.455911 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 08:17:57.455911 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 08:17:57.455911 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 08:17:57.455911 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application with Lock( Jul 21 08:17:57.455911 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 08:17:57.455911 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self._enter() Jul 21 08:17:57.455911 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 08:17:57.455911 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 08:17:57.455911 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 08:17:57.455911 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self.creator() Jul 21 08:17:57.455911 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 08:17:57.455911 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application created_value = creator( Jul 21 08:17:57.456816 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 21 08:17:57.456816 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 21 08:17:57.456816 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 21 08:17:57.456816 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 21 08:17:57.456816 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 21 08:17:57.456816 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 21 08:17:57.456816 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 21 08:17:57.456816 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application Jul 21 08:17:57.459387 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 491/980] 10.222.0.47 () {56 vars in 1111 bytes} [Thu Jul 21 08:17:57 2022] GET /identity/v3/projects/service => generated 88 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 08:17:57.461938 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fa58fb5-fee8-4c1c-b33d-e4d447c05ba7 None admin] Authenticating user token {{(pid=80245) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:17:57.522561 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG dbcounter [-] [80245] Writing DB stats keystone:SELECT=100 {{(pid=80245) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:17:57.569799 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fa58fb5-fee8-4c1c-b33d-e4d447c05ba7 None admin] Validating token access rules against request {{(pid=80245) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:17:57.603207 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfd3b9c1-54d1-423a-bf03-a860702bcef9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80245) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:17:57.603925 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfd3b9c1-54d1-423a-bf03-a860702bcef9 None admin] REQUEST_METHOD: `GET` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:17:57.604206 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfd3b9c1-54d1-423a-bf03-a860702bcef9 None admin] SCRIPT_NAME: `/identity` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:17:57.604529 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfd3b9c1-54d1-423a-bf03-a860702bcef9 None admin] PATH_INFO: `/v3/projects` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:17:57.605303 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfd3b9c1-54d1-423a-bf03-a860702bcef9 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80245) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 08:17:57.605611 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfd3b9c1-54d1-423a-bf03-a860702bcef9 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:17:57.607086 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfd3b9c1-54d1-423a-bf03-a860702bcef9 None admin] RBAC: Authorization granted {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:17:57.619876 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: [pid: 80245|app: 0|req: 490/981] 10.222.0.47 () {56 vars in 1148 bytes} [Thu Jul 21 08:17:57 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 08:17:57.625693 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5083af9a-b37c-4743-9421-9356367b695a None admin] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:17:57.654825 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG dbcounter [-] [80244] Writing DB stats keystone:SELECT=100 {{(pid=80244) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:17:57.734693 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5083af9a-b37c-4743-9421-9356367b695a None admin] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:17:57.768859 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b2fb3c7-9e57-4702-b8a3-3cdabbd2c66b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:17:57.769575 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b2fb3c7-9e57-4702-b8a3-3cdabbd2c66b None admin] REQUEST_METHOD: `GET` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:17:57.769864 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b2fb3c7-9e57-4702-b8a3-3cdabbd2c66b None admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:17:57.770142 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b2fb3c7-9e57-4702-b8a3-3cdabbd2c66b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:17:57.770975 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b2fb3c7-9e57-4702-b8a3-3cdabbd2c66b None admin] RBAC: Adding query filter params (role.id=4b8dff36c32b49618b303451de1d1f69, scope.project.id=4faef13aa50e4e36b6c265fb1398fa14, user.id=2ef7f1e91e554a1f88eb6a63e60a475f) {{(pid=80244) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 08:17:57.771286 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b2fb3c7-9e57-4702-b8a3-3cdabbd2c66b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:17:57.772741 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b2fb3c7-9e57-4702-b8a3-3cdabbd2c66b None admin] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:17:57.787355 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 492/982] 10.222.0.47 () {56 vars in 1374 bytes} [Thu Jul 21 08:17:57 2022] GET /identity/v3/role_assignments?user.id=2ef7f1e91e554a1f88eb6a63e60a475f&scope.project.id=4faef13aa50e4e36b6c265fb1398fa14&role.id=4b8dff36c32b49618b303451de1d1f69 => generated 608 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 08:17:58.301610 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f50ac4ba-5924-4d0a-8421-374cc92b5e6f None None] REQUEST_METHOD: `GET` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:17:58.302224 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f50ac4ba-5924-4d0a-8421-374cc92b5e6f None None] SCRIPT_NAME: `/identity` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:17:58.302224 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f50ac4ba-5924-4d0a-8421-374cc92b5e6f None None] PATH_INFO: `/` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:17:58.303686 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: [pid: 80245|app: 0|req: 491/983] 10.222.0.47 () {54 vars in 926 bytes} [Thu Jul 21 08:17:58 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 08:17:58.308754 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7958f6a-baf6-4a9d-a56f-3a84f16bba95 None None] REQUEST_METHOD: `POST` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:17:58.309189 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7958f6a-baf6-4a9d-a56f-3a84f16bba95 None None] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:17:58.309370 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7958f6a-baf6-4a9d-a56f-3a84f16bba95 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:17:58.326840 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.auth.core [None req-f7958f6a-baf6-4a9d-a56f-3a84f16bba95 None None] MFA Rules not processed for user `2dce40b8d74c41ec94cc36e6ff05d412`. Rule list: `[]` (Enabled: `True`). {{(pid=80244) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 08:17:58.328033 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.fernet_utils [None req-f7958f6a-baf6-4a9d-a56f-3a84f16bba95 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80244) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 08:17:58.394779 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 493/984] 10.222.0.47 () {58 vars in 1023 bytes} [Thu Jul 21 08:17:58 2022] POST /identity/v3/auth/tokens => generated 1560 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 21 08:17:58.416679 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f50ac4ba-5924-4d0a-8421-374cc92b5e6f None None] Authenticating user token {{(pid=80245) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:17:58.535068 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f50ac4ba-5924-4d0a-8421-374cc92b5e6f None None] Validating token access rules against request {{(pid=80245) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:17:58.553402 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG dbcounter [-] [80245] Writing DB stats keystone:SELECT=100 {{(pid=80245) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:17:58.573306 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23f9cd70-36ee-4390-a330-fb9b940e8883 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80245) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:17:58.574125 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23f9cd70-36ee-4390-a330-fb9b940e8883 None admin] REQUEST_METHOD: `GET` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:17:58.574410 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23f9cd70-36ee-4390-a330-fb9b940e8883 None admin] SCRIPT_NAME: `/identity` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:17:58.574679 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23f9cd70-36ee-4390-a330-fb9b940e8883 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:17:58.575419 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23f9cd70-36ee-4390-a330-fb9b940e8883 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:17:58.576846 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23f9cd70-36ee-4390-a330-fb9b940e8883 None admin] RBAC: Authorization granted {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:17:58.589026 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application [None req-23f9cd70-36ee-4390-a330-fb9b940e8883 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 21 08:17:58.589026 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 08:17:58.589026 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 21 08:17:58.589026 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 21 08:17:58.589026 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 21 08:17:58.589026 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 21 08:17:58.589026 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 21 08:17:58.589026 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 21 08:17:58.589026 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 21 08:17:58.589026 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application Jul 21 08:17:58.589026 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 21 08:17:58.589026 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application Jul 21 08:17:58.589026 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 08:17:58.589026 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 08:17:58.589026 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 08:17:58.589026 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 08:17:58.589026 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 08:17:58.590030 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 08:17:58.590030 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 08:17:58.590030 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 08:17:58.590030 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 08:17:58.590030 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 08:17:58.590030 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 08:17:58.590030 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 21 08:17:58.590030 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 21 08:17:58.590030 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 21 08:17:58.590030 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 21 08:17:58.590030 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 08:17:58.590030 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 08:17:58.590030 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 08:17:58.590030 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 08:17:58.590030 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 08:17:58.590030 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 08:17:58.590890 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 08:17:58.590890 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application with Lock( Jul 21 08:17:58.590890 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 08:17:58.590890 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application return self._enter() Jul 21 08:17:58.590890 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 08:17:58.590890 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 08:17:58.590890 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 08:17:58.590890 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application return self.creator() Jul 21 08:17:58.590890 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 08:17:58.590890 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application created_value = creator( Jul 21 08:17:58.590890 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 21 08:17:58.590890 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 21 08:17:58.590890 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 21 08:17:58.590890 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application Jul 21 08:17:58.590890 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: [pid: 80245|app: 0|req: 492/985] 10.222.0.47 () {56 vars in 1126 bytes} [Thu Jul 21 08:17:58 2022] GET /identity/v3/services/object-store => generated 93 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 08:17:58.596261 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7958f6a-baf6-4a9d-a56f-3a84f16bba95 None None] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:17:58.619625 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG dbcounter [-] [80244] Writing DB stats keystone:SELECT=100 {{(pid=80244) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:17:58.623903 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf90dbd2-9fd7-4f0b-ac56-8257ca81e11c None None] REQUEST_METHOD: `GET` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:17:58.624809 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf90dbd2-9fd7-4f0b-ac56-8257ca81e11c None None] SCRIPT_NAME: `/identity` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:17:58.625532 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf90dbd2-9fd7-4f0b-ac56-8257ca81e11c None None] PATH_INFO: `/` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:17:58.627803 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: [pid: 80245|app: 0|req: 493/986] 10.222.0.47 () {54 vars in 926 bytes} [Thu Jul 21 08:17:58 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 08:17:58.632581 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bad3dd0a-5131-4a66-8ebc-79c3cf89adba None None] REQUEST_METHOD: `POST` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:17:58.633056 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bad3dd0a-5131-4a66-8ebc-79c3cf89adba None None] SCRIPT_NAME: `/identity` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:17:58.633908 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bad3dd0a-5131-4a66-8ebc-79c3cf89adba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:17:58.654400 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.auth.core [None req-bad3dd0a-5131-4a66-8ebc-79c3cf89adba None None] MFA Rules not processed for user `2dce40b8d74c41ec94cc36e6ff05d412`. Rule list: `[]` (Enabled: `True`). {{(pid=80245) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 08:17:58.656126 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.fernet_utils [None req-bad3dd0a-5131-4a66-8ebc-79c3cf89adba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80245) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 08:17:58.700961 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7958f6a-baf6-4a9d-a56f-3a84f16bba95 None None] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:17:58.734132 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83e8494d-0269-4aed-b8e4-0d38ac9c2b74 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:17:58.734799 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83e8494d-0269-4aed-b8e4-0d38ac9c2b74 None admin] REQUEST_METHOD: `GET` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:17:58.735082 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83e8494d-0269-4aed-b8e4-0d38ac9c2b74 None admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:17:58.735477 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83e8494d-0269-4aed-b8e4-0d38ac9c2b74 None admin] PATH_INFO: `/v3/services` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:17:58.736087 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83e8494d-0269-4aed-b8e4-0d38ac9c2b74 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=80244) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 08:17:58.736388 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83e8494d-0269-4aed-b8e4-0d38ac9c2b74 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:17:58.739381 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: [pid: 80245|app: 0|req: 494/987] 10.222.0.47 () {58 vars in 1023 bytes} [Thu Jul 21 08:17:58 2022] POST /identity/v3/auth/tokens => generated 1560 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 21 08:17:58.739573 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83e8494d-0269-4aed-b8e4-0d38ac9c2b74 None admin] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:17:58.744503 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 494/988] 10.222.0.47 () {56 vars in 1122 bytes} [Thu Jul 21 08:17:58 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 21 08:17:58.749929 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bad3dd0a-5131-4a66-8ebc-79c3cf89adba None None] Authenticating user token {{(pid=80245) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:17:58.764409 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83e8494d-0269-4aed-b8e4-0d38ac9c2b74 None admin] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:17:58.848960 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bad3dd0a-5131-4a66-8ebc-79c3cf89adba None None] Validating token access rules against request {{(pid=80245) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:17:58.854329 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG dbcounter [-] [80244] Writing DB stats keystone:SELECT=100 {{(pid=80244) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:17:58.863729 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG dbcounter [-] [80245] Writing DB stats keystone:SELECT=100 {{(pid=80245) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:17:58.873856 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83e8494d-0269-4aed-b8e4-0d38ac9c2b74 None admin] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:17:58.886555 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38f370e8-3033-4822-840e-1eaccb4e76ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80245) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:17:58.887242 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38f370e8-3033-4822-840e-1eaccb4e76ed None admin] REQUEST_METHOD: `GET` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:17:58.887558 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38f370e8-3033-4822-840e-1eaccb4e76ed None admin] SCRIPT_NAME: `/identity` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:17:58.887866 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38f370e8-3033-4822-840e-1eaccb4e76ed None admin] PATH_INFO: `/v3/services` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:17:58.888544 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38f370e8-3033-4822-840e-1eaccb4e76ed None admin] RBAC: Adding query filter params (type=object-store) {{(pid=80245) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 08:17:58.888855 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38f370e8-3033-4822-840e-1eaccb4e76ed None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:17:58.890245 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38f370e8-3033-4822-840e-1eaccb4e76ed None admin] RBAC: Authorization granted {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:17:58.901383 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: [pid: 80245|app: 0|req: 495/989] 10.222.0.47 () {56 vars in 1122 bytes} [Thu Jul 21 08:17:58 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 21 08:17:58.902750 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38f370e8-3033-4822-840e-1eaccb4e76ed None admin] Authenticating user token {{(pid=80245) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:17:58.917584 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99fae951-1edb-4775-8852-8e7260785b9f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:17:58.918390 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99fae951-1edb-4775-8852-8e7260785b9f None admin] REQUEST_METHOD: `GET` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:17:58.918673 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99fae951-1edb-4775-8852-8e7260785b9f None admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:17:58.918947 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99fae951-1edb-4775-8852-8e7260785b9f None admin] PATH_INFO: `/v3/services/compute` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:17:58.919643 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99fae951-1edb-4775-8852-8e7260785b9f None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:17:58.921076 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99fae951-1edb-4775-8852-8e7260785b9f None admin] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:17:58.932966 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application [None req-99fae951-1edb-4775-8852-8e7260785b9f None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Jul 21 08:17:58.932966 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 08:17:58.932966 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 21 08:17:58.932966 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 21 08:17:58.932966 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 21 08:17:58.932966 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 21 08:17:58.932966 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 21 08:17:58.932966 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 21 08:17:58.932966 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Jul 21 08:17:58.932966 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application Jul 21 08:17:58.932966 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 21 08:17:58.932966 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application Jul 21 08:17:58.932966 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 08:17:58.932966 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 08:17:58.932966 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 08:17:58.932966 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 08:17:58.932966 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 08:17:58.933967 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 08:17:58.933967 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 08:17:58.933967 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 08:17:58.933967 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 08:17:58.933967 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 08:17:58.933967 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 08:17:58.933967 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 21 08:17:58.933967 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 21 08:17:58.933967 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 21 08:17:58.933967 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 21 08:17:58.933967 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 08:17:58.933967 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 08:17:58.933967 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 08:17:58.933967 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 08:17:58.933967 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 08:17:58.933967 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 08:17:58.934780 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 08:17:58.934780 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application with Lock( Jul 21 08:17:58.934780 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 08:17:58.934780 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self._enter() Jul 21 08:17:58.934780 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 08:17:58.934780 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 08:17:58.934780 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 08:17:58.934780 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self.creator() Jul 21 08:17:58.934780 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 08:17:58.934780 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application created_value = creator( Jul 21 08:17:58.934780 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 21 08:17:58.934780 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 21 08:17:58.934780 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Jul 21 08:17:58.934780 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application Jul 21 08:17:58.934780 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 495/990] 10.222.0.47 () {56 vars in 1111 bytes} [Thu Jul 21 08:17:58 2022] GET /identity/v3/services/compute => generated 88 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 21 08:17:58.940371 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99fae951-1edb-4775-8852-8e7260785b9f None admin] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:17:59.015984 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38f370e8-3033-4822-840e-1eaccb4e76ed None admin] Validating token access rules against request {{(pid=80245) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:17:59.048833 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99fae951-1edb-4775-8852-8e7260785b9f None admin] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:17:59.054023 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1167ce8-9ee2-4ea9-91c4-320d547b4254 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80245) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:17:59.054869 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1167ce8-9ee2-4ea9-91c4-320d547b4254 None admin] REQUEST_METHOD: `GET` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:17:59.055301 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1167ce8-9ee2-4ea9-91c4-320d547b4254 None admin] SCRIPT_NAME: `/identity` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:17:59.055893 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1167ce8-9ee2-4ea9-91c4-320d547b4254 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:17:59.056694 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1167ce8-9ee2-4ea9-91c4-320d547b4254 None admin] RBAC: Adding query filter params (interface=admin, service_id=53d339aeb0184b07a50e71eed278f656, region_id=RegionOne) {{(pid=80245) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 08:17:59.057003 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1167ce8-9ee2-4ea9-91c4-320d547b4254 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:17:59.058476 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1167ce8-9ee2-4ea9-91c4-320d547b4254 None admin] RBAC: Authorization granted {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:17:59.068595 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: [pid: 80245|app: 0|req: 496/991] 10.222.0.47 () {56 vars in 1249 bytes} [Thu Jul 21 08:17:58 2022] GET /identity/v3/endpoints?service_id=53d339aeb0184b07a50e71eed278f656&interface=admin®ion_id=RegionOne => generated 193 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 21 08:17:59.069444 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1167ce8-9ee2-4ea9-91c4-320d547b4254 None admin] Authenticating user token {{(pid=80245) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:17:59.085214 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d8b51e8-3c10-4518-b8ad-aac10cd7520f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:17:59.085892 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d8b51e8-3c10-4518-b8ad-aac10cd7520f None admin] REQUEST_METHOD: `GET` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:17:59.086177 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d8b51e8-3c10-4518-b8ad-aac10cd7520f None admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:17:59.086451 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d8b51e8-3c10-4518-b8ad-aac10cd7520f None admin] PATH_INFO: `/v3/services` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:17:59.087155 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d8b51e8-3c10-4518-b8ad-aac10cd7520f None admin] RBAC: Adding query filter params (name=compute) {{(pid=80244) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 08:17:59.087564 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d8b51e8-3c10-4518-b8ad-aac10cd7520f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:17:59.088930 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d8b51e8-3c10-4518-b8ad-aac10cd7520f None admin] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:17:59.094075 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 496/992] 10.222.0.47 () {56 vars in 1112 bytes} [Thu Jul 21 08:17:58 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 08:17:59.100926 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d8b51e8-3c10-4518-b8ad-aac10cd7520f None admin] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:17:59.114813 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG dbcounter [-] [80245] Writing DB stats keystone:SELECT=100 {{(pid=80245) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:17:59.117419 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG dbcounter [-] [80244] Writing DB stats keystone:SELECT=100 {{(pid=80244) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:17:59.175340 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1167ce8-9ee2-4ea9-91c4-320d547b4254 None admin] Validating token access rules against request {{(pid=80245) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:17:59.211684 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-112285e8-c36d-4b33-b6c3-fe3e4bb803b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80245) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:17:59.212585 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-112285e8-c36d-4b33-b6c3-fe3e4bb803b2 None admin] REQUEST_METHOD: `GET` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:17:59.213015 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-112285e8-c36d-4b33-b6c3-fe3e4bb803b2 None admin] SCRIPT_NAME: `/identity` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:17:59.213435 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-112285e8-c36d-4b33-b6c3-fe3e4bb803b2 None admin] PATH_INFO: `/v3/services` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:17:59.214289 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-112285e8-c36d-4b33-b6c3-fe3e4bb803b2 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:17:59.215739 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-112285e8-c36d-4b33-b6c3-fe3e4bb803b2 None admin] RBAC: Authorization granted {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:17:59.218383 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d8b51e8-3c10-4518-b8ad-aac10cd7520f None admin] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:17:59.223188 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: [pid: 80245|app: 0|req: 497/993] 10.222.0.47 () {56 vars in 1087 bytes} [Thu Jul 21 08:17:59 2022] GET /identity/v3/services => generated 1727 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 21 08:17:59.257606 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bd840c9-b34e-4be6-8051-e07da2262596 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:17:59.258553 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bd840c9-b34e-4be6-8051-e07da2262596 None admin] REQUEST_METHOD: `GET` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:17:59.258838 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bd840c9-b34e-4be6-8051-e07da2262596 None admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:17:59.259115 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bd840c9-b34e-4be6-8051-e07da2262596 None admin] PATH_INFO: `/v3/services` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:17:59.259877 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bd840c9-b34e-4be6-8051-e07da2262596 None admin] RBAC: Adding query filter params (type=compute) {{(pid=80244) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 08:17:59.260190 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bd840c9-b34e-4be6-8051-e07da2262596 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:17:59.261594 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bd840c9-b34e-4be6-8051-e07da2262596 None admin] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:17:59.272794 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 497/994] 10.222.0.47 () {56 vars in 1112 bytes} [Thu Jul 21 08:17:59 2022] GET /identity/v3/services?type=compute => generated 352 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 21 08:17:59.274824 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-112285e8-c36d-4b33-b6c3-fe3e4bb803b2 None admin] Authenticating user token {{(pid=80245) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:17:59.386402 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-112285e8-c36d-4b33-b6c3-fe3e4bb803b2 None admin] Validating token access rules against request {{(pid=80245) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:17:59.389496 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG dbcounter [-] [80245] Writing DB stats keystone:SELECT=100 {{(pid=80245) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:17:59.423502 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fe195ce-28d6-4597-86ff-a7d503120e0d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80245) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:17:59.424180 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fe195ce-28d6-4597-86ff-a7d503120e0d None admin] REQUEST_METHOD: `GET` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:17:59.424465 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fe195ce-28d6-4597-86ff-a7d503120e0d None admin] SCRIPT_NAME: `/identity` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:17:59.424774 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fe195ce-28d6-4597-86ff-a7d503120e0d None admin] PATH_INFO: `/v3/endpoints` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:17:59.425559 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fe195ce-28d6-4597-86ff-a7d503120e0d None admin] RBAC: Adding query filter params (interface=public, service_id=5521c9abfffd4df2b165d2d2e6dacbe5, region_id=RegionOne) {{(pid=80245) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 08:17:59.425870 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fe195ce-28d6-4597-86ff-a7d503120e0d None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:17:59.427235 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fe195ce-28d6-4597-86ff-a7d503120e0d None admin] RBAC: Authorization granted {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:17:59.433010 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: [pid: 80245|app: 0|req: 498/995] 10.222.0.47 () {56 vars in 1251 bytes} [Thu Jul 21 08:17:59 2022] GET /identity/v3/endpoints?service_id=5521c9abfffd4df2b165d2d2e6dacbe5&interface=public®ion_id=RegionOne => generated 194 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 08:17:59.438472 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bd840c9-b34e-4be6-8051-e07da2262596 None admin] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:17:59.523450 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG dbcounter [-] [80244] Writing DB stats keystone:SELECT=100 {{(pid=80244) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:17:59.548035 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bd840c9-b34e-4be6-8051-e07da2262596 None admin] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:17:59.585447 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d57cf5e-9e46-46b5-b593-15af51bf2387 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:17:59.586130 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d57cf5e-9e46-46b5-b593-15af51bf2387 None admin] REQUEST_METHOD: `GET` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:17:59.586422 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d57cf5e-9e46-46b5-b593-15af51bf2387 None admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:17:59.586696 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d57cf5e-9e46-46b5-b593-15af51bf2387 None admin] PATH_INFO: `/v3/services` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:17:59.587410 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d57cf5e-9e46-46b5-b593-15af51bf2387 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:17:59.588788 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d57cf5e-9e46-46b5-b593-15af51bf2387 None admin] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:17:59.595623 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 498/996] 10.222.0.47 () {56 vars in 1087 bytes} [Thu Jul 21 08:17:59 2022] GET /identity/v3/services => generated 1727 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 21 08:17:59.897607 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-465657f2-2df4-4370-9c3a-988cde5c90d3 None None] REQUEST_METHOD: `GET` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:17:59.898004 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-465657f2-2df4-4370-9c3a-988cde5c90d3 None None] SCRIPT_NAME: `/identity` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:17:59.898181 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-465657f2-2df4-4370-9c3a-988cde5c90d3 None None] PATH_INFO: `/` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:17:59.899750 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: [pid: 80245|app: 0|req: 499/997] 10.222.0.47 () {54 vars in 926 bytes} [Thu Jul 21 08:17:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 08:17:59.904880 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99d9a750-0f48-4e59-9d2d-4982fad56929 None None] REQUEST_METHOD: `POST` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:17:59.905173 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99d9a750-0f48-4e59-9d2d-4982fad56929 None None] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:17:59.905456 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99d9a750-0f48-4e59-9d2d-4982fad56929 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:17:59.922356 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.auth.core [None req-99d9a750-0f48-4e59-9d2d-4982fad56929 None None] MFA Rules not processed for user `2dce40b8d74c41ec94cc36e6ff05d412`. Rule list: `[]` (Enabled: `True`). {{(pid=80244) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 08:17:59.923585 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.fernet_utils [None req-99d9a750-0f48-4e59-9d2d-4982fad56929 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80244) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 08:17:59.997333 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 499/998] 10.222.0.47 () {58 vars in 1023 bytes} [Thu Jul 21 08:17:59 2022] POST /identity/v3/auth/tokens => generated 1560 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 21 08:18:00.022590 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-465657f2-2df4-4370-9c3a-988cde5c90d3 None None] Authenticating user token {{(pid=80245) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:18:00.139630 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-465657f2-2df4-4370-9c3a-988cde5c90d3 None None] Validating token access rules against request {{(pid=80245) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:18:00.180154 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d35be744-c921-4f9f-8f21-64ff9352dd8a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80245) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:18:00.180874 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d35be744-c921-4f9f-8f21-64ff9352dd8a None admin] REQUEST_METHOD: `GET` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:18:00.181158 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d35be744-c921-4f9f-8f21-64ff9352dd8a None admin] SCRIPT_NAME: `/identity` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:18:00.181438 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d35be744-c921-4f9f-8f21-64ff9352dd8a None admin] PATH_INFO: `/v3/services/image` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:18:00.182105 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d35be744-c921-4f9f-8f21-64ff9352dd8a None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:18:00.183512 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d35be744-c921-4f9f-8f21-64ff9352dd8a None admin] RBAC: Authorization granted {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:18:00.201292 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application [None req-d35be744-c921-4f9f-8f21-64ff9352dd8a None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Jul 21 08:18:00.201292 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 08:18:00.201292 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 21 08:18:00.201292 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 21 08:18:00.201292 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 21 08:18:00.201292 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 21 08:18:00.201292 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 21 08:18:00.201292 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 21 08:18:00.201292 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Jul 21 08:18:00.201292 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application Jul 21 08:18:00.201292 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 21 08:18:00.201292 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application Jul 21 08:18:00.201292 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 08:18:00.201292 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 08:18:00.201292 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 08:18:00.201292 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 08:18:00.201292 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 08:18:00.202304 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 08:18:00.202304 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 08:18:00.202304 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 08:18:00.202304 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 08:18:00.202304 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 08:18:00.202304 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 08:18:00.202304 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 21 08:18:00.202304 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 21 08:18:00.202304 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 21 08:18:00.202304 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 21 08:18:00.202304 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 08:18:00.202304 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 08:18:00.202304 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 08:18:00.202304 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 08:18:00.202304 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 08:18:00.202304 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 08:18:00.203157 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 08:18:00.203157 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application with Lock( Jul 21 08:18:00.203157 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 08:18:00.203157 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application return self._enter() Jul 21 08:18:00.203157 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 08:18:00.203157 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 08:18:00.203157 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 08:18:00.203157 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application return self.creator() Jul 21 08:18:00.203157 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 08:18:00.203157 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application created_value = creator( Jul 21 08:18:00.203157 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 21 08:18:00.203157 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 21 08:18:00.203157 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Jul 21 08:18:00.203157 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application Jul 21 08:18:00.203925 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: [pid: 80245|app: 0|req: 500/999] 10.222.0.47 () {56 vars in 1105 bytes} [Thu Jul 21 08:18:00 2022] GET /identity/v3/services/image => generated 86 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 08:18:00.209232 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99d9a750-0f48-4e59-9d2d-4982fad56929 None None] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:18:00.283224 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG dbcounter [-] [80244] Writing DB stats keystone:SELECT=100 {{(pid=80244) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:18:00.316504 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99d9a750-0f48-4e59-9d2d-4982fad56929 None None] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:18:00.353633 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12a25e83-2f3f-4199-80ba-a726d50c670d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:18:00.354310 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12a25e83-2f3f-4199-80ba-a726d50c670d None admin] REQUEST_METHOD: `GET` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:18:00.354593 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12a25e83-2f3f-4199-80ba-a726d50c670d None admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:18:00.354869 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12a25e83-2f3f-4199-80ba-a726d50c670d None admin] PATH_INFO: `/v3/services` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:18:00.355607 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12a25e83-2f3f-4199-80ba-a726d50c670d None admin] RBAC: Adding query filter params (name=image) {{(pid=80244) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 08:18:00.355914 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12a25e83-2f3f-4199-80ba-a726d50c670d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:18:00.357282 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12a25e83-2f3f-4199-80ba-a726d50c670d None admin] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:18:00.367706 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 500/1000] 10.222.0.47 () {56 vars in 1108 bytes} [Thu Jul 21 08:18:00 2022] GET /identity/v3/services?name=image => generated 122 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 21 08:18:00.369437 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d35be744-c921-4f9f-8f21-64ff9352dd8a None admin] Authenticating user token {{(pid=80245) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:18:00.409086 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG dbcounter [-] [80245] Writing DB stats keystone:SELECT=100 {{(pid=80245) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:18:00.487965 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d35be744-c921-4f9f-8f21-64ff9352dd8a None admin] Validating token access rules against request {{(pid=80245) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:18:00.543111 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f392b5d3-2680-4cbd-9d3d-c47e5a7ae722 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80245) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:18:00.544764 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f392b5d3-2680-4cbd-9d3d-c47e5a7ae722 None admin] REQUEST_METHOD: `GET` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:18:00.544764 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f392b5d3-2680-4cbd-9d3d-c47e5a7ae722 None admin] SCRIPT_NAME: `/identity` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:18:00.544928 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f392b5d3-2680-4cbd-9d3d-c47e5a7ae722 None admin] PATH_INFO: `/v3/services` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:18:00.545916 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f392b5d3-2680-4cbd-9d3d-c47e5a7ae722 None admin] RBAC: Adding query filter params (type=image) {{(pid=80245) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 08:18:00.546385 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f392b5d3-2680-4cbd-9d3d-c47e5a7ae722 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:18:00.548601 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f392b5d3-2680-4cbd-9d3d-c47e5a7ae722 None admin] RBAC: Authorization granted {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:18:00.563419 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: [pid: 80245|app: 0|req: 501/1001] 10.222.0.47 () {56 vars in 1108 bytes} [Thu Jul 21 08:18:00 2022] GET /identity/v3/services?type=image => generated 122 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 08:18:01.372119 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f78bbca-5698-4136-afe7-3b3341833a3b None None] REQUEST_METHOD: `GET` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:18:01.372665 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f78bbca-5698-4136-afe7-3b3341833a3b None None] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:18:01.372755 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f78bbca-5698-4136-afe7-3b3341833a3b None None] PATH_INFO: `/` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:18:01.374162 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 501/1002] 10.222.0.47 () {54 vars in 926 bytes} [Thu Jul 21 08:18:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 08:18:01.380567 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4253215-a68b-47f4-b165-6922247ed61d None None] REQUEST_METHOD: `POST` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:18:01.380852 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4253215-a68b-47f4-b165-6922247ed61d None None] SCRIPT_NAME: `/identity` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:18:01.381152 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4253215-a68b-47f4-b165-6922247ed61d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:18:01.399308 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.auth.core [None req-d4253215-a68b-47f4-b165-6922247ed61d None None] MFA Rules not processed for user `2dce40b8d74c41ec94cc36e6ff05d412`. Rule list: `[]` (Enabled: `True`). {{(pid=80245) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 08:18:01.400528 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.fernet_utils [None req-d4253215-a68b-47f4-b165-6922247ed61d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80245) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 08:18:01.478753 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: [pid: 80245|app: 0|req: 502/1003] 10.222.0.47 () {58 vars in 1023 bytes} [Thu Jul 21 08:18:01 2022] POST /identity/v3/auth/tokens => generated 1560 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 21 08:18:01.511256 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f78bbca-5698-4136-afe7-3b3341833a3b None None] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:18:01.617389 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f78bbca-5698-4136-afe7-3b3341833a3b None None] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:18:01.652226 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad9cb74b-a118-45b8-a56c-d80d6e42c84c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:18:01.652906 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad9cb74b-a118-45b8-a56c-d80d6e42c84c None admin] REQUEST_METHOD: `GET` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:18:01.653188 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad9cb74b-a118-45b8-a56c-d80d6e42c84c None admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:18:01.653454 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad9cb74b-a118-45b8-a56c-d80d6e42c84c None admin] PATH_INFO: `/v3/services/object-store` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:18:01.654102 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad9cb74b-a118-45b8-a56c-d80d6e42c84c None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:18:01.655498 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad9cb74b-a118-45b8-a56c-d80d6e42c84c None admin] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:18:01.657546 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG dbcounter [-] [80244] Writing DB stats keystone:SELECT=100 {{(pid=80244) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:18:01.669298 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application [None req-ad9cb74b-a118-45b8-a56c-d80d6e42c84c None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 21 08:18:01.669298 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 08:18:01.669298 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 21 08:18:01.669298 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 21 08:18:01.669298 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 21 08:18:01.669298 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 21 08:18:01.669298 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 21 08:18:01.669298 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 21 08:18:01.669298 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 21 08:18:01.669298 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application Jul 21 08:18:01.669298 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 21 08:18:01.669298 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application Jul 21 08:18:01.669298 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 08:18:01.669298 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 08:18:01.669298 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 08:18:01.669298 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 08:18:01.669298 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 08:18:01.670310 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 08:18:01.670310 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 08:18:01.670310 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 08:18:01.670310 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 08:18:01.670310 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 08:18:01.670310 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 08:18:01.670310 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 21 08:18:01.670310 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 21 08:18:01.670310 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 21 08:18:01.670310 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 21 08:18:01.670310 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 08:18:01.670310 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 08:18:01.670310 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 08:18:01.670310 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 08:18:01.670310 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 08:18:01.670310 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 08:18:01.671142 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 08:18:01.671142 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application with Lock( Jul 21 08:18:01.671142 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 08:18:01.671142 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self._enter() Jul 21 08:18:01.671142 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 08:18:01.671142 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 08:18:01.671142 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 08:18:01.671142 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self.creator() Jul 21 08:18:01.671142 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 08:18:01.671142 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application created_value = creator( Jul 21 08:18:01.671142 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 21 08:18:01.671142 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 21 08:18:01.671142 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 21 08:18:01.671142 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application Jul 21 08:18:01.674509 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 502/1004] 10.222.0.47 () {56 vars in 1126 bytes} [Thu Jul 21 08:18:01 2022] GET /identity/v3/services/object-store => generated 93 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 08:18:01.677459 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4253215-a68b-47f4-b165-6922247ed61d None None] Authenticating user token {{(pid=80245) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:18:01.695748 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a849dce8-cabb-4c21-a70a-57c362a5d93d None None] REQUEST_METHOD: `GET` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:18:01.696089 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a849dce8-cabb-4c21-a70a-57c362a5d93d None None] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:18:01.696309 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a849dce8-cabb-4c21-a70a-57c362a5d93d None None] PATH_INFO: `/` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:18:01.697928 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 503/1005] 10.222.0.47 () {54 vars in 926 bytes} [Thu Jul 21 08:18:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 21 08:18:01.703906 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edd65bc6-f7bf-45b9-9355-9bf368e8eb82 None None] REQUEST_METHOD: `POST` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:18:01.703906 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edd65bc6-f7bf-45b9-9355-9bf368e8eb82 None None] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:18:01.703906 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edd65bc6-f7bf-45b9-9355-9bf368e8eb82 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:18:01.717652 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG dbcounter [-] [80245] Writing DB stats keystone:SELECT=100 {{(pid=80245) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:18:01.723107 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.auth.core [None req-edd65bc6-f7bf-45b9-9355-9bf368e8eb82 None None] MFA Rules not processed for user `2dce40b8d74c41ec94cc36e6ff05d412`. Rule list: `[]` (Enabled: `True`). {{(pid=80244) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 08:18:01.724515 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.fernet_utils [None req-edd65bc6-f7bf-45b9-9355-9bf368e8eb82 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80244) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 08:18:01.803990 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 504/1006] 10.222.0.47 () {58 vars in 1023 bytes} [Thu Jul 21 08:18:01 2022] POST /identity/v3/auth/tokens => generated 1560 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 21 08:18:01.810902 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4253215-a68b-47f4-b165-6922247ed61d None None] Validating token access rules against request {{(pid=80245) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:18:01.827061 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edd65bc6-f7bf-45b9-9355-9bf368e8eb82 None None] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:18:01.865878 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8f00dbb-ffd9-4643-a686-f6693bc3724a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80245) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:18:01.867012 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8f00dbb-ffd9-4643-a686-f6693bc3724a None admin] REQUEST_METHOD: `GET` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:18:01.867535 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8f00dbb-ffd9-4643-a686-f6693bc3724a None admin] SCRIPT_NAME: `/identity` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:18:01.868038 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8f00dbb-ffd9-4643-a686-f6693bc3724a None admin] PATH_INFO: `/v3/services` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:18:01.869343 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8f00dbb-ffd9-4643-a686-f6693bc3724a None admin] RBAC: Adding query filter params (name=object-store) {{(pid=80245) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 08:18:01.869893 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8f00dbb-ffd9-4643-a686-f6693bc3724a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:18:01.872379 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8f00dbb-ffd9-4643-a686-f6693bc3724a None admin] RBAC: Authorization granted {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:18:01.885854 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: [pid: 80245|app: 0|req: 503/1007] 10.222.0.47 () {56 vars in 1122 bytes} [Thu Jul 21 08:18:01 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 08:18:01.888319 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8f00dbb-ffd9-4643-a686-f6693bc3724a None admin] Authenticating user token {{(pid=80245) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:18:01.948567 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edd65bc6-f7bf-45b9-9355-9bf368e8eb82 None None] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:18:01.984033 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG dbcounter [-] [80244] Writing DB stats keystone:SELECT=100 {{(pid=80244) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:18:01.990820 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG dbcounter [-] [80245] Writing DB stats keystone:SELECT=100 {{(pid=80245) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:18:01.992957 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e10322b-153c-4a64-a0e8-00408872ed62 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:18:01.994002 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e10322b-153c-4a64-a0e8-00408872ed62 None admin] REQUEST_METHOD: `GET` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:18:01.994389 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e10322b-153c-4a64-a0e8-00408872ed62 None admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:18:01.994662 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e10322b-153c-4a64-a0e8-00408872ed62 None admin] PATH_INFO: `/v3/services/compute` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:18:01.995246 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e10322b-153c-4a64-a0e8-00408872ed62 None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:18:01.996571 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e10322b-153c-4a64-a0e8-00408872ed62 None admin] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:18:02.002462 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8f00dbb-ffd9-4643-a686-f6693bc3724a None admin] Validating token access rules against request {{(pid=80245) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:18:02.011585 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application [None req-2e10322b-153c-4a64-a0e8-00408872ed62 None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Jul 21 08:18:02.011585 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 08:18:02.011585 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 21 08:18:02.011585 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 21 08:18:02.011585 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 21 08:18:02.011585 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 21 08:18:02.011585 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 21 08:18:02.011585 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 21 08:18:02.011585 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Jul 21 08:18:02.011585 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application Jul 21 08:18:02.011585 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 21 08:18:02.011585 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application Jul 21 08:18:02.011585 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 08:18:02.011585 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 08:18:02.011585 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 08:18:02.011585 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 08:18:02.011585 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 08:18:02.012645 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 08:18:02.012645 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 08:18:02.012645 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 08:18:02.012645 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 08:18:02.012645 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 08:18:02.012645 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 08:18:02.012645 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 21 08:18:02.012645 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 21 08:18:02.012645 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 21 08:18:02.012645 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 21 08:18:02.012645 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 08:18:02.012645 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 08:18:02.012645 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 08:18:02.012645 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 08:18:02.012645 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 08:18:02.012645 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 08:18:02.013516 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 08:18:02.013516 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application with Lock( Jul 21 08:18:02.013516 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 08:18:02.013516 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self._enter() Jul 21 08:18:02.013516 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 08:18:02.013516 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 08:18:02.013516 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 08:18:02.013516 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self.creator() Jul 21 08:18:02.013516 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 08:18:02.013516 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application created_value = creator( Jul 21 08:18:02.013516 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 21 08:18:02.013516 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 21 08:18:02.013516 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Jul 21 08:18:02.013516 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application Jul 21 08:18:02.013516 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 505/1008] 10.222.0.47 () {56 vars in 1111 bytes} [Thu Jul 21 08:18:01 2022] GET /identity/v3/services/compute => generated 88 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 21 08:18:02.019260 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e10322b-153c-4a64-a0e8-00408872ed62 None admin] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:18:02.043692 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6b9afb0-0172-4390-b56d-55851da49263 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80245) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:18:02.044368 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6b9afb0-0172-4390-b56d-55851da49263 None admin] REQUEST_METHOD: `GET` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:18:02.044658 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6b9afb0-0172-4390-b56d-55851da49263 None admin] SCRIPT_NAME: `/identity` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:18:02.044929 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6b9afb0-0172-4390-b56d-55851da49263 None admin] PATH_INFO: `/v3/services` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:18:02.045633 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6b9afb0-0172-4390-b56d-55851da49263 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=80245) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 08:18:02.045941 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6b9afb0-0172-4390-b56d-55851da49263 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:18:02.047366 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6b9afb0-0172-4390-b56d-55851da49263 None admin] RBAC: Authorization granted {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:18:02.057079 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: [pid: 80245|app: 0|req: 504/1009] 10.222.0.47 () {56 vars in 1122 bytes} [Thu Jul 21 08:18:01 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 21 08:18:02.058150 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6b9afb0-0172-4390-b56d-55851da49263 None admin] Authenticating user token {{(pid=80245) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:18:02.137048 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e10322b-153c-4a64-a0e8-00408872ed62 None admin] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:18:02.176454 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6b9afb0-0172-4390-b56d-55851da49263 None admin] Validating token access rules against request {{(pid=80245) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:18:02.180742 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09250153-6fca-4a27-95a6-5f6afb50d310 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:18:02.181419 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09250153-6fca-4a27-95a6-5f6afb50d310 None admin] REQUEST_METHOD: `GET` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:18:02.181697 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09250153-6fca-4a27-95a6-5f6afb50d310 None admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:18:02.181970 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09250153-6fca-4a27-95a6-5f6afb50d310 None admin] PATH_INFO: `/v3/services` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:18:02.182680 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09250153-6fca-4a27-95a6-5f6afb50d310 None admin] RBAC: Adding query filter params (name=compute) {{(pid=80244) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 08:18:02.182982 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09250153-6fca-4a27-95a6-5f6afb50d310 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:18:02.184397 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09250153-6fca-4a27-95a6-5f6afb50d310 None admin] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:18:02.190285 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 506/1010] 10.222.0.47 () {56 vars in 1112 bytes} [Thu Jul 21 08:18:02 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 21 08:18:02.196141 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09250153-6fca-4a27-95a6-5f6afb50d310 None admin] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:18:02.220052 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0c93473-08c3-448f-a5b2-468a1063a187 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80245) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:18:02.220623 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0c93473-08c3-448f-a5b2-468a1063a187 None admin] REQUEST_METHOD: `POST` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:18:02.220903 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0c93473-08c3-448f-a5b2-468a1063a187 None admin] SCRIPT_NAME: `/identity` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:18:02.221199 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0c93473-08c3-448f-a5b2-468a1063a187 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:18:02.222502 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0c93473-08c3-448f-a5b2-468a1063a187 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:18:02.224011 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0c93473-08c3-448f-a5b2-468a1063a187 None admin] RBAC: Authorization granted {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:18:02.240974 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: [pid: 80245|app: 0|req: 505/1011] 10.222.0.47 () {60 vars in 1144 bytes} [Thu Jul 21 08:18:02 2022] POST /identity/v3/endpoints => generated 325 bytes in 183 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 08:18:02.273484 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG dbcounter [-] [80244] Writing DB stats keystone:SELECT=100 {{(pid=80244) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:18:02.324114 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09250153-6fca-4a27-95a6-5f6afb50d310 None admin] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:18:02.371703 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8c996d9-0fe4-45f5-83b5-6ddb1267ae4d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:18:02.372398 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8c996d9-0fe4-45f5-83b5-6ddb1267ae4d None admin] REQUEST_METHOD: `GET` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:18:02.372972 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8c996d9-0fe4-45f5-83b5-6ddb1267ae4d None admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:18:02.372972 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8c996d9-0fe4-45f5-83b5-6ddb1267ae4d None admin] PATH_INFO: `/v3/services` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:18:02.373649 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8c996d9-0fe4-45f5-83b5-6ddb1267ae4d None admin] RBAC: Adding query filter params (type=compute) {{(pid=80244) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 08:18:02.373957 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8c996d9-0fe4-45f5-83b5-6ddb1267ae4d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:18:02.375398 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8c996d9-0fe4-45f5-83b5-6ddb1267ae4d None admin] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:18:02.389497 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 507/1012] 10.222.0.47 () {56 vars in 1112 bytes} [Thu Jul 21 08:18:02 2022] GET /identity/v3/services?type=compute => generated 352 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 21 08:18:02.390971 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0c93473-08c3-448f-a5b2-468a1063a187 None admin] Authenticating user token {{(pid=80245) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:18:02.409461 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG dbcounter [-] [80245] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80245) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:18:02.520143 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0c93473-08c3-448f-a5b2-468a1063a187 None admin] Validating token access rules against request {{(pid=80245) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:18:02.557617 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b52aacc5-cfcc-4aa4-8ff5-b67c93d23bdd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80245) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:18:02.558319 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b52aacc5-cfcc-4aa4-8ff5-b67c93d23bdd None admin] REQUEST_METHOD: `POST` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:18:02.558597 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b52aacc5-cfcc-4aa4-8ff5-b67c93d23bdd None admin] SCRIPT_NAME: `/identity` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:18:02.558949 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b52aacc5-cfcc-4aa4-8ff5-b67c93d23bdd None admin] PATH_INFO: `/v3/endpoints` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:18:02.559766 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b52aacc5-cfcc-4aa4-8ff5-b67c93d23bdd None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:18:02.561184 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b52aacc5-cfcc-4aa4-8ff5-b67c93d23bdd None admin] RBAC: Authorization granted {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:18:02.585274 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: [pid: 80245|app: 0|req: 506/1013] 10.222.0.47 () {60 vars in 1144 bytes} [Thu Jul 21 08:18:02 2022] POST /identity/v3/endpoints => generated 334 bytes in 195 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 08:18:02.655590 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6611e7c-66d7-4248-bbfb-5a11e8af532e None None] REQUEST_METHOD: `GET` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:18:02.656149 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6611e7c-66d7-4248-bbfb-5a11e8af532e None None] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:18:02.656603 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6611e7c-66d7-4248-bbfb-5a11e8af532e None None] PATH_INFO: `/` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:18:02.658404 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 508/1014] 10.222.0.47 () {54 vars in 926 bytes} [Thu Jul 21 08:18:02 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 08:18:02.665407 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3de451e8-3c07-4151-b2ea-6c837a8ae6cb None None] REQUEST_METHOD: `POST` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:18:02.665800 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3de451e8-3c07-4151-b2ea-6c837a8ae6cb None None] SCRIPT_NAME: `/identity` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:18:02.666005 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3de451e8-3c07-4151-b2ea-6c837a8ae6cb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:18:02.685592 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.auth.core [None req-3de451e8-3c07-4151-b2ea-6c837a8ae6cb None None] MFA Rules not processed for user `2dce40b8d74c41ec94cc36e6ff05d412`. Rule list: `[]` (Enabled: `True`). {{(pid=80245) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 08:18:02.686975 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.fernet_utils [None req-3de451e8-3c07-4151-b2ea-6c837a8ae6cb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80245) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 08:18:02.763500 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG dbcounter [-] [80245] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80245) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:18:02.778626 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: [pid: 80245|app: 0|req: 507/1015] 10.222.0.47 () {58 vars in 1023 bytes} [Thu Jul 21 08:18:02 2022] POST /identity/v3/auth/tokens => generated 1863 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 21 08:18:02.802286 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6611e7c-66d7-4248-bbfb-5a11e8af532e None None] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:18:02.925353 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6611e7c-66d7-4248-bbfb-5a11e8af532e None None] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:18:02.946229 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG dbcounter [-] [80244] Writing DB stats keystone:SELECT=100 {{(pid=80244) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:18:02.968056 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53dcf36a-e54d-4c32-9ecc-94b38fbc8562 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:18:02.968757 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53dcf36a-e54d-4c32-9ecc-94b38fbc8562 None admin] REQUEST_METHOD: `POST` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:18:02.969047 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53dcf36a-e54d-4c32-9ecc-94b38fbc8562 None admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:18:02.969493 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53dcf36a-e54d-4c32-9ecc-94b38fbc8562 None admin] PATH_INFO: `/v3/services` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:18:02.970086 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53dcf36a-e54d-4c32-9ecc-94b38fbc8562 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:18:02.972196 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53dcf36a-e54d-4c32-9ecc-94b38fbc8562 None admin] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:18:02.988620 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 509/1016] 10.222.0.47 () {60 vars in 1141 bytes} [Thu Jul 21 08:18:02 2022] POST /identity/v3/services => generated 242 bytes in 187 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 08:18:04.386445 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57e4e8a6-e806-48dd-8ba0-1938661e3473 None None] REQUEST_METHOD: `GET` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:18:04.387003 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57e4e8a6-e806-48dd-8ba0-1938661e3473 None None] SCRIPT_NAME: `/identity` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:18:04.387003 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57e4e8a6-e806-48dd-8ba0-1938661e3473 None None] PATH_INFO: `/` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:18:04.388456 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: [pid: 80245|app: 0|req: 508/1017] 10.222.0.47 () {54 vars in 926 bytes} [Thu Jul 21 08:18:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 08:18:04.393671 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e18065d-8259-4213-823b-5e9e3e39a871 None None] REQUEST_METHOD: `POST` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:18:04.394085 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e18065d-8259-4213-823b-5e9e3e39a871 None None] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:18:04.394246 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e18065d-8259-4213-823b-5e9e3e39a871 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:18:04.411258 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.auth.core [None req-9e18065d-8259-4213-823b-5e9e3e39a871 None None] MFA Rules not processed for user `2dce40b8d74c41ec94cc36e6ff05d412`. Rule list: `[]` (Enabled: `True`). {{(pid=80244) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 08:18:04.412504 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.fernet_utils [None req-9e18065d-8259-4213-823b-5e9e3e39a871 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80244) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 08:18:04.495063 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 510/1018] 10.222.0.47 () {58 vars in 1023 bytes} [Thu Jul 21 08:18:04 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 21 08:18:04.523355 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57e4e8a6-e806-48dd-8ba0-1938661e3473 None None] Authenticating user token {{(pid=80245) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:18:04.642502 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57e4e8a6-e806-48dd-8ba0-1938661e3473 None None] Validating token access rules against request {{(pid=80245) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:18:04.679027 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83007dec-3d69-448c-87fd-eaf26ec901f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80245) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:18:04.679863 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83007dec-3d69-448c-87fd-eaf26ec901f8 None admin] REQUEST_METHOD: `GET` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:18:04.680146 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83007dec-3d69-448c-87fd-eaf26ec901f8 None admin] SCRIPT_NAME: `/identity` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:18:04.680419 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83007dec-3d69-448c-87fd-eaf26ec901f8 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:18:04.681602 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83007dec-3d69-448c-87fd-eaf26ec901f8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:18:04.683044 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83007dec-3d69-448c-87fd-eaf26ec901f8 None admin] RBAC: Authorization granted {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:18:04.689458 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: [pid: 80245|app: 0|req: 509/1019] 10.222.0.47 () {56 vars in 1108 bytes} [Thu Jul 21 08:18:04 2022] GET /identity/v3/domains/default => generated 199 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 08:18:04.690700 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e18065d-8259-4213-823b-5e9e3e39a871 None None] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:18:04.791395 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e18065d-8259-4213-823b-5e9e3e39a871 None None] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:18:04.792961 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG dbcounter [-] [80244] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80244) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:18:04.824343 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fd139ac-7f84-4258-ac14-532854ea5aff None None] REQUEST_METHOD: `GET` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:18:04.824760 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fd139ac-7f84-4258-ac14-532854ea5aff None None] SCRIPT_NAME: `/identity` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:18:04.824892 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fd139ac-7f84-4258-ac14-532854ea5aff None None] PATH_INFO: `/` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:18:04.826484 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: [pid: 80245|app: 0|req: 510/1020] 10.222.0.47 () {54 vars in 926 bytes} [Thu Jul 21 08:18:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 21 08:18:04.829821 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdb547fa-a648-40ab-9412-57c7db42e17b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:18:04.830497 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdb547fa-a648-40ab-9412-57c7db42e17b None admin] REQUEST_METHOD: `POST` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:18:04.830899 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdb547fa-a648-40ab-9412-57c7db42e17b None admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:18:04.831310 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdb547fa-a648-40ab-9412-57c7db42e17b None admin] PATH_INFO: `/v3/projects` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:18:04.831589 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31a85ce5-69bb-4af0-b0d7-6d12a44bac5d None None] REQUEST_METHOD: `POST` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:18:04.831882 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31a85ce5-69bb-4af0-b0d7-6d12a44bac5d None None] SCRIPT_NAME: `/identity` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:18:04.832160 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31a85ce5-69bb-4af0-b0d7-6d12a44bac5d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:18:04.832319 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdb547fa-a648-40ab-9412-57c7db42e17b None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:18:04.833723 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdb547fa-a648-40ab-9412-57c7db42e17b None admin] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:18:04.851355 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.auth.core [None req-31a85ce5-69bb-4af0-b0d7-6d12a44bac5d None None] MFA Rules not processed for user `2dce40b8d74c41ec94cc36e6ff05d412`. Rule list: `[]` (Enabled: `True`). {{(pid=80245) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 08:18:04.853231 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.fernet_utils [None req-31a85ce5-69bb-4af0-b0d7-6d12a44bac5d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80245) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 08:18:04.879356 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 511/1021] 10.222.0.47 () {60 vars in 1141 bytes} [Thu Jul 21 08:18:04 2022] POST /identity/v3/projects => generated 311 bytes in 179 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 21 08:18:04.924814 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG dbcounter [-] [80245] Writing DB stats keystone:SELECT=100 {{(pid=80245) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:18:04.935968 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: [pid: 80245|app: 0|req: 511/1022] 10.222.0.47 () {58 vars in 1023 bytes} [Thu Jul 21 08:18:04 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 21 08:18:04.959992 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdb547fa-a648-40ab-9412-57c7db42e17b None admin] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:18:05.076424 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdb547fa-a648-40ab-9412-57c7db42e17b None admin] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:18:05.091266 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a78160c-a7bf-4980-b97f-e14491a2bf00 None None] REQUEST_METHOD: `GET` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:18:05.091600 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a78160c-a7bf-4980-b97f-e14491a2bf00 None None] SCRIPT_NAME: `/identity` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:18:05.091888 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a78160c-a7bf-4980-b97f-e14491a2bf00 None None] PATH_INFO: `/` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:18:05.093400 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: [pid: 80245|app: 0|req: 512/1023] 10.222.0.47 () {54 vars in 926 bytes} [Thu Jul 21 08:18:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 21 08:18:05.099366 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87355d43-9e7f-4e9d-aa39-5cc4c8b3c91f None None] REQUEST_METHOD: `POST` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:18:05.099648 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87355d43-9e7f-4e9d-aa39-5cc4c8b3c91f None None] SCRIPT_NAME: `/identity` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:18:05.099923 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87355d43-9e7f-4e9d-aa39-5cc4c8b3c91f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:18:05.113156 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c857300-ba26-46e8-8c00-03d02d79767e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:18:05.114115 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c857300-ba26-46e8-8c00-03d02d79767e None admin] REQUEST_METHOD: `GET` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:18:05.114399 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c857300-ba26-46e8-8c00-03d02d79767e None admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:18:05.114673 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c857300-ba26-46e8-8c00-03d02d79767e None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:18:05.117176 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.auth.core [None req-87355d43-9e7f-4e9d-aa39-5cc4c8b3c91f None None] MFA Rules not processed for user `2dce40b8d74c41ec94cc36e6ff05d412`. Rule list: `[]` (Enabled: `True`). {{(pid=80245) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 08:18:05.118349 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.fernet_utils [None req-87355d43-9e7f-4e9d-aa39-5cc4c8b3c91f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80245) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 08:18:05.125781 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c857300-ba26-46e8-8c00-03d02d79767e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 21 08:18:05.134702 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application [None req-0c857300-ba26-46e8-8c00-03d02d79767e None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 21 08:18:05.134702 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 08:18:05.134702 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 08:18:05.134702 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 08:18:05.134702 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 08:18:05.134702 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 08:18:05.134702 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 08:18:05.134702 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 08:18:05.134702 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 08:18:05.134702 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 08:18:05.134702 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 08:18:05.134702 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 08:18:05.134702 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 21 08:18:05.134702 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 21 08:18:05.134702 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 21 08:18:05.134702 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 21 08:18:05.135693 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 21 08:18:05.135693 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 21 08:18:05.135693 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 21 08:18:05.135693 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 21 08:18:05.135693 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 08:18:05.135693 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 08:18:05.135693 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 08:18:05.135693 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 08:18:05.135693 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 08:18:05.135693 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 08:18:05.135693 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 08:18:05.135693 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application with Lock( Jul 21 08:18:05.135693 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 08:18:05.135693 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self._enter() Jul 21 08:18:05.135693 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 08:18:05.135693 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 08:18:05.136526 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 08:18:05.136526 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self.creator() Jul 21 08:18:05.136526 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 08:18:05.136526 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application created_value = creator( Jul 21 08:18:05.136526 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 21 08:18:05.136526 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 21 08:18:05.136526 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 21 08:18:05.136526 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 21 08:18:05.136526 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 21 08:18:05.136526 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 21 08:18:05.136526 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 21 08:18:05.136526 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application Jul 21 08:18:05.136526 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 512/1024] 10.222.0.47 () {56 vars in 1120 bytes} [Thu Jul 21 08:18:04 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 08:18:05.141925 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c857300-ba26-46e8-8c00-03d02d79767e None admin] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:18:05.161044 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG dbcounter [-] [80244] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80244) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:18:05.201036 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: [pid: 80245|app: 0|req: 513/1025] 10.222.0.47 () {58 vars in 1023 bytes} [Thu Jul 21 08:18:05 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 21 08:18:05.224055 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87355d43-9e7f-4e9d-aa39-5cc4c8b3c91f None None] Authenticating user token {{(pid=80245) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:18:05.258043 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c857300-ba26-46e8-8c00-03d02d79767e None admin] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:18:05.295640 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-538ca947-0f36-4d8a-8433-ed01962f3712 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:18:05.295882 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-538ca947-0f36-4d8a-8433-ed01962f3712 None admin] REQUEST_METHOD: `GET` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:18:05.296131 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-538ca947-0f36-4d8a-8433-ed01962f3712 None admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:18:05.296351 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-538ca947-0f36-4d8a-8433-ed01962f3712 None admin] PATH_INFO: `/v3/roles` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:18:05.297096 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-538ca947-0f36-4d8a-8433-ed01962f3712 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80244) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 08:18:05.297405 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-538ca947-0f36-4d8a-8433-ed01962f3712 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:18:05.298779 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-538ca947-0f36-4d8a-8433-ed01962f3712 None admin] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:18:05.311843 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 513/1026] 10.222.0.47 () {56 vars in 1115 bytes} [Thu Jul 21 08:18:05 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 08:18:05.312856 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-538ca947-0f36-4d8a-8433-ed01962f3712 None admin] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:18:05.326883 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87355d43-9e7f-4e9d-aa39-5cc4c8b3c91f None None] Validating token access rules against request {{(pid=80245) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:18:05.352283 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG dbcounter [-] [80245] Writing DB stats keystone:SELECT=100 {{(pid=80245) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:18:05.364160 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f36f14af-0a9c-48ab-b7c0-a5873cf6c745 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80245) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:18:05.364968 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f36f14af-0a9c-48ab-b7c0-a5873cf6c745 None admin] REQUEST_METHOD: `GET` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:18:05.365251 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f36f14af-0a9c-48ab-b7c0-a5873cf6c745 None admin] SCRIPT_NAME: `/identity` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:18:05.365530 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f36f14af-0a9c-48ab-b7c0-a5873cf6c745 None admin] PATH_INFO: `/v3/services/image` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:18:05.366184 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f36f14af-0a9c-48ab-b7c0-a5873cf6c745 None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:18:05.367587 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f36f14af-0a9c-48ab-b7c0-a5873cf6c745 None admin] RBAC: Authorization granted {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:18:05.379741 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application [None req-f36f14af-0a9c-48ab-b7c0-a5873cf6c745 None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Jul 21 08:18:05.379741 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 08:18:05.379741 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 21 08:18:05.379741 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 21 08:18:05.379741 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 21 08:18:05.379741 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 21 08:18:05.379741 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 21 08:18:05.379741 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 21 08:18:05.379741 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Jul 21 08:18:05.379741 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application Jul 21 08:18:05.379741 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 21 08:18:05.379741 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application Jul 21 08:18:05.379741 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 08:18:05.379741 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 08:18:05.379741 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 08:18:05.379741 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 08:18:05.379741 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 08:18:05.380933 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 08:18:05.380933 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 08:18:05.380933 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 08:18:05.380933 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 08:18:05.380933 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 08:18:05.380933 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 08:18:05.380933 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 21 08:18:05.380933 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 21 08:18:05.380933 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 21 08:18:05.380933 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 21 08:18:05.380933 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 08:18:05.380933 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 08:18:05.380933 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 08:18:05.380933 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 08:18:05.380933 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 08:18:05.380933 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 08:18:05.381786 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 08:18:05.381786 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application with Lock( Jul 21 08:18:05.381786 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 08:18:05.381786 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application return self._enter() Jul 21 08:18:05.381786 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 08:18:05.381786 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 08:18:05.381786 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 08:18:05.381786 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application return self.creator() Jul 21 08:18:05.381786 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 08:18:05.381786 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application created_value = creator( Jul 21 08:18:05.381786 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 21 08:18:05.381786 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 21 08:18:05.381786 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Jul 21 08:18:05.381786 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application Jul 21 08:18:05.381786 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: [pid: 80245|app: 0|req: 514/1027] 10.222.0.47 () {56 vars in 1105 bytes} [Thu Jul 21 08:18:05 2022] GET /identity/v3/services/image => generated 86 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 21 08:18:05.387080 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f36f14af-0a9c-48ab-b7c0-a5873cf6c745 None admin] Authenticating user token {{(pid=80245) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:18:05.402314 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG dbcounter [-] [80244] Writing DB stats keystone:SELECT=100 {{(pid=80244) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:18:05.427822 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-538ca947-0f36-4d8a-8433-ed01962f3712 None admin] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:18:05.466720 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de1c0540-bcb4-4187-98cc-b53474b53d7d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:18:05.467474 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de1c0540-bcb4-4187-98cc-b53474b53d7d None admin] REQUEST_METHOD: `GET` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:18:05.467777 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de1c0540-bcb4-4187-98cc-b53474b53d7d None admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:18:05.468050 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de1c0540-bcb4-4187-98cc-b53474b53d7d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:18:05.479877 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de1c0540-bcb4-4187-98cc-b53474b53d7d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:18:05.481458 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de1c0540-bcb4-4187-98cc-b53474b53d7d None admin] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:18:05.500598 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f36f14af-0a9c-48ab-b7c0-a5873cf6c745 None admin] Validating token access rules against request {{(pid=80245) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:18:05.502347 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application [None req-de1c0540-bcb4-4187-98cc-b53474b53d7d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 08:18:05.502347 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 08:18:05.502347 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 08:18:05.502347 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 08:18:05.502347 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 08:18:05.502347 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 08:18:05.502347 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 08:18:05.502347 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 08:18:05.502347 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 08:18:05.502347 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 08:18:05.502347 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 08:18:05.502347 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 08:18:05.502347 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 21 08:18:05.502347 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 21 08:18:05.502347 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 21 08:18:05.504718 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 21 08:18:05.504718 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 08:18:05.504718 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 08:18:05.504718 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 08:18:05.504718 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 08:18:05.504718 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 08:18:05.504718 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 08:18:05.504718 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 08:18:05.504718 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application with Lock( Jul 21 08:18:05.504718 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 08:18:05.504718 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self._enter() Jul 21 08:18:05.504718 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 08:18:05.504718 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 08:18:05.504718 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 08:18:05.504718 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self.creator() Jul 21 08:18:05.504718 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 08:18:05.504718 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application created_value = creator( Jul 21 08:18:05.505629 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 21 08:18:05.505629 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 21 08:18:05.505629 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 08:18:05.505629 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application Jul 21 08:18:05.505629 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 514/1028] 10.222.0.47 () {56 vars in 1108 bytes} [Thu Jul 21 08:18:05 2022] GET /identity/v3/domains/Default => generated 87 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 08:18:05.511347 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de1c0540-bcb4-4187-98cc-b53474b53d7d None admin] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:18:05.541346 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05914e75-04d0-4d08-9597-45af958c3e05 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80245) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:18:05.542026 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05914e75-04d0-4d08-9597-45af958c3e05 None admin] REQUEST_METHOD: `GET` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:18:05.542314 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05914e75-04d0-4d08-9597-45af958c3e05 None admin] SCRIPT_NAME: `/identity` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:18:05.542593 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05914e75-04d0-4d08-9597-45af958c3e05 None admin] PATH_INFO: `/v3/services` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:18:05.543299 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05914e75-04d0-4d08-9597-45af958c3e05 None admin] RBAC: Adding query filter params (name=image) {{(pid=80245) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 08:18:05.543636 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05914e75-04d0-4d08-9597-45af958c3e05 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:18:05.545012 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05914e75-04d0-4d08-9597-45af958c3e05 None admin] RBAC: Authorization granted {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:18:05.551018 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: [pid: 80245|app: 0|req: 515/1029] 10.222.0.47 () {56 vars in 1108 bytes} [Thu Jul 21 08:18:05 2022] GET /identity/v3/services?name=image => generated 122 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 08:18:05.556316 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05914e75-04d0-4d08-9597-45af958c3e05 None admin] Authenticating user token {{(pid=80245) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:18:05.612639 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG dbcounter [-] [80245] Writing DB stats keystone:SELECT=100 {{(pid=80245) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:18:05.621245 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de1c0540-bcb4-4187-98cc-b53474b53d7d None admin] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:18:05.649205 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG dbcounter [-] [80244] Writing DB stats keystone:SELECT=100 {{(pid=80244) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:18:05.657807 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28135d98-45e8-41c6-b18e-df66f7f22ce7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:18:05.658480 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28135d98-45e8-41c6-b18e-df66f7f22ce7 None admin] REQUEST_METHOD: `GET` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:18:05.658772 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28135d98-45e8-41c6-b18e-df66f7f22ce7 None admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:18:05.659046 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28135d98-45e8-41c6-b18e-df66f7f22ce7 None admin] PATH_INFO: `/v3/domains` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:18:05.659818 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28135d98-45e8-41c6-b18e-df66f7f22ce7 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80244) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 08:18:05.660165 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28135d98-45e8-41c6-b18e-df66f7f22ce7 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:18:05.661506 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28135d98-45e8-41c6-b18e-df66f7f22ce7 None admin] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:18:05.666047 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05914e75-04d0-4d08-9597-45af958c3e05 None admin] Validating token access rules against request {{(pid=80245) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:18:05.676208 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 515/1030] 10.222.0.47 () {56 vars in 1109 bytes} [Thu Jul 21 08:18:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 21 08:18:05.682255 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28135d98-45e8-41c6-b18e-df66f7f22ce7 None admin] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:18:05.705097 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f01f198b-1f72-4228-bc83-c0b30aadce38 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80245) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:18:05.705853 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f01f198b-1f72-4228-bc83-c0b30aadce38 None admin] REQUEST_METHOD: `GET` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:18:05.706166 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f01f198b-1f72-4228-bc83-c0b30aadce38 None admin] SCRIPT_NAME: `/identity` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:18:05.706435 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f01f198b-1f72-4228-bc83-c0b30aadce38 None admin] PATH_INFO: `/v3/services` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:18:05.707126 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f01f198b-1f72-4228-bc83-c0b30aadce38 None admin] RBAC: Adding query filter params (type=image) {{(pid=80245) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 08:18:05.707542 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f01f198b-1f72-4228-bc83-c0b30aadce38 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:18:05.708860 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f01f198b-1f72-4228-bc83-c0b30aadce38 None admin] RBAC: Authorization granted {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:18:05.718724 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: [pid: 80245|app: 0|req: 516/1031] 10.222.0.47 () {56 vars in 1108 bytes} [Thu Jul 21 08:18:05 2022] GET /identity/v3/services?type=image => generated 350 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 21 08:18:05.719675 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f01f198b-1f72-4228-bc83-c0b30aadce38 None admin] Authenticating user token {{(pid=80245) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:18:05.791856 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28135d98-45e8-41c6-b18e-df66f7f22ce7 None admin] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:18:05.827189 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f01f198b-1f72-4228-bc83-c0b30aadce38 None admin] Validating token access rules against request {{(pid=80245) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:18:05.827675 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b70fd848-3349-4093-b2d5-3502f0435de5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:18:05.828470 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b70fd848-3349-4093-b2d5-3502f0435de5 None admin] REQUEST_METHOD: `GET` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:18:05.828781 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b70fd848-3349-4093-b2d5-3502f0435de5 None admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:18:05.829015 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b70fd848-3349-4093-b2d5-3502f0435de5 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:18:05.836383 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b70fd848-3349-4093-b2d5-3502f0435de5 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:18:05.837825 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b70fd848-3349-4093-b2d5-3502f0435de5 None admin] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:18:05.843998 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG dbcounter [-] [80245] Writing DB stats keystone:SELECT=100 {{(pid=80245) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:18:05.853541 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application [None req-b70fd848-3349-4093-b2d5-3502f0435de5 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jul 21 08:18:05.853541 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 08:18:05.853541 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 08:18:05.853541 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 08:18:05.853541 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 08:18:05.853541 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 08:18:05.853541 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 08:18:05.853541 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 08:18:05.853541 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 08:18:05.853541 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 08:18:05.853541 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 08:18:05.853541 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 08:18:05.853541 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 21 08:18:05.853541 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 21 08:18:05.853541 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 21 08:18:05.853541 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 21 08:18:05.854502 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 08:18:05.854502 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 08:18:05.854502 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 21 08:18:05.854502 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 21 08:18:05.854502 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 21 08:18:05.854502 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 21 08:18:05.854502 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 08:18:05.854502 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 08:18:05.854502 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 08:18:05.854502 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 08:18:05.854502 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 08:18:05.854502 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application with Lock( Jul 21 08:18:05.854502 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 08:18:05.854502 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self._enter() Jul 21 08:18:05.854502 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 08:18:05.854502 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 08:18:05.854502 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 08:18:05.854502 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self.creator() Jul 21 08:18:05.855506 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 08:18:05.855506 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application created_value = creator( Jul 21 08:18:05.855506 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 21 08:18:05.855506 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 21 08:18:05.855506 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 21 08:18:05.855506 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 21 08:18:05.855506 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 21 08:18:05.855506 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 21 08:18:05.855506 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jul 21 08:18:05.855506 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application Jul 21 08:18:05.855506 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 516/1032] 10.222.0.47 () {56 vars in 1093 bytes} [Thu Jul 21 08:18:05 2022] GET /identity/v3/users/nova => generated 82 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 21 08:18:05.860404 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b70fd848-3349-4093-b2d5-3502f0435de5 None admin] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:18:05.866149 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7b4b125-23bb-4d92-bff3-4a2101da2d88 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80245) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:18:05.867165 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7b4b125-23bb-4d92-bff3-4a2101da2d88 None admin] REQUEST_METHOD: `GET` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:18:05.867671 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7b4b125-23bb-4d92-bff3-4a2101da2d88 None admin] SCRIPT_NAME: `/identity` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:18:05.868124 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7b4b125-23bb-4d92-bff3-4a2101da2d88 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:18:05.869093 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7b4b125-23bb-4d92-bff3-4a2101da2d88 None admin] RBAC: Adding query filter params (interface=public, service_id=572cad225133496896f33dec6ef08ab4, region_id=RegionOne) {{(pid=80245) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 08:18:05.869592 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7b4b125-23bb-4d92-bff3-4a2101da2d88 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:18:05.870962 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7b4b125-23bb-4d92-bff3-4a2101da2d88 None admin] RBAC: Authorization granted {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:18:05.885242 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: [pid: 80245|app: 0|req: 517/1033] 10.222.0.47 () {56 vars in 1251 bytes} [Thu Jul 21 08:18:05 2022] GET /identity/v3/endpoints?service_id=572cad225133496896f33dec6ef08ab4&interface=public®ion_id=RegionOne => generated 194 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 08:18:05.886648 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7b4b125-23bb-4d92-bff3-4a2101da2d88 None admin] Authenticating user token {{(pid=80245) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:18:05.944755 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG dbcounter [-] [80244] Writing DB stats keystone:SELECT=100 {{(pid=80244) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:18:06.029701 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b70fd848-3349-4093-b2d5-3502f0435de5 None admin] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:18:06.045625 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7b4b125-23bb-4d92-bff3-4a2101da2d88 None admin] Validating token access rules against request {{(pid=80245) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:18:06.072040 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f595ddfb-8087-46dd-a649-52ca25dddfca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:18:06.072715 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f595ddfb-8087-46dd-a649-52ca25dddfca None admin] REQUEST_METHOD: `GET` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:18:06.073000 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f595ddfb-8087-46dd-a649-52ca25dddfca None admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:18:06.073274 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f595ddfb-8087-46dd-a649-52ca25dddfca None admin] PATH_INFO: `/v3/users` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:18:06.074233 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f595ddfb-8087-46dd-a649-52ca25dddfca None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80244) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 08:18:06.074664 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f595ddfb-8087-46dd-a649-52ca25dddfca None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:18:06.076111 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f595ddfb-8087-46dd-a649-52ca25dddfca None admin] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:18:06.082651 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60e264d6-22be-4669-ad77-f30254d1de7b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80245) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:18:06.083343 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60e264d6-22be-4669-ad77-f30254d1de7b None admin] REQUEST_METHOD: `GET` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:18:06.083631 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60e264d6-22be-4669-ad77-f30254d1de7b None admin] SCRIPT_NAME: `/identity` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:18:06.083908 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60e264d6-22be-4669-ad77-f30254d1de7b None admin] PATH_INFO: `/v3/services` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:18:06.084699 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60e264d6-22be-4669-ad77-f30254d1de7b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:18:06.086159 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60e264d6-22be-4669-ad77-f30254d1de7b None admin] RBAC: Authorization granted {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:18:06.103020 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: [pid: 80245|app: 0|req: 518/1034] 10.222.0.47 () {56 vars in 1087 bytes} [Thu Jul 21 08:18:05 2022] GET /identity/v3/services => generated 1957 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 08:18:06.103254 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 517/1035] 10.222.0.47 () {56 vars in 1133 bytes} [Thu Jul 21 08:18:05 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 234 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 21 08:18:06.103418 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60e264d6-22be-4669-ad77-f30254d1de7b None admin] Authenticating user token {{(pid=80245) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:18:06.153153 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG dbcounter [-] [80245] Writing DB stats keystone:SELECT=100 {{(pid=80245) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:18:06.235769 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60e264d6-22be-4669-ad77-f30254d1de7b None admin] Validating token access rules against request {{(pid=80245) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:18:06.272763 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e11bb9d5-1ada-4b3b-bbfc-636aa1d5fc29 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80245) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:18:06.273427 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e11bb9d5-1ada-4b3b-bbfc-636aa1d5fc29 None admin] REQUEST_METHOD: `GET` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:18:06.273711 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e11bb9d5-1ada-4b3b-bbfc-636aa1d5fc29 None admin] SCRIPT_NAME: `/identity` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:18:06.273989 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e11bb9d5-1ada-4b3b-bbfc-636aa1d5fc29 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:18:06.347572 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e11bb9d5-1ada-4b3b-bbfc-636aa1d5fc29 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:18:06.348945 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e11bb9d5-1ada-4b3b-bbfc-636aa1d5fc29 None admin] RBAC: Authorization granted {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:18:06.395388 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG dbcounter [-] [80245] Writing DB stats keystone:SELECT=100 {{(pid=80245) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:18:06.456713 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: [pid: 80245|app: 0|req: 519/1036] 10.222.0.47 () {58 vars in 1282 bytes} [Thu Jul 21 08:18:06 2022] GET /identity/v3/auth/tokens => generated 1959 bytes in 352 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 21 08:18:06.458199 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f595ddfb-8087-46dd-a649-52ca25dddfca None admin] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:18:06.572486 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f595ddfb-8087-46dd-a649-52ca25dddfca None admin] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:18:06.581394 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG dbcounter [-] [80244] Writing DB stats keystone:SELECT=100 {{(pid=80244) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:18:06.611372 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52a909b5-7527-4c88-831e-ea637d7913e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:18:06.612110 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52a909b5-7527-4c88-831e-ea637d7913e6 None admin] REQUEST_METHOD: `GET` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:18:06.612397 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52a909b5-7527-4c88-831e-ea637d7913e6 None admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:18:06.612673 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52a909b5-7527-4c88-831e-ea637d7913e6 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:18:06.623234 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52a909b5-7527-4c88-831e-ea637d7913e6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:18:06.624723 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52a909b5-7527-4c88-831e-ea637d7913e6 None admin] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:18:06.644699 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application [None req-52a909b5-7527-4c88-831e-ea637d7913e6 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 08:18:06.644699 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 08:18:06.644699 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 08:18:06.644699 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 08:18:06.644699 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 08:18:06.644699 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 08:18:06.644699 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 08:18:06.644699 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 08:18:06.644699 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 08:18:06.644699 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 08:18:06.644699 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 08:18:06.644699 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 08:18:06.644699 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 21 08:18:06.644699 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 21 08:18:06.644699 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 21 08:18:06.645644 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 21 08:18:06.645644 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 08:18:06.645644 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 08:18:06.645644 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 08:18:06.645644 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 08:18:06.645644 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 08:18:06.645644 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 08:18:06.645644 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 08:18:06.645644 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application with Lock( Jul 21 08:18:06.645644 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 08:18:06.645644 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self._enter() Jul 21 08:18:06.645644 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 08:18:06.645644 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 08:18:06.645644 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 08:18:06.645644 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self.creator() Jul 21 08:18:06.645644 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 08:18:06.645644 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application created_value = creator( Jul 21 08:18:06.646567 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 21 08:18:06.646567 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 21 08:18:06.646567 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 08:18:06.646567 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application Jul 21 08:18:06.646567 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 518/1037] 10.222.0.47 () {56 vars in 1108 bytes} [Thu Jul 21 08:18:06 2022] GET /identity/v3/domains/Default => generated 87 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 21 08:18:06.652903 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e11bb9d5-1ada-4b3b-bbfc-636aa1d5fc29 None admin] Authenticating user token {{(pid=80245) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:18:06.756177 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e11bb9d5-1ada-4b3b-bbfc-636aa1d5fc29 None admin] Validating token access rules against request {{(pid=80245) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:18:06.790991 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c98807bb-9a5f-4869-bee5-4ee97d98af7e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80245) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:18:06.791708 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c98807bb-9a5f-4869-bee5-4ee97d98af7e None admin] REQUEST_METHOD: `GET` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:18:06.791996 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c98807bb-9a5f-4869-bee5-4ee97d98af7e None admin] SCRIPT_NAME: `/identity` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:18:06.792344 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c98807bb-9a5f-4869-bee5-4ee97d98af7e None admin] PATH_INFO: `/v3/domains` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:18:06.792995 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c98807bb-9a5f-4869-bee5-4ee97d98af7e None admin] RBAC: Adding query filter params (name=Default) {{(pid=80245) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 08:18:06.793303 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c98807bb-9a5f-4869-bee5-4ee97d98af7e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:18:06.794673 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c98807bb-9a5f-4869-bee5-4ee97d98af7e None admin] RBAC: Authorization granted {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:18:06.807070 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: [pid: 80245|app: 0|req: 520/1038] 10.222.0.47 () {56 vars in 1109 bytes} [Thu Jul 21 08:18:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 08:18:06.811747 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52a909b5-7527-4c88-831e-ea637d7913e6 None admin] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:18:06.917965 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52a909b5-7527-4c88-831e-ea637d7913e6 None admin] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:18:06.952322 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d221c07-98ed-43fb-b9f9-d710c5b9dbbe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:18:06.953118 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d221c07-98ed-43fb-b9f9-d710c5b9dbbe None admin] REQUEST_METHOD: `GET` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:18:06.953401 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d221c07-98ed-43fb-b9f9-d710c5b9dbbe None admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:18:06.953676 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d221c07-98ed-43fb-b9f9-d710c5b9dbbe None admin] PATH_INFO: `/v3/projects/service` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:18:06.960999 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d221c07-98ed-43fb-b9f9-d710c5b9dbbe None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:18:06.962440 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d221c07-98ed-43fb-b9f9-d710c5b9dbbe None admin] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:18:06.975089 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58f29452-f940-4e93-8a34-4651691b33fe None None] REQUEST_METHOD: `GET` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:18:06.975403 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58f29452-f940-4e93-8a34-4651691b33fe None None] SCRIPT_NAME: `/identity` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:18:06.975690 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58f29452-f940-4e93-8a34-4651691b33fe None None] PATH_INFO: `/` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:18:06.976014 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application [None req-2d221c07-98ed-43fb-b9f9-d710c5b9dbbe None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 21 08:18:06.976014 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 08:18:06.976014 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 08:18:06.976014 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 08:18:06.976014 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 08:18:06.976014 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 08:18:06.976014 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 08:18:06.976014 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 08:18:06.976014 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 08:18:06.976014 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 08:18:06.976014 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 08:18:06.976014 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 08:18:06.976014 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 21 08:18:06.976014 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 21 08:18:06.976014 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 21 08:18:06.976853 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 21 08:18:06.976853 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 08:18:06.976853 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 08:18:06.976853 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 08:18:06.976853 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 08:18:06.976853 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 08:18:06.976853 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 08:18:06.976853 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 08:18:06.976853 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application with Lock( Jul 21 08:18:06.976853 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 08:18:06.976853 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self._enter() Jul 21 08:18:06.976853 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 08:18:06.976853 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 08:18:06.976853 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 08:18:06.976853 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self.creator() Jul 21 08:18:06.976853 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 08:18:06.976853 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application created_value = creator( Jul 21 08:18:06.982659 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 21 08:18:06.982659 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 21 08:18:06.982659 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 21 08:18:06.982659 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 21 08:18:06.982659 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 21 08:18:06.982659 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 21 08:18:06.982659 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 21 08:18:06.982659 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application Jul 21 08:18:06.982659 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 519/1040] 10.222.0.47 () {56 vars in 1111 bytes} [Thu Jul 21 08:18:06 2022] GET /identity/v3/projects/service => generated 88 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 21 08:18:06.983149 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: [pid: 80245|app: 0|req: 521/1040] 10.222.0.47 () {54 vars in 926 bytes} [Thu Jul 21 08:18:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 21 08:18:06.983221 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17fc6c37-f77e-4ae2-aceb-31d63ff61071 None None] REQUEST_METHOD: `POST` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:18:06.983221 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17fc6c37-f77e-4ae2-aceb-31d63ff61071 None None] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:18:06.983221 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17fc6c37-f77e-4ae2-aceb-31d63ff61071 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:18:06.984423 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58f29452-f940-4e93-8a34-4651691b33fe None None] Authenticating user token {{(pid=80245) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:18:06.988239 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG dbcounter [-] [80245] Writing DB stats keystone:SELECT=100 {{(pid=80245) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:18:07.006232 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.auth.core [None req-17fc6c37-f77e-4ae2-aceb-31d63ff61071 None None] MFA Rules not processed for user `2dce40b8d74c41ec94cc36e6ff05d412`. Rule list: `[]` (Enabled: `True`). {{(pid=80244) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 08:18:07.007442 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.fernet_utils [None req-17fc6c37-f77e-4ae2-aceb-31d63ff61071 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80244) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 08:18:07.022583 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG dbcounter [-] [80244] Writing DB stats keystone:SELECT=100 {{(pid=80244) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:18:07.085751 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 520/1041] 10.222.0.47 () {58 vars in 1023 bytes} [Thu Jul 21 08:18:06 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 21 08:18:07.095762 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58f29452-f940-4e93-8a34-4651691b33fe None None] Validating token access rules against request {{(pid=80245) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:18:07.109558 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17fc6c37-f77e-4ae2-aceb-31d63ff61071 None None] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:18:07.131318 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e31806c5-abba-417c-ab15-4591b6591dd7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80245) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:18:07.132332 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e31806c5-abba-417c-ab15-4591b6591dd7 None admin] REQUEST_METHOD: `GET` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:18:07.132617 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e31806c5-abba-417c-ab15-4591b6591dd7 None admin] SCRIPT_NAME: `/identity` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:18:07.132895 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e31806c5-abba-417c-ab15-4591b6591dd7 None admin] PATH_INFO: `/v3/projects` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:18:07.133931 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e31806c5-abba-417c-ab15-4591b6591dd7 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80245) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 08:18:07.134241 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e31806c5-abba-417c-ab15-4591b6591dd7 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:18:07.135710 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e31806c5-abba-417c-ab15-4591b6591dd7 None admin] RBAC: Authorization granted {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:18:07.152556 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: [pid: 80245|app: 0|req: 522/1042] 10.222.0.47 () {56 vars in 1148 bytes} [Thu Jul 21 08:18:06 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 08:18:07.159280 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e31806c5-abba-417c-ab15-4591b6591dd7 None admin] Authenticating user token {{(pid=80245) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:18:07.224995 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG dbcounter [-] [80245] Writing DB stats keystone:SELECT=100 {{(pid=80245) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:18:07.226202 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17fc6c37-f77e-4ae2-aceb-31d63ff61071 None None] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:18:07.266221 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86eae561-431e-49c5-a2de-b528e76fd214 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:18:07.267104 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86eae561-431e-49c5-a2de-b528e76fd214 None admin] REQUEST_METHOD: `GET` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:18:07.267424 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86eae561-431e-49c5-a2de-b528e76fd214 None admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:18:07.267699 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86eae561-431e-49c5-a2de-b528e76fd214 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:18:07.268907 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86eae561-431e-49c5-a2de-b528e76fd214 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:18:07.269163 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e31806c5-abba-417c-ab15-4591b6591dd7 None admin] Validating token access rules against request {{(pid=80245) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:18:07.270346 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86eae561-431e-49c5-a2de-b528e76fd214 None admin] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:18:07.272312 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 521/1043] 10.222.0.47 () {56 vars in 1108 bytes} [Thu Jul 21 08:18:07 2022] GET /identity/v3/domains/default => generated 199 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 21 08:18:07.277611 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86eae561-431e-49c5-a2de-b528e76fd214 None admin] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:18:07.296425 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG dbcounter [-] [80244] Writing DB stats keystone:SELECT=100 {{(pid=80244) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:18:07.308102 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b657abb2-4966-4325-9706-de93e12672d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80245) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:18:07.308770 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b657abb2-4966-4325-9706-de93e12672d0 None admin] REQUEST_METHOD: `GET` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:18:07.309099 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b657abb2-4966-4325-9706-de93e12672d0 None admin] SCRIPT_NAME: `/identity` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:18:07.309389 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b657abb2-4966-4325-9706-de93e12672d0 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:18:07.310214 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b657abb2-4966-4325-9706-de93e12672d0 None admin] RBAC: Adding query filter params (role.id=4b8dff36c32b49618b303451de1d1f69, scope.project.id=4faef13aa50e4e36b6c265fb1398fa14, user.id=ac2b5b99a36442148fe743269054e991) {{(pid=80245) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 08:18:07.310525 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b657abb2-4966-4325-9706-de93e12672d0 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:18:07.312051 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b657abb2-4966-4325-9706-de93e12672d0 None admin] RBAC: Authorization granted {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:18:07.318379 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: [pid: 80245|app: 0|req: 523/1044] 10.222.0.47 () {56 vars in 1374 bytes} [Thu Jul 21 08:18:07 2022] GET /identity/v3/role_assignments?user.id=ac2b5b99a36442148fe743269054e991&scope.project.id=4faef13aa50e4e36b6c265fb1398fa14&role.id=4b8dff36c32b49618b303451de1d1f69 => generated 259 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 21 08:18:07.397553 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86eae561-431e-49c5-a2de-b528e76fd214 None admin] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:18:07.435107 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ce9552b-e57a-4c9f-9818-7569eedf7bee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:18:07.435895 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ce9552b-e57a-4c9f-9818-7569eedf7bee None admin] REQUEST_METHOD: `POST` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:18:07.436182 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ce9552b-e57a-4c9f-9818-7569eedf7bee None admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:18:07.436457 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ce9552b-e57a-4c9f-9818-7569eedf7bee None admin] PATH_INFO: `/v3/users` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:18:07.437188 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ce9552b-e57a-4c9f-9818-7569eedf7bee None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:18:07.438638 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ce9552b-e57a-4c9f-9818-7569eedf7bee None admin] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:18:07.460958 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 522/1045] 10.222.0.47 () {60 vars in 1132 bytes} [Thu Jul 21 08:18:07 2022] POST /identity/v3/users => generated 285 bytes in 184 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 21 08:18:08.182327 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e56718a-d2f7-47e7-9245-bf81231d158f None None] REQUEST_METHOD: `GET` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:18:08.182876 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e56718a-d2f7-47e7-9245-bf81231d158f None None] SCRIPT_NAME: `/identity` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:18:08.182968 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e56718a-d2f7-47e7-9245-bf81231d158f None None] PATH_INFO: `/` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:18:08.184388 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: [pid: 80245|app: 0|req: 524/1046] 10.222.0.47 () {54 vars in 926 bytes} [Thu Jul 21 08:18:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 08:18:08.189521 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4c8f252-3669-41d6-9041-7506b05d5683 None None] REQUEST_METHOD: `POST` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:18:08.189934 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4c8f252-3669-41d6-9041-7506b05d5683 None None] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:18:08.190304 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4c8f252-3669-41d6-9041-7506b05d5683 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:18:08.208410 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.auth.core [None req-a4c8f252-3669-41d6-9041-7506b05d5683 None None] MFA Rules not processed for user `2dce40b8d74c41ec94cc36e6ff05d412`. Rule list: `[]` (Enabled: `True`). {{(pid=80244) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 08:18:08.209920 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.fernet_utils [None req-a4c8f252-3669-41d6-9041-7506b05d5683 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80244) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 08:18:08.282478 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 523/1047] 10.222.0.47 () {58 vars in 1023 bytes} [Thu Jul 21 08:18:08 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 21 08:18:08.305032 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e56718a-d2f7-47e7-9245-bf81231d158f None None] Authenticating user token {{(pid=80245) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:18:08.407396 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e56718a-d2f7-47e7-9245-bf81231d158f None None] Validating token access rules against request {{(pid=80245) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:18:08.434020 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG dbcounter [-] [80245] Writing DB stats keystone:SELECT=100 {{(pid=80245) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:18:08.445675 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3c8e62d-9eab-406c-afc3-6521ac407f11 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80245) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:18:08.446496 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3c8e62d-9eab-406c-afc3-6521ac407f11 None admin] REQUEST_METHOD: `GET` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:18:08.446779 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3c8e62d-9eab-406c-afc3-6521ac407f11 None admin] SCRIPT_NAME: `/identity` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:18:08.447055 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3c8e62d-9eab-406c-afc3-6521ac407f11 None admin] PATH_INFO: `/v3/services/image` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:18:08.447755 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3c8e62d-9eab-406c-afc3-6521ac407f11 None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:18:08.449138 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3c8e62d-9eab-406c-afc3-6521ac407f11 None admin] RBAC: Authorization granted {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:18:08.462010 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application [None req-f3c8e62d-9eab-406c-afc3-6521ac407f11 None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Jul 21 08:18:08.462010 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 08:18:08.462010 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 21 08:18:08.462010 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 21 08:18:08.462010 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 21 08:18:08.462010 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 21 08:18:08.462010 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 21 08:18:08.462010 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 21 08:18:08.462010 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Jul 21 08:18:08.462010 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application Jul 21 08:18:08.462010 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 21 08:18:08.462010 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application Jul 21 08:18:08.462010 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 08:18:08.462010 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 08:18:08.462010 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 08:18:08.462010 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 08:18:08.462010 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 08:18:08.463016 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 08:18:08.463016 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 08:18:08.463016 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 08:18:08.463016 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 08:18:08.463016 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 08:18:08.463016 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 08:18:08.463016 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 21 08:18:08.463016 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 21 08:18:08.463016 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 21 08:18:08.463016 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 21 08:18:08.463016 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 08:18:08.463016 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 08:18:08.463016 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 08:18:08.463016 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 08:18:08.463016 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 08:18:08.463016 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 08:18:08.463902 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 08:18:08.463902 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application with Lock( Jul 21 08:18:08.463902 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 08:18:08.463902 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application return self._enter() Jul 21 08:18:08.463902 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 08:18:08.463902 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 08:18:08.463902 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 08:18:08.463902 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application return self.creator() Jul 21 08:18:08.463902 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 08:18:08.463902 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application created_value = creator( Jul 21 08:18:08.463902 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 21 08:18:08.463902 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 21 08:18:08.463902 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Jul 21 08:18:08.463902 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application Jul 21 08:18:08.465288 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: [pid: 80245|app: 0|req: 525/1048] 10.222.0.47 () {56 vars in 1105 bytes} [Thu Jul 21 08:18:08 2022] GET /identity/v3/services/image => generated 86 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 08:18:08.469180 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4c8f252-3669-41d6-9041-7506b05d5683 None None] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:18:08.471607 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG dbcounter [-] [80244] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80244) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:18:08.576586 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4c8f252-3669-41d6-9041-7506b05d5683 None None] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:18:08.611387 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90d8071e-9c9e-472a-a290-63f69515dec9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:18:08.612616 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90d8071e-9c9e-472a-a290-63f69515dec9 None admin] REQUEST_METHOD: `GET` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:18:08.612899 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90d8071e-9c9e-472a-a290-63f69515dec9 None admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:18:08.613175 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90d8071e-9c9e-472a-a290-63f69515dec9 None admin] PATH_INFO: `/v3/services` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:18:08.613883 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90d8071e-9c9e-472a-a290-63f69515dec9 None admin] RBAC: Adding query filter params (name=image) {{(pid=80244) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 08:18:08.614189 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90d8071e-9c9e-472a-a290-63f69515dec9 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:18:08.615582 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90d8071e-9c9e-472a-a290-63f69515dec9 None admin] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:18:08.624980 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 524/1049] 10.222.0.47 () {56 vars in 1108 bytes} [Thu Jul 21 08:18:08 2022] GET /identity/v3/services?name=image => generated 122 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 08:18:08.626172 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3c8e62d-9eab-406c-afc3-6521ac407f11 None admin] Authenticating user token {{(pid=80245) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:18:08.730150 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3c8e62d-9eab-406c-afc3-6521ac407f11 None admin] Validating token access rules against request {{(pid=80245) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:18:08.763956 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2395fdc-ab01-4c38-8480-b9e32cc0e5c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80245) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:18:08.764619 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2395fdc-ab01-4c38-8480-b9e32cc0e5c9 None admin] REQUEST_METHOD: `GET` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:18:08.764904 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2395fdc-ab01-4c38-8480-b9e32cc0e5c9 None admin] SCRIPT_NAME: `/identity` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:18:08.765181 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2395fdc-ab01-4c38-8480-b9e32cc0e5c9 None admin] PATH_INFO: `/v3/services` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:18:08.765878 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2395fdc-ab01-4c38-8480-b9e32cc0e5c9 None admin] RBAC: Adding query filter params (type=image) {{(pid=80245) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 08:18:08.766183 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2395fdc-ab01-4c38-8480-b9e32cc0e5c9 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:18:08.767569 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2395fdc-ab01-4c38-8480-b9e32cc0e5c9 None admin] RBAC: Authorization granted {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:18:08.776851 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: [pid: 80245|app: 0|req: 526/1050] 10.222.0.47 () {56 vars in 1108 bytes} [Thu Jul 21 08:18:08 2022] GET /identity/v3/services?type=image => generated 350 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 21 08:18:08.778260 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90d8071e-9c9e-472a-a290-63f69515dec9 None admin] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:18:08.845489 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG dbcounter [-] [80244] Writing DB stats keystone:SELECT=100 {{(pid=80244) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:18:08.882310 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90d8071e-9c9e-472a-a290-63f69515dec9 None admin] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:18:08.917373 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1338d7fe-a500-4140-9038-4246e2ecda48 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:18:08.918028 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1338d7fe-a500-4140-9038-4246e2ecda48 None admin] REQUEST_METHOD: `POST` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:18:08.918306 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1338d7fe-a500-4140-9038-4246e2ecda48 None admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:18:08.918576 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1338d7fe-a500-4140-9038-4246e2ecda48 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:18:08.919313 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1338d7fe-a500-4140-9038-4246e2ecda48 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:18:08.920718 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1338d7fe-a500-4140-9038-4246e2ecda48 None admin] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:18:08.941315 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 525/1051] 10.222.0.47 () {60 vars in 1144 bytes} [Thu Jul 21 08:18:08 2022] POST /identity/v3/endpoints => generated 327 bytes in 164 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 21 08:18:09.387279 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e9f154f-0017-42ca-b2f7-3995ea088fdf None None] REQUEST_METHOD: `GET` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:18:09.387827 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e9f154f-0017-42ca-b2f7-3995ea088fdf None None] SCRIPT_NAME: `/identity` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:18:09.387827 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e9f154f-0017-42ca-b2f7-3995ea088fdf None None] PATH_INFO: `/` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:18:09.389244 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: [pid: 80245|app: 0|req: 527/1052] 10.222.0.47 () {54 vars in 926 bytes} [Thu Jul 21 08:18:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 08:18:09.394409 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45f08917-75a4-488e-b7de-f7638b6f2c22 None None] REQUEST_METHOD: `POST` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:18:09.394699 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45f08917-75a4-488e-b7de-f7638b6f2c22 None None] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:18:09.394976 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45f08917-75a4-488e-b7de-f7638b6f2c22 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:18:09.411427 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.auth.core [None req-45f08917-75a4-488e-b7de-f7638b6f2c22 None None] MFA Rules not processed for user `2dce40b8d74c41ec94cc36e6ff05d412`. Rule list: `[]` (Enabled: `True`). {{(pid=80244) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 08:18:09.412599 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.fernet_utils [None req-45f08917-75a4-488e-b7de-f7638b6f2c22 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80244) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 08:18:09.491399 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 526/1053] 10.222.0.47 () {58 vars in 1023 bytes} [Thu Jul 21 08:18:09 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 21 08:18:09.511302 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e9f154f-0017-42ca-b2f7-3995ea088fdf None None] Authenticating user token {{(pid=80245) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:18:09.571850 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG dbcounter [-] [80245] Writing DB stats keystone:SELECT=100 {{(pid=80245) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:18:09.582523 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37153d46-c956-427c-a172-db5193b2a012 None None] REQUEST_METHOD: `GET` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:18:09.582861 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37153d46-c956-427c-a172-db5193b2a012 None None] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:18:09.583088 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37153d46-c956-427c-a172-db5193b2a012 None None] PATH_INFO: `/` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:18:09.584670 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 527/1054] 10.222.0.47 () {54 vars in 926 bytes} [Thu Jul 21 08:18:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 08:18:09.589973 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dddb8a49-afee-4d8e-acf9-de0b7d63f27b None None] REQUEST_METHOD: `POST` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:18:09.590264 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dddb8a49-afee-4d8e-acf9-de0b7d63f27b None None] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:18:09.590548 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dddb8a49-afee-4d8e-acf9-de0b7d63f27b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:18:09.608125 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.auth.core [None req-dddb8a49-afee-4d8e-acf9-de0b7d63f27b None None] MFA Rules not processed for user `2dce40b8d74c41ec94cc36e6ff05d412`. Rule list: `[]` (Enabled: `True`). {{(pid=80244) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 08:18:09.609318 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.fernet_utils [None req-dddb8a49-afee-4d8e-acf9-de0b7d63f27b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80244) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 08:18:09.621968 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e9f154f-0017-42ca-b2f7-3995ea088fdf None None] Validating token access rules against request {{(pid=80245) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:18:09.655152 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG dbcounter [-] [80244] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80244) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:18:09.658063 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd8b7d60-e2e9-42d9-a736-3856806670a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80245) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:18:09.658927 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd8b7d60-e2e9-42d9-a736-3856806670a9 None admin] REQUEST_METHOD: `GET` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:18:09.659203 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd8b7d60-e2e9-42d9-a736-3856806670a9 None admin] SCRIPT_NAME: `/identity` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:18:09.659503 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd8b7d60-e2e9-42d9-a736-3856806670a9 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:18:09.671007 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd8b7d60-e2e9-42d9-a736-3856806670a9 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 21 08:18:09.679650 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application [None req-cd8b7d60-e2e9-42d9-a736-3856806670a9 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 21 08:18:09.679650 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 08:18:09.679650 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 08:18:09.679650 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 08:18:09.679650 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 08:18:09.679650 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 08:18:09.679650 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 08:18:09.679650 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 08:18:09.679650 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 08:18:09.679650 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 08:18:09.679650 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 08:18:09.679650 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 08:18:09.679650 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 21 08:18:09.679650 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 21 08:18:09.679650 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 21 08:18:09.679650 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 21 08:18:09.680761 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 21 08:18:09.680761 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 21 08:18:09.680761 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 21 08:18:09.680761 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 21 08:18:09.680761 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 08:18:09.680761 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 08:18:09.680761 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 08:18:09.680761 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 08:18:09.680761 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 08:18:09.680761 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 08:18:09.680761 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 08:18:09.680761 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application with Lock( Jul 21 08:18:09.680761 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 08:18:09.680761 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application return self._enter() Jul 21 08:18:09.680761 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 08:18:09.680761 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 08:18:09.681597 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 08:18:09.681597 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application return self.creator() Jul 21 08:18:09.681597 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 08:18:09.681597 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application created_value = creator( Jul 21 08:18:09.681597 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 21 08:18:09.681597 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 21 08:18:09.681597 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 21 08:18:09.681597 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 21 08:18:09.681597 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 21 08:18:09.681597 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 21 08:18:09.681597 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 21 08:18:09.681597 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application Jul 21 08:18:09.684562 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 528/1055] 10.222.0.47 () {58 vars in 1023 bytes} [Thu Jul 21 08:18:09 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 21 08:18:09.685882 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: [pid: 80245|app: 0|req: 528/1056] 10.222.0.47 () {56 vars in 1120 bytes} [Thu Jul 21 08:18:09 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 08:18:09.688298 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dddb8a49-afee-4d8e-acf9-de0b7d63f27b None None] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:18:09.708403 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd8b7d60-e2e9-42d9-a736-3856806670a9 None admin] Authenticating user token {{(pid=80245) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:18:09.794748 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dddb8a49-afee-4d8e-acf9-de0b7d63f27b None None] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:18:09.816152 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd8b7d60-e2e9-42d9-a736-3856806670a9 None admin] Validating token access rules against request {{(pid=80245) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:18:09.830428 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e49eed5d-6bad-4a8a-9ec7-2f9c7befcd6d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:18:09.831109 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e49eed5d-6bad-4a8a-9ec7-2f9c7befcd6d None admin] REQUEST_METHOD: `GET` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:18:09.831424 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e49eed5d-6bad-4a8a-9ec7-2f9c7befcd6d None admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:18:09.831706 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e49eed5d-6bad-4a8a-9ec7-2f9c7befcd6d None admin] PATH_INFO: `/v3/roles` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:18:09.832460 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e49eed5d-6bad-4a8a-9ec7-2f9c7befcd6d None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80244) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 08:18:09.832768 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e49eed5d-6bad-4a8a-9ec7-2f9c7befcd6d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:18:09.834142 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e49eed5d-6bad-4a8a-9ec7-2f9c7befcd6d None admin] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:18:09.834474 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG dbcounter [-] [80245] Writing DB stats keystone:SELECT=100 {{(pid=80245) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:18:09.842686 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 529/1057] 10.222.0.47 () {56 vars in 1115 bytes} [Thu Jul 21 08:18:09 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 08:18:09.848947 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e49eed5d-6bad-4a8a-9ec7-2f9c7befcd6d None admin] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:18:09.861109 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58c4c056-935f-45f8-81f6-9c777710cc5e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80245) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:18:09.861973 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58c4c056-935f-45f8-81f6-9c777710cc5e None admin] REQUEST_METHOD: `GET` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:18:09.862329 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58c4c056-935f-45f8-81f6-9c777710cc5e None admin] SCRIPT_NAME: `/identity` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:18:09.862547 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58c4c056-935f-45f8-81f6-9c777710cc5e None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:18:09.874395 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58c4c056-935f-45f8-81f6-9c777710cc5e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 21 08:18:09.882765 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application [None req-58c4c056-935f-45f8-81f6-9c777710cc5e None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 21 08:18:09.882765 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 08:18:09.882765 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 08:18:09.882765 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 08:18:09.882765 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 08:18:09.882765 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 08:18:09.882765 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 08:18:09.882765 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 08:18:09.882765 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 08:18:09.882765 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 08:18:09.882765 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 08:18:09.882765 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 08:18:09.882765 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 21 08:18:09.882765 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 21 08:18:09.882765 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 21 08:18:09.882765 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 21 08:18:09.884020 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 21 08:18:09.884020 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 21 08:18:09.884020 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 21 08:18:09.884020 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 21 08:18:09.884020 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 08:18:09.884020 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 08:18:09.884020 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 08:18:09.884020 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 08:18:09.884020 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 08:18:09.884020 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 08:18:09.884020 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 08:18:09.884020 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application with Lock( Jul 21 08:18:09.884020 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 08:18:09.884020 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application return self._enter() Jul 21 08:18:09.884020 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 08:18:09.884020 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 08:18:09.884907 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 08:18:09.884907 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application return self.creator() Jul 21 08:18:09.884907 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 08:18:09.884907 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application created_value = creator( Jul 21 08:18:09.884907 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 21 08:18:09.884907 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 21 08:18:09.884907 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 21 08:18:09.884907 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 21 08:18:09.884907 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 21 08:18:09.884907 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 21 08:18:09.884907 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 21 08:18:09.884907 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application Jul 21 08:18:09.884907 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: [pid: 80245|app: 0|req: 529/1058] 10.222.0.47 () {56 vars in 1096 bytes} [Thu Jul 21 08:18:09 2022] GET /identity/v3/roles/admin => generated 83 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 21 08:18:09.895993 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58c4c056-935f-45f8-81f6-9c777710cc5e None admin] Authenticating user token {{(pid=80245) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:18:09.898586 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG dbcounter [-] [80244] Writing DB stats keystone:SELECT=100 {{(pid=80244) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:18:09.958059 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e49eed5d-6bad-4a8a-9ec7-2f9c7befcd6d None admin] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:18:09.995526 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0c7c739-fabe-437d-bf4b-ca9993f7048d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:18:09.996322 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0c7c739-fabe-437d-bf4b-ca9993f7048d None admin] REQUEST_METHOD: `GET` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:18:09.996607 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0c7c739-fabe-437d-bf4b-ca9993f7048d None admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:18:09.996938 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0c7c739-fabe-437d-bf4b-ca9993f7048d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:18:10.006198 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58c4c056-935f-45f8-81f6-9c777710cc5e None admin] Validating token access rules against request {{(pid=80245) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:18:10.008946 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0c7c739-fabe-437d-bf4b-ca9993f7048d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:18:10.010410 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0c7c739-fabe-437d-bf4b-ca9993f7048d None admin] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:18:10.027376 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application [None req-b0c7c739-fabe-437d-bf4b-ca9993f7048d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 08:18:10.027376 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 08:18:10.027376 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 08:18:10.027376 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 08:18:10.027376 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 08:18:10.027376 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 08:18:10.027376 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 08:18:10.027376 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 08:18:10.027376 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 08:18:10.027376 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 08:18:10.027376 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 08:18:10.027376 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 08:18:10.027376 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 21 08:18:10.027376 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 21 08:18:10.027376 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 21 08:18:10.028292 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 21 08:18:10.028292 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 08:18:10.028292 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 08:18:10.028292 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 08:18:10.028292 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 08:18:10.028292 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 08:18:10.028292 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 08:18:10.028292 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 08:18:10.028292 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application with Lock( Jul 21 08:18:10.028292 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 08:18:10.028292 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self._enter() Jul 21 08:18:10.028292 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 08:18:10.028292 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 08:18:10.028292 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 08:18:10.028292 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self.creator() Jul 21 08:18:10.028292 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 08:18:10.028292 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application created_value = creator( Jul 21 08:18:10.029168 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 21 08:18:10.029168 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 21 08:18:10.029168 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 08:18:10.029168 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application Jul 21 08:18:10.029168 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 530/1059] 10.222.0.47 () {56 vars in 1108 bytes} [Thu Jul 21 08:18:09 2022] GET /identity/v3/domains/Default => generated 87 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 08:18:10.035148 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0c7c739-fabe-437d-bf4b-ca9993f7048d None admin] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:18:10.045500 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9d1909f-7bd8-4c79-ab03-617d5802e5bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80245) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:18:10.046583 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9d1909f-7bd8-4c79-ab03-617d5802e5bb None admin] REQUEST_METHOD: `GET` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:18:10.046875 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9d1909f-7bd8-4c79-ab03-617d5802e5bb None admin] SCRIPT_NAME: `/identity` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:18:10.047153 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9d1909f-7bd8-4c79-ab03-617d5802e5bb None admin] PATH_INFO: `/v3/roles` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:18:10.048097 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9d1909f-7bd8-4c79-ab03-617d5802e5bb None admin] RBAC: Adding query filter params (name=admin) {{(pid=80245) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 08:18:10.048412 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9d1909f-7bd8-4c79-ab03-617d5802e5bb None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:18:10.050340 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9d1909f-7bd8-4c79-ab03-617d5802e5bb None admin] RBAC: Authorization granted {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:18:10.059628 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: [pid: 80245|app: 0|req: 530/1060] 10.222.0.47 () {56 vars in 1099 bytes} [Thu Jul 21 08:18:09 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 21 08:18:10.065764 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9d1909f-7bd8-4c79-ab03-617d5802e5bb None admin] Authenticating user token {{(pid=80245) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:18:10.107581 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG dbcounter [-] [80245] Writing DB stats keystone:SELECT=100 {{(pid=80245) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:18:10.138212 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG dbcounter [-] [80244] Writing DB stats keystone:SELECT=100 {{(pid=80244) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:18:10.148765 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0c7c739-fabe-437d-bf4b-ca9993f7048d None admin] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:18:10.175013 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9d1909f-7bd8-4c79-ab03-617d5802e5bb None admin] Validating token access rules against request {{(pid=80245) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:18:10.183645 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-881a5144-f41d-4a25-9054-abb61023428c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:18:10.184314 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-881a5144-f41d-4a25-9054-abb61023428c None admin] REQUEST_METHOD: `GET` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:18:10.184603 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-881a5144-f41d-4a25-9054-abb61023428c None admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:18:10.184882 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-881a5144-f41d-4a25-9054-abb61023428c None admin] PATH_INFO: `/v3/domains` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:18:10.185614 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-881a5144-f41d-4a25-9054-abb61023428c None admin] RBAC: Adding query filter params (name=Default) {{(pid=80244) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 08:18:10.185929 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-881a5144-f41d-4a25-9054-abb61023428c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:18:10.187293 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-881a5144-f41d-4a25-9054-abb61023428c None admin] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:18:10.199523 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 531/1061] 10.222.0.47 () {56 vars in 1109 bytes} [Thu Jul 21 08:18:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 21 08:18:10.205316 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-881a5144-f41d-4a25-9054-abb61023428c None admin] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:18:10.218710 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-425162ec-07fc-43d9-a905-92266db8274f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80245) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:18:10.219600 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-425162ec-07fc-43d9-a905-92266db8274f None admin] REQUEST_METHOD: `GET` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:18:10.219837 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-425162ec-07fc-43d9-a905-92266db8274f None admin] SCRIPT_NAME: `/identity` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:18:10.220193 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-425162ec-07fc-43d9-a905-92266db8274f None admin] PATH_INFO: `/v3/users/20ce84c6ca164b98ab3d21058729f19f` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:18:10.235172 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-425162ec-07fc-43d9-a905-92266db8274f None admin] RBAC: Authorizing `identity:get_user(user_id=20ce84c6ca164b98ab3d21058729f19f)` {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:18:10.237216 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-425162ec-07fc-43d9-a905-92266db8274f None admin] RBAC: Authorization granted {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:18:10.238945 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: [pid: 80245|app: 0|req: 531/1062] 10.222.0.47 () {56 vars in 1177 bytes} [Thu Jul 21 08:18:10 2022] GET /identity/v3/users/20ce84c6ca164b98ab3d21058729f19f => generated 285 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 21 08:18:10.245106 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-425162ec-07fc-43d9-a905-92266db8274f None admin] Authenticating user token {{(pid=80245) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:18:10.325936 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-881a5144-f41d-4a25-9054-abb61023428c None admin] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:18:10.348755 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG dbcounter [-] [80245] Writing DB stats keystone:SELECT=100 {{(pid=80245) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:18:10.355956 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-425162ec-07fc-43d9-a905-92266db8274f None admin] Validating token access rules against request {{(pid=80245) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:18:10.364284 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e50ff467-5976-4fa8-bbcd-4fb247529ca1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:18:10.365088 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e50ff467-5976-4fa8-bbcd-4fb247529ca1 None admin] REQUEST_METHOD: `GET` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:18:10.365376 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e50ff467-5976-4fa8-bbcd-4fb247529ca1 None admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:18:10.365657 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e50ff467-5976-4fa8-bbcd-4fb247529ca1 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:18:10.373360 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e50ff467-5976-4fa8-bbcd-4fb247529ca1 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:18:10.374907 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e50ff467-5976-4fa8-bbcd-4fb247529ca1 None admin] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:18:10.390654 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application [None req-e50ff467-5976-4fa8-bbcd-4fb247529ca1 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jul 21 08:18:10.390654 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 08:18:10.390654 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 08:18:10.390654 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 08:18:10.390654 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 08:18:10.390654 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 08:18:10.390654 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 08:18:10.390654 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 08:18:10.390654 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 08:18:10.390654 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 08:18:10.390654 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 08:18:10.390654 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 08:18:10.390654 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 21 08:18:10.390654 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 21 08:18:10.390654 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 21 08:18:10.390654 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 21 08:18:10.391644 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 08:18:10.391644 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 08:18:10.391644 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 21 08:18:10.391644 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 21 08:18:10.391644 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 21 08:18:10.391644 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 21 08:18:10.391644 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 08:18:10.391644 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 08:18:10.391644 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 08:18:10.391644 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 08:18:10.391644 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 08:18:10.391644 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application with Lock( Jul 21 08:18:10.391644 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 08:18:10.391644 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self._enter() Jul 21 08:18:10.391644 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 08:18:10.391644 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 08:18:10.391644 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 08:18:10.391644 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self.creator() Jul 21 08:18:10.392549 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 08:18:10.392549 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application created_value = creator( Jul 21 08:18:10.392549 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 21 08:18:10.392549 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 21 08:18:10.392549 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 21 08:18:10.392549 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 21 08:18:10.392549 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 21 08:18:10.392549 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 21 08:18:10.392549 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jul 21 08:18:10.392549 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application Jul 21 08:18:10.395338 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-435dec79-84b6-4559-b9c5-411823e35eae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80245) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:18:10.396004 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-435dec79-84b6-4559-b9c5-411823e35eae None admin] REQUEST_METHOD: `GET` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:18:10.396292 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-435dec79-84b6-4559-b9c5-411823e35eae None admin] SCRIPT_NAME: `/identity` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:18:10.396867 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-435dec79-84b6-4559-b9c5-411823e35eae None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:18:10.397030 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 532/1063] 10.222.0.47 () {56 vars in 1093 bytes} [Thu Jul 21 08:18:10 2022] GET /identity/v3/users/nova => generated 82 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 21 08:18:10.397859 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e50ff467-5976-4fa8-bbcd-4fb247529ca1 None admin] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:18:10.412605 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG dbcounter [-] [80244] Writing DB stats keystone:SELECT=100 {{(pid=80244) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:18:10.477702 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-435dec79-84b6-4559-b9c5-411823e35eae None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:18:10.479159 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-435dec79-84b6-4559-b9c5-411823e35eae None admin] RBAC: Authorization granted {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:18:10.509789 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e50ff467-5976-4fa8-bbcd-4fb247529ca1 None admin] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:18:10.547739 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-848a5d5e-50a5-4df1-922e-add3119abb07 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:18:10.548696 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-848a5d5e-50a5-4df1-922e-add3119abb07 None admin] REQUEST_METHOD: `GET` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:18:10.549732 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-848a5d5e-50a5-4df1-922e-add3119abb07 None admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:18:10.550158 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-848a5d5e-50a5-4df1-922e-add3119abb07 None admin] PATH_INFO: `/v3/users` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:18:10.551315 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-848a5d5e-50a5-4df1-922e-add3119abb07 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80244) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 08:18:10.551643 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-848a5d5e-50a5-4df1-922e-add3119abb07 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:18:10.553084 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-848a5d5e-50a5-4df1-922e-add3119abb07 None admin] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:18:10.576118 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 533/1064] 10.222.0.47 () {56 vars in 1133 bytes} [Thu Jul 21 08:18:10 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 08:18:10.577458 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-848a5d5e-50a5-4df1-922e-add3119abb07 None admin] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:18:10.584593 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG dbcounter [-] [80245] Writing DB stats keystone:SELECT=100 {{(pid=80245) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:18:10.598934 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: [pid: 80245|app: 0|req: 532/1065] 10.222.0.47 () {58 vars in 1282 bytes} [Thu Jul 21 08:18:10 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 349 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 21 08:18:10.600024 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-435dec79-84b6-4559-b9c5-411823e35eae None admin] Authenticating user token {{(pid=80245) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:18:10.658340 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG dbcounter [-] [80244] Writing DB stats keystone:SELECT=100 {{(pid=80244) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:18:10.692336 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-848a5d5e-50a5-4df1-922e-add3119abb07 None admin] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:18:10.708185 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-435dec79-84b6-4559-b9c5-411823e35eae None admin] Validating token access rules against request {{(pid=80245) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:18:10.729165 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b09df743-b800-4748-b18a-8faa6a1add03 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:18:10.730345 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b09df743-b800-4748-b18a-8faa6a1add03 None admin] REQUEST_METHOD: `GET` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:18:10.730631 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b09df743-b800-4748-b18a-8faa6a1add03 None admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:18:10.731061 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b09df743-b800-4748-b18a-8faa6a1add03 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:18:10.741992 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b09df743-b800-4748-b18a-8faa6a1add03 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:18:10.743405 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b09df743-b800-4748-b18a-8faa6a1add03 None admin] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:18:10.746690 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1965444-8e52-4820-8b42-8348e7b72a33 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80245) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:18:10.747830 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1965444-8e52-4820-8b42-8348e7b72a33 None admin] REQUEST_METHOD: `GET` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:18:10.748297 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1965444-8e52-4820-8b42-8348e7b72a33 None admin] SCRIPT_NAME: `/identity` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:18:10.748749 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1965444-8e52-4820-8b42-8348e7b72a33 None admin] PATH_INFO: `/v3/projects/057e4c8ef4634149b372875de3e7f3ef` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:18:10.750134 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1965444-8e52-4820-8b42-8348e7b72a33 None admin] RBAC: Authorizing `identity:get_project(project_id=057e4c8ef4634149b372875de3e7f3ef)` {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:18:10.752344 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1965444-8e52-4820-8b42-8348e7b72a33 None admin] RBAC: Authorization granted {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:18:10.755375 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: [pid: 80245|app: 0|req: 533/1066] 10.222.0.47 () {56 vars in 1186 bytes} [Thu Jul 21 08:18:10 2022] GET /identity/v3/projects/057e4c8ef4634149b372875de3e7f3ef => generated 311 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 21 08:18:10.761342 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1965444-8e52-4820-8b42-8348e7b72a33 None admin] Authenticating user token {{(pid=80245) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:18:10.762692 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application [None req-b09df743-b800-4748-b18a-8faa6a1add03 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 08:18:10.762692 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 08:18:10.762692 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 08:18:10.762692 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 08:18:10.762692 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 08:18:10.762692 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 08:18:10.762692 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 08:18:10.762692 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 08:18:10.762692 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 08:18:10.762692 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 08:18:10.762692 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 08:18:10.762692 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 08:18:10.762692 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 21 08:18:10.762692 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 21 08:18:10.762692 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 21 08:18:10.763603 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 21 08:18:10.763603 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 08:18:10.763603 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 08:18:10.763603 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 08:18:10.763603 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 08:18:10.763603 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 08:18:10.763603 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 08:18:10.763603 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 08:18:10.763603 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application with Lock( Jul 21 08:18:10.763603 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 08:18:10.763603 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self._enter() Jul 21 08:18:10.763603 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 08:18:10.763603 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 08:18:10.763603 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 08:18:10.763603 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self.creator() Jul 21 08:18:10.763603 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 08:18:10.763603 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application created_value = creator( Jul 21 08:18:10.764473 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 21 08:18:10.764473 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 21 08:18:10.764473 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 08:18:10.764473 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application Jul 21 08:18:10.764473 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 534/1067] 10.222.0.47 () {56 vars in 1108 bytes} [Thu Jul 21 08:18:10 2022] GET /identity/v3/domains/Default => generated 87 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 21 08:18:10.772026 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b09df743-b800-4748-b18a-8faa6a1add03 None admin] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:18:10.836515 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG dbcounter [-] [80245] Writing DB stats keystone:SELECT=100 {{(pid=80245) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:18:10.876504 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b09df743-b800-4748-b18a-8faa6a1add03 None admin] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:18:10.877800 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1965444-8e52-4820-8b42-8348e7b72a33 None admin] Validating token access rules against request {{(pid=80245) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:18:10.901784 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG dbcounter [-] [80244] Writing DB stats keystone:SELECT=100 {{(pid=80244) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:18:10.916736 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cca71308-bf6c-4335-b4b8-789ec35e1caa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:18:10.917408 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cca71308-bf6c-4335-b4b8-789ec35e1caa None admin] REQUEST_METHOD: `GET` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:18:10.917696 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cca71308-bf6c-4335-b4b8-789ec35e1caa None admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:18:10.917975 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cca71308-bf6c-4335-b4b8-789ec35e1caa None admin] PATH_INFO: `/v3/domains` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:18:10.918700 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cca71308-bf6c-4335-b4b8-789ec35e1caa None admin] RBAC: Adding query filter params (name=Default) {{(pid=80244) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 08:18:10.919010 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cca71308-bf6c-4335-b4b8-789ec35e1caa None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:18:10.920420 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cca71308-bf6c-4335-b4b8-789ec35e1caa None admin] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:18:10.922136 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f306cf8-92c6-47dc-b230-9e65993ac84e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80245) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:18:10.922800 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f306cf8-92c6-47dc-b230-9e65993ac84e None admin] REQUEST_METHOD: `GET` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:18:10.923078 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f306cf8-92c6-47dc-b230-9e65993ac84e None admin] SCRIPT_NAME: `/identity` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:18:10.923374 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f306cf8-92c6-47dc-b230-9e65993ac84e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:18:10.924222 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f306cf8-92c6-47dc-b230-9e65993ac84e None admin] RBAC: Adding query filter params (role.id=e29870f6e9e2411a9db0d5bf45f32ca5, scope.project.id=057e4c8ef4634149b372875de3e7f3ef, user.id=20ce84c6ca164b98ab3d21058729f19f) {{(pid=80245) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 08:18:10.924530 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f306cf8-92c6-47dc-b230-9e65993ac84e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:18:10.925929 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f306cf8-92c6-47dc-b230-9e65993ac84e None admin] RBAC: Authorization granted {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:18:10.931644 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: [pid: 80245|app: 0|req: 534/1068] 10.222.0.47 () {56 vars in 1374 bytes} [Thu Jul 21 08:18:10 2022] GET /identity/v3/role_assignments?user.id=20ce84c6ca164b98ab3d21058729f19f&scope.project.id=057e4c8ef4634149b372875de3e7f3ef&role.id=e29870f6e9e2411a9db0d5bf45f32ca5 => generated 259 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 21 08:18:10.937088 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 535/1069] 10.222.0.47 () {56 vars in 1109 bytes} [Thu Jul 21 08:18:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 08:18:10.940531 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f306cf8-92c6-47dc-b230-9e65993ac84e None admin] Authenticating user token {{(pid=80245) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:18:11.000015 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bf58e7f-4a98-47e3-9dc4-e486ae6d06c3 None None] REQUEST_METHOD: `GET` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:18:11.000269 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bf58e7f-4a98-47e3-9dc4-e486ae6d06c3 None None] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:18:11.000504 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bf58e7f-4a98-47e3-9dc4-e486ae6d06c3 None None] PATH_INFO: `/` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:18:11.002258 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 536/1070] 10.222.0.47 () {54 vars in 926 bytes} [Thu Jul 21 08:18:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 08:18:11.007739 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c5bd37f-1b3a-44bc-9c1c-17cced149817 None None] REQUEST_METHOD: `POST` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:18:11.008063 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c5bd37f-1b3a-44bc-9c1c-17cced149817 None None] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:18:11.008562 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c5bd37f-1b3a-44bc-9c1c-17cced149817 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:18:11.026582 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.auth.core [None req-2c5bd37f-1b3a-44bc-9c1c-17cced149817 None None] MFA Rules not processed for user `2dce40b8d74c41ec94cc36e6ff05d412`. Rule list: `[]` (Enabled: `True`). {{(pid=80244) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 08:18:11.027850 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.fernet_utils [None req-2c5bd37f-1b3a-44bc-9c1c-17cced149817 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80244) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 08:18:11.058537 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f306cf8-92c6-47dc-b230-9e65993ac84e None admin] Validating token access rules against request {{(pid=80245) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:18:11.092574 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG dbcounter [-] [80245] Writing DB stats keystone:SELECT=100 {{(pid=80245) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:18:11.100984 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 537/1071] 10.222.0.47 () {58 vars in 1023 bytes} [Thu Jul 21 08:18:11 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 21 08:18:11.101227 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-688b321a-6198-47f1-a609-26eba37d0a57 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80245) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:18:11.102164 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-688b321a-6198-47f1-a609-26eba37d0a57 None admin] REQUEST_METHOD: `GET` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:18:11.102451 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-688b321a-6198-47f1-a609-26eba37d0a57 None admin] SCRIPT_NAME: `/identity` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:18:11.102730 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-688b321a-6198-47f1-a609-26eba37d0a57 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:18:11.110093 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-688b321a-6198-47f1-a609-26eba37d0a57 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:18:11.111600 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-688b321a-6198-47f1-a609-26eba37d0a57 None admin] RBAC: Authorization granted {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:18:11.122629 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c5bd37f-1b3a-44bc-9c1c-17cced149817 None None] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:18:11.125528 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application [None req-688b321a-6198-47f1-a609-26eba37d0a57 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 21 08:18:11.125528 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 08:18:11.125528 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 08:18:11.125528 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 08:18:11.125528 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 08:18:11.125528 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 08:18:11.125528 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 08:18:11.125528 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 08:18:11.125528 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 08:18:11.125528 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 08:18:11.125528 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 08:18:11.125528 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 08:18:11.125528 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 21 08:18:11.125528 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 21 08:18:11.125528 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 21 08:18:11.126402 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 21 08:18:11.126402 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 08:18:11.126402 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 08:18:11.126402 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 08:18:11.126402 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 08:18:11.126402 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 08:18:11.126402 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 08:18:11.126402 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 08:18:11.126402 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application with Lock( Jul 21 08:18:11.126402 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 08:18:11.126402 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application return self._enter() Jul 21 08:18:11.126402 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 08:18:11.126402 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 08:18:11.126402 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 08:18:11.126402 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application return self.creator() Jul 21 08:18:11.126402 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 08:18:11.126402 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application created_value = creator( Jul 21 08:18:11.127355 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 21 08:18:11.127355 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 21 08:18:11.127355 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 21 08:18:11.127355 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 21 08:18:11.127355 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 21 08:18:11.127355 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 21 08:18:11.127355 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 21 08:18:11.127355 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application Jul 21 08:18:11.131685 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: [pid: 80245|app: 0|req: 535/1072] 10.222.0.47 () {56 vars in 1111 bytes} [Thu Jul 21 08:18:10 2022] GET /identity/v3/projects/service => generated 88 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 08:18:11.132331 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-688b321a-6198-47f1-a609-26eba37d0a57 None admin] Authenticating user token {{(pid=80245) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:18:11.227919 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c5bd37f-1b3a-44bc-9c1c-17cced149817 None None] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:18:11.243662 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG dbcounter [-] [80244] Writing DB stats keystone:SELECT=100 {{(pid=80244) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:18:11.247418 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-688b321a-6198-47f1-a609-26eba37d0a57 None admin] Validating token access rules against request {{(pid=80245) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:18:11.266844 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-914e6319-d842-4f50-b6b9-61c5026ed5ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:18:11.267753 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-914e6319-d842-4f50-b6b9-61c5026ed5ea None admin] REQUEST_METHOD: `GET` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:18:11.268665 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-914e6319-d842-4f50-b6b9-61c5026ed5ea None admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:18:11.268950 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-914e6319-d842-4f50-b6b9-61c5026ed5ea None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:18:11.285091 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffe423ba-4870-4ea4-bf1d-850f112da7d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80245) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:18:11.285910 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffe423ba-4870-4ea4-bf1d-850f112da7d3 None admin] REQUEST_METHOD: `GET` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:18:11.286020 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffe423ba-4870-4ea4-bf1d-850f112da7d3 None admin] SCRIPT_NAME: `/identity` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:18:11.286297 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffe423ba-4870-4ea4-bf1d-850f112da7d3 None admin] PATH_INFO: `/v3/projects` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:18:11.287110 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffe423ba-4870-4ea4-bf1d-850f112da7d3 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80245) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 08:18:11.287540 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffe423ba-4870-4ea4-bf1d-850f112da7d3 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:18:11.288867 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffe423ba-4870-4ea4-bf1d-850f112da7d3 None admin] RBAC: Authorization granted {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:18:11.303037 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: [pid: 80245|app: 0|req: 536/1073] 10.222.0.47 () {56 vars in 1148 bytes} [Thu Jul 21 08:18:11 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 08:18:11.310481 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffe423ba-4870-4ea4-bf1d-850f112da7d3 None admin] Authenticating user token {{(pid=80245) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:18:11.357970 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-914e6319-d842-4f50-b6b9-61c5026ed5ea None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:18:11.359501 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-914e6319-d842-4f50-b6b9-61c5026ed5ea None admin] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:18:11.369937 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG dbcounter [-] [80245] Writing DB stats keystone:SELECT=100 {{(pid=80245) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:18:11.426394 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffe423ba-4870-4ea4-bf1d-850f112da7d3 None admin] Validating token access rules against request {{(pid=80245) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:18:11.467547 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46148d67-6e9f-48fd-bf9a-91f8cd1cffdc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80245) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:18:11.468163 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 538/1074] 10.222.0.47 () {58 vars in 1282 bytes} [Thu Jul 21 08:18:11 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 346 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Jul 21 08:18:11.473392 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46148d67-6e9f-48fd-bf9a-91f8cd1cffdc None admin] REQUEST_METHOD: `PUT` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:18:11.473392 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46148d67-6e9f-48fd-bf9a-91f8cd1cffdc None admin] SCRIPT_NAME: `/identity` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:18:11.473392 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46148d67-6e9f-48fd-bf9a-91f8cd1cffdc None admin] PATH_INFO: `/v3/projects/4faef13aa50e4e36b6c265fb1398fa14/users/ac2b5b99a36442148fe743269054e991/roles/4b8dff36c32b49618b303451de1d1f69` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:18:11.474134 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-914e6319-d842-4f50-b6b9-61c5026ed5ea None admin] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:18:11.478384 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46148d67-6e9f-48fd-bf9a-91f8cd1cffdc None admin] RBAC: Authorizing `identity:create_grant(project_id=4faef13aa50e4e36b6c265fb1398fa14, user_id=ac2b5b99a36442148fe743269054e991, role_id=4b8dff36c32b49618b303451de1d1f69)` {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:18:11.479873 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46148d67-6e9f-48fd-bf9a-91f8cd1cffdc None admin] RBAC: Authorization granted {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:18:11.489844 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG dbcounter [-] [80244] Writing DB stats keystone:SELECT=100 {{(pid=80244) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:18:11.497888 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: [pid: 80245|app: 0|req: 537/1075] 10.222.0.47 () {58 vars in 1439 bytes} [Thu Jul 21 08:18:11 2022] PUT /identity/v3/projects/4faef13aa50e4e36b6c265fb1398fa14/users/ac2b5b99a36442148fe743269054e991/roles/4b8dff36c32b49618b303451de1d1f69 => generated 0 bytes in 189 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 21 08:18:11.599033 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-914e6319-d842-4f50-b6b9-61c5026ed5ea None admin] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:18:11.637187 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f432cede-6b56-46bb-9c54-f0cbd42904a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:18:11.638005 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f432cede-6b56-46bb-9c54-f0cbd42904a0 None admin] REQUEST_METHOD: `GET` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:18:11.638291 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f432cede-6b56-46bb-9c54-f0cbd42904a0 None admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:18:11.638570 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f432cede-6b56-46bb-9c54-f0cbd42904a0 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:18:11.653206 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f432cede-6b56-46bb-9c54-f0cbd42904a0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:18:11.654690 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f432cede-6b56-46bb-9c54-f0cbd42904a0 None admin] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:18:11.673914 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application [None req-f432cede-6b56-46bb-9c54-f0cbd42904a0 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 08:18:11.673914 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 08:18:11.673914 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 08:18:11.673914 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 08:18:11.673914 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 08:18:11.673914 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 08:18:11.673914 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 08:18:11.673914 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 08:18:11.673914 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 08:18:11.673914 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 08:18:11.673914 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 08:18:11.673914 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 08:18:11.673914 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 21 08:18:11.673914 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 21 08:18:11.673914 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 21 08:18:11.675609 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 21 08:18:11.675609 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 08:18:11.675609 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 08:18:11.675609 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 08:18:11.675609 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 08:18:11.675609 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 08:18:11.675609 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 08:18:11.675609 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 08:18:11.675609 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application with Lock( Jul 21 08:18:11.675609 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 08:18:11.675609 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self._enter() Jul 21 08:18:11.675609 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 08:18:11.675609 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 08:18:11.675609 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 08:18:11.675609 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self.creator() Jul 21 08:18:11.675609 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 08:18:11.675609 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application created_value = creator( Jul 21 08:18:11.676808 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 21 08:18:11.676808 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 21 08:18:11.676808 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 08:18:11.676808 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application Jul 21 08:18:11.676808 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 539/1076] 10.222.0.47 () {56 vars in 1108 bytes} [Thu Jul 21 08:18:11 2022] GET /identity/v3/domains/Default => generated 87 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 08:18:11.681908 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46148d67-6e9f-48fd-bf9a-91f8cd1cffdc None admin] Authenticating user token {{(pid=80245) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:18:11.781998 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46148d67-6e9f-48fd-bf9a-91f8cd1cffdc None admin] Validating token access rules against request {{(pid=80245) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:18:11.791297 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG dbcounter [-] [80245] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80245) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:18:11.819782 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-834db6ec-86ac-4a83-8fb3-7d3e93ced21d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80245) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:18:11.820693 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-834db6ec-86ac-4a83-8fb3-7d3e93ced21d None admin] REQUEST_METHOD: `GET` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:18:11.821145 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-834db6ec-86ac-4a83-8fb3-7d3e93ced21d None admin] SCRIPT_NAME: `/identity` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:18:11.821583 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-834db6ec-86ac-4a83-8fb3-7d3e93ced21d None admin] PATH_INFO: `/v3/domains` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:18:11.822453 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-834db6ec-86ac-4a83-8fb3-7d3e93ced21d None admin] RBAC: Adding query filter params (name=Default) {{(pid=80245) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 08:18:11.822905 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-834db6ec-86ac-4a83-8fb3-7d3e93ced21d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:18:11.824452 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-834db6ec-86ac-4a83-8fb3-7d3e93ced21d None admin] RBAC: Authorization granted {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:18:11.836897 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: [pid: 80245|app: 0|req: 538/1077] 10.222.0.47 () {56 vars in 1109 bytes} [Thu Jul 21 08:18:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 08:18:12.962692 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bd7f06b-cb98-4537-9545-94aca00c8ec5 None None] REQUEST_METHOD: `GET` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:18:12.963213 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bd7f06b-cb98-4537-9545-94aca00c8ec5 None None] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:18:12.963213 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bd7f06b-cb98-4537-9545-94aca00c8ec5 None None] PATH_INFO: `/` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:18:12.964621 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 540/1078] 10.222.0.47 () {54 vars in 926 bytes} [Thu Jul 21 08:18:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 08:18:12.969784 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e480ac0-5382-4df0-9df8-d05fcc89435c None None] REQUEST_METHOD: `POST` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:18:12.970230 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e480ac0-5382-4df0-9df8-d05fcc89435c None None] SCRIPT_NAME: `/identity` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:18:12.970347 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e480ac0-5382-4df0-9df8-d05fcc89435c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:18:12.986749 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.auth.core [None req-5e480ac0-5382-4df0-9df8-d05fcc89435c None None] MFA Rules not processed for user `2dce40b8d74c41ec94cc36e6ff05d412`. Rule list: `[]` (Enabled: `True`). {{(pid=80245) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 08:18:12.987960 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.fernet_utils [None req-5e480ac0-5382-4df0-9df8-d05fcc89435c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80245) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 08:18:13.056308 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: [pid: 80245|app: 0|req: 539/1079] 10.222.0.47 () {58 vars in 1023 bytes} [Thu Jul 21 08:18:12 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 21 08:18:13.078983 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bd7f06b-cb98-4537-9545-94aca00c8ec5 None None] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:18:13.142969 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG dbcounter [-] [80244] Writing DB stats keystone:SELECT=100 {{(pid=80244) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:18:13.181550 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bd7f06b-cb98-4537-9545-94aca00c8ec5 None None] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:18:13.215028 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef4220db-cf7e-4d0b-aca7-28ca8650e22e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:18:13.215770 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef4220db-cf7e-4d0b-aca7-28ca8650e22e None admin] REQUEST_METHOD: `GET` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:18:13.216056 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef4220db-cf7e-4d0b-aca7-28ca8650e22e None admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:18:13.216334 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef4220db-cf7e-4d0b-aca7-28ca8650e22e None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:18:13.226380 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef4220db-cf7e-4d0b-aca7-28ca8650e22e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 21 08:18:13.234071 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application [None req-ef4220db-cf7e-4d0b-aca7-28ca8650e22e None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 21 08:18:13.234071 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 08:18:13.234071 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 08:18:13.234071 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 08:18:13.234071 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 08:18:13.234071 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 08:18:13.234071 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 08:18:13.234071 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 08:18:13.234071 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 08:18:13.234071 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 08:18:13.234071 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 08:18:13.234071 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 08:18:13.234071 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 21 08:18:13.234071 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 21 08:18:13.234071 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 21 08:18:13.234071 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 21 08:18:13.235169 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 21 08:18:13.235169 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 21 08:18:13.235169 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 21 08:18:13.235169 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 21 08:18:13.235169 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 08:18:13.235169 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 08:18:13.235169 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 08:18:13.235169 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 08:18:13.235169 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 08:18:13.235169 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 08:18:13.235169 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 08:18:13.235169 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application with Lock( Jul 21 08:18:13.235169 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 08:18:13.235169 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self._enter() Jul 21 08:18:13.235169 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 08:18:13.235169 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 08:18:13.237056 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 08:18:13.237056 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self.creator() Jul 21 08:18:13.237056 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 08:18:13.237056 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application created_value = creator( Jul 21 08:18:13.237056 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 21 08:18:13.237056 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 21 08:18:13.237056 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 21 08:18:13.237056 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 21 08:18:13.237056 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 21 08:18:13.237056 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 21 08:18:13.237056 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 21 08:18:13.237056 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application Jul 21 08:18:13.237056 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 541/1080] 10.222.0.47 () {56 vars in 1096 bytes} [Thu Jul 21 08:18:13 2022] GET /identity/v3/roles/admin => generated 83 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 08:18:13.242022 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e480ac0-5382-4df0-9df8-d05fcc89435c None None] Authenticating user token {{(pid=80245) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:18:13.350137 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG dbcounter [-] [80245] Writing DB stats keystone:SELECT=100 {{(pid=80245) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:18:13.356902 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e480ac0-5382-4df0-9df8-d05fcc89435c None None] Validating token access rules against request {{(pid=80245) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:18:13.413338 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76942f23-41a8-4c95-bf1b-00228ee8281c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80245) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:18:13.414462 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76942f23-41a8-4c95-bf1b-00228ee8281c None admin] REQUEST_METHOD: `GET` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:18:13.414960 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76942f23-41a8-4c95-bf1b-00228ee8281c None admin] SCRIPT_NAME: `/identity` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:18:13.415493 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76942f23-41a8-4c95-bf1b-00228ee8281c None admin] PATH_INFO: `/v3/roles` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:18:13.416928 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76942f23-41a8-4c95-bf1b-00228ee8281c None admin] RBAC: Adding query filter params (name=admin) {{(pid=80245) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 08:18:13.417488 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76942f23-41a8-4c95-bf1b-00228ee8281c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:18:13.420079 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76942f23-41a8-4c95-bf1b-00228ee8281c None admin] RBAC: Authorization granted {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:18:13.434610 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: [pid: 80245|app: 0|req: 540/1081] 10.222.0.47 () {56 vars in 1099 bytes} [Thu Jul 21 08:18:13 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 08:18:13.440946 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef4220db-cf7e-4d0b-aca7-28ca8650e22e None admin] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:18:13.543494 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef4220db-cf7e-4d0b-aca7-28ca8650e22e None admin] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:18:13.567099 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG dbcounter [-] [80244] Writing DB stats keystone:SELECT=100 {{(pid=80244) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:18:13.578309 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2def245-bbd5-42de-b023-411f0bd1b8ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:18:13.579174 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2def245-bbd5-42de-b023-411f0bd1b8ab None admin] REQUEST_METHOD: `GET` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:18:13.579482 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2def245-bbd5-42de-b023-411f0bd1b8ab None admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:18:13.579752 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2def245-bbd5-42de-b023-411f0bd1b8ab None admin] PATH_INFO: `/v3/users/20ce84c6ca164b98ab3d21058729f19f` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:18:13.580981 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2def245-bbd5-42de-b023-411f0bd1b8ab None admin] RBAC: Authorizing `identity:get_user(user_id=20ce84c6ca164b98ab3d21058729f19f)` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:18:13.582406 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2def245-bbd5-42de-b023-411f0bd1b8ab None admin] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:18:13.584070 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 542/1082] 10.222.0.47 () {56 vars in 1177 bytes} [Thu Jul 21 08:18:13 2022] GET /identity/v3/users/20ce84c6ca164b98ab3d21058729f19f => generated 285 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 08:18:13.590115 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76942f23-41a8-4c95-bf1b-00228ee8281c None admin] Authenticating user token {{(pid=80245) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:18:13.609753 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-692061ac-48dc-4055-8810-f6f8d16863c8 None None] REQUEST_METHOD: `GET` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:18:13.610041 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-692061ac-48dc-4055-8810-f6f8d16863c8 None None] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:18:13.610324 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-692061ac-48dc-4055-8810-f6f8d16863c8 None None] PATH_INFO: `/` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:18:13.612269 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 543/1083] 10.222.0.47 () {54 vars in 926 bytes} [Thu Jul 21 08:18:13 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 21 08:18:13.617320 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c978cd1d-1826-4e9a-863c-934829dfa07d None None] REQUEST_METHOD: `POST` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:18:13.617598 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c978cd1d-1826-4e9a-863c-934829dfa07d None None] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:18:13.617868 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c978cd1d-1826-4e9a-863c-934829dfa07d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:18:13.634310 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.auth.core [None req-c978cd1d-1826-4e9a-863c-934829dfa07d None None] MFA Rules not processed for user `2dce40b8d74c41ec94cc36e6ff05d412`. Rule list: `[]` (Enabled: `True`). {{(pid=80244) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 08:18:13.635524 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.fernet_utils [None req-c978cd1d-1826-4e9a-863c-934829dfa07d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80244) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 08:18:13.699792 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76942f23-41a8-4c95-bf1b-00228ee8281c None admin] Validating token access rules against request {{(pid=80245) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:18:13.706495 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 544/1084] 10.222.0.47 () {58 vars in 1023 bytes} [Thu Jul 21 08:18:13 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 21 08:18:13.730633 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c978cd1d-1826-4e9a-863c-934829dfa07d None None] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:18:13.732850 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a74bda9-7eb6-474e-a8a2-9f6b9166b781 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80245) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:18:13.733717 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a74bda9-7eb6-474e-a8a2-9f6b9166b781 None admin] REQUEST_METHOD: `GET` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:18:13.733996 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a74bda9-7eb6-474e-a8a2-9f6b9166b781 None admin] SCRIPT_NAME: `/identity` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:18:13.734266 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a74bda9-7eb6-474e-a8a2-9f6b9166b781 None admin] PATH_INFO: `/v3/projects/057e4c8ef4634149b372875de3e7f3ef` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:18:13.735467 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a74bda9-7eb6-474e-a8a2-9f6b9166b781 None admin] RBAC: Authorizing `identity:get_project(project_id=057e4c8ef4634149b372875de3e7f3ef)` {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:18:13.737556 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a74bda9-7eb6-474e-a8a2-9f6b9166b781 None admin] RBAC: Authorization granted {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:18:13.739195 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: [pid: 80245|app: 0|req: 541/1085] 10.222.0.47 () {56 vars in 1186 bytes} [Thu Jul 21 08:18:13 2022] GET /identity/v3/projects/057e4c8ef4634149b372875de3e7f3ef => generated 311 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 08:18:13.745066 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a74bda9-7eb6-474e-a8a2-9f6b9166b781 None admin] Authenticating user token {{(pid=80245) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:18:13.772621 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG dbcounter [-] [80245] Writing DB stats keystone:SELECT=100 {{(pid=80245) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:18:13.835175 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c978cd1d-1826-4e9a-863c-934829dfa07d None None] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:18:13.843923 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a74bda9-7eb6-474e-a8a2-9f6b9166b781 None admin] Validating token access rules against request {{(pid=80245) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:18:13.857718 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG dbcounter [-] [80244] Writing DB stats keystone:SELECT=100 {{(pid=80244) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:18:13.870278 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ee2700a-12b0-4f8b-8dec-dc8bfa57b18f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:18:13.871094 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ee2700a-12b0-4f8b-8dec-dc8bfa57b18f None admin] REQUEST_METHOD: `GET` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:18:13.871406 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ee2700a-12b0-4f8b-8dec-dc8bfa57b18f None admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:18:13.871690 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ee2700a-12b0-4f8b-8dec-dc8bfa57b18f None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:18:13.878492 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b379ceb-22ba-4c3f-99e0-ddc4249d0f00 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80245) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:18:13.879406 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b379ceb-22ba-4c3f-99e0-ddc4249d0f00 None admin] REQUEST_METHOD: `PUT` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:18:13.879687 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b379ceb-22ba-4c3f-99e0-ddc4249d0f00 None admin] SCRIPT_NAME: `/identity` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:18:13.879960 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b379ceb-22ba-4c3f-99e0-ddc4249d0f00 None admin] PATH_INFO: `/v3/projects/057e4c8ef4634149b372875de3e7f3ef/users/20ce84c6ca164b98ab3d21058729f19f/roles/e29870f6e9e2411a9db0d5bf45f32ca5` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:18:13.881712 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ee2700a-12b0-4f8b-8dec-dc8bfa57b18f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 21 08:18:13.881934 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b379ceb-22ba-4c3f-99e0-ddc4249d0f00 None admin] RBAC: Authorizing `identity:create_grant(project_id=057e4c8ef4634149b372875de3e7f3ef, user_id=20ce84c6ca164b98ab3d21058729f19f, role_id=e29870f6e9e2411a9db0d5bf45f32ca5)` {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:18:13.883627 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b379ceb-22ba-4c3f-99e0-ddc4249d0f00 None admin] RBAC: Authorization granted {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:18:13.890512 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application [None req-1ee2700a-12b0-4f8b-8dec-dc8bfa57b18f None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 21 08:18:13.890512 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 08:18:13.890512 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 08:18:13.890512 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 08:18:13.890512 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 08:18:13.890512 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 08:18:13.890512 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 08:18:13.890512 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 08:18:13.890512 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 08:18:13.890512 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 08:18:13.890512 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 08:18:13.890512 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 08:18:13.890512 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 21 08:18:13.890512 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 21 08:18:13.890512 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 21 08:18:13.890512 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 21 08:18:13.891659 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 21 08:18:13.891659 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 21 08:18:13.891659 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 21 08:18:13.891659 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 21 08:18:13.891659 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 08:18:13.891659 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 08:18:13.891659 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 08:18:13.891659 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 08:18:13.891659 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 08:18:13.891659 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 08:18:13.891659 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 08:18:13.891659 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application with Lock( Jul 21 08:18:13.891659 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 08:18:13.891659 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self._enter() Jul 21 08:18:13.891659 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 08:18:13.891659 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 08:18:13.892569 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 08:18:13.892569 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self.creator() Jul 21 08:18:13.892569 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 08:18:13.892569 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application created_value = creator( Jul 21 08:18:13.892569 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 21 08:18:13.892569 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 21 08:18:13.892569 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 21 08:18:13.892569 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 21 08:18:13.892569 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 21 08:18:13.892569 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 21 08:18:13.892569 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 21 08:18:13.892569 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application Jul 21 08:18:13.892569 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 545/1086] 10.222.0.47 () {56 vars in 1120 bytes} [Thu Jul 21 08:18:13 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 21 08:18:13.899159 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ee2700a-12b0-4f8b-8dec-dc8bfa57b18f None admin] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:18:13.900135 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: [pid: 80245|app: 0|req: 542/1087] 10.222.0.47 () {58 vars in 1439 bytes} [Thu Jul 21 08:18:13 2022] PUT /identity/v3/projects/057e4c8ef4634149b372875de3e7f3ef/users/20ce84c6ca164b98ab3d21058729f19f/roles/e29870f6e9e2411a9db0d5bf45f32ca5 => generated 0 bytes in 155 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 21 08:18:14.009612 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ee2700a-12b0-4f8b-8dec-dc8bfa57b18f None admin] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:18:14.045993 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f8e756c-353e-4a17-b5ac-634f4227fe0e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:18:14.046677 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f8e756c-353e-4a17-b5ac-634f4227fe0e None admin] REQUEST_METHOD: `GET` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:18:14.046965 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f8e756c-353e-4a17-b5ac-634f4227fe0e None admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:18:14.047245 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f8e756c-353e-4a17-b5ac-634f4227fe0e None admin] PATH_INFO: `/v3/roles` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:18:14.048037 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f8e756c-353e-4a17-b5ac-634f4227fe0e None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80244) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 08:18:14.048346 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f8e756c-353e-4a17-b5ac-634f4227fe0e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:18:14.049815 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f8e756c-353e-4a17-b5ac-634f4227fe0e None admin] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:18:14.059528 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 546/1088] 10.222.0.47 () {56 vars in 1115 bytes} [Thu Jul 21 08:18:13 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 21 08:18:14.065655 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b379ceb-22ba-4c3f-99e0-ddc4249d0f00 None admin] Authenticating user token {{(pid=80245) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:18:14.155010 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG dbcounter [-] [80245] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80245) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:18:14.169228 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b379ceb-22ba-4c3f-99e0-ddc4249d0f00 None admin] Validating token access rules against request {{(pid=80245) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:18:14.202082 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc2fbadf-243b-4bd9-a9b3-e04cad46a1b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80245) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:18:14.202878 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc2fbadf-243b-4bd9-a9b3-e04cad46a1b1 None admin] REQUEST_METHOD: `GET` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:18:14.203164 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc2fbadf-243b-4bd9-a9b3-e04cad46a1b1 None admin] SCRIPT_NAME: `/identity` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:18:14.203477 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc2fbadf-243b-4bd9-a9b3-e04cad46a1b1 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:18:14.214972 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc2fbadf-243b-4bd9-a9b3-e04cad46a1b1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:18:14.216459 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc2fbadf-243b-4bd9-a9b3-e04cad46a1b1 None admin] RBAC: Authorization granted {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:18:14.233653 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application [None req-fc2fbadf-243b-4bd9-a9b3-e04cad46a1b1 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 08:18:14.233653 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 08:18:14.233653 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 08:18:14.233653 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 08:18:14.233653 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 08:18:14.233653 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 08:18:14.233653 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 08:18:14.233653 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 08:18:14.233653 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 08:18:14.233653 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 08:18:14.233653 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 08:18:14.233653 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 08:18:14.233653 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 21 08:18:14.233653 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 21 08:18:14.233653 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 21 08:18:14.234719 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 21 08:18:14.234719 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 08:18:14.234719 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 08:18:14.234719 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 08:18:14.234719 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 08:18:14.234719 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 08:18:14.234719 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 08:18:14.234719 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 08:18:14.234719 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application with Lock( Jul 21 08:18:14.234719 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 08:18:14.234719 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application return self._enter() Jul 21 08:18:14.234719 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 08:18:14.234719 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 08:18:14.234719 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 08:18:14.234719 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application return self.creator() Jul 21 08:18:14.234719 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 08:18:14.234719 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application created_value = creator( Jul 21 08:18:14.236035 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 21 08:18:14.236035 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 21 08:18:14.236035 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 08:18:14.236035 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application Jul 21 08:18:14.236035 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: [pid: 80245|app: 0|req: 543/1089] 10.222.0.47 () {56 vars in 1108 bytes} [Thu Jul 21 08:18:14 2022] GET /identity/v3/domains/Default => generated 87 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 08:18:14.241420 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f8e756c-353e-4a17-b5ac-634f4227fe0e None admin] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:18:14.288185 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG dbcounter [-] [80244] Writing DB stats keystone:SELECT=100 {{(pid=80244) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:18:14.344480 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f8e756c-353e-4a17-b5ac-634f4227fe0e None admin] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:18:14.380196 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fb2e88b-3a61-4cf1-acaf-c3c79f3d4b0e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:18:14.380866 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fb2e88b-3a61-4cf1-acaf-c3c79f3d4b0e None admin] REQUEST_METHOD: `GET` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:18:14.381153 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fb2e88b-3a61-4cf1-acaf-c3c79f3d4b0e None admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:18:14.381438 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fb2e88b-3a61-4cf1-acaf-c3c79f3d4b0e None admin] PATH_INFO: `/v3/domains` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:18:14.382154 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fb2e88b-3a61-4cf1-acaf-c3c79f3d4b0e None admin] RBAC: Adding query filter params (name=Default) {{(pid=80244) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 08:18:14.382460 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fb2e88b-3a61-4cf1-acaf-c3c79f3d4b0e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:18:14.384158 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fb2e88b-3a61-4cf1-acaf-c3c79f3d4b0e None admin] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:18:14.396642 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 547/1090] 10.222.0.47 () {56 vars in 1109 bytes} [Thu Jul 21 08:18:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 08:18:14.402222 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc2fbadf-243b-4bd9-a9b3-e04cad46a1b1 None admin] Authenticating user token {{(pid=80245) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:18:14.505419 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc2fbadf-243b-4bd9-a9b3-e04cad46a1b1 None admin] Validating token access rules against request {{(pid=80245) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:18:14.539046 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d1fed34-3b7f-4fd8-adcd-d833af04e3a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80245) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:18:14.539795 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d1fed34-3b7f-4fd8-adcd-d833af04e3a7 None admin] REQUEST_METHOD: `GET` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:18:14.540076 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d1fed34-3b7f-4fd8-adcd-d833af04e3a7 None admin] SCRIPT_NAME: `/identity` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:18:14.540349 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d1fed34-3b7f-4fd8-adcd-d833af04e3a7 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:18:14.548772 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d1fed34-3b7f-4fd8-adcd-d833af04e3a7 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:18:14.550200 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d1fed34-3b7f-4fd8-adcd-d833af04e3a7 None admin] RBAC: Authorization granted {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:18:14.552752 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG dbcounter [-] [80245] Writing DB stats keystone:SELECT=100 {{(pid=80245) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:18:14.565542 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application [None req-0d1fed34-3b7f-4fd8-adcd-d833af04e3a7 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jul 21 08:18:14.565542 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 08:18:14.565542 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 08:18:14.565542 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 08:18:14.565542 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 08:18:14.565542 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 08:18:14.565542 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 08:18:14.565542 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 08:18:14.565542 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 08:18:14.565542 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 08:18:14.565542 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 08:18:14.565542 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 08:18:14.565542 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 21 08:18:14.565542 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 21 08:18:14.565542 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 21 08:18:14.565542 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 21 08:18:14.566659 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 08:18:14.566659 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 08:18:14.566659 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 21 08:18:14.566659 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 21 08:18:14.566659 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 21 08:18:14.566659 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 21 08:18:14.566659 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 08:18:14.566659 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 08:18:14.566659 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 08:18:14.566659 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 08:18:14.566659 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 08:18:14.566659 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application with Lock( Jul 21 08:18:14.566659 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 08:18:14.566659 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application return self._enter() Jul 21 08:18:14.566659 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 08:18:14.566659 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 08:18:14.566659 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 08:18:14.566659 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application return self.creator() Jul 21 08:18:14.568188 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 08:18:14.568188 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application created_value = creator( Jul 21 08:18:14.568188 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 21 08:18:14.568188 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 21 08:18:14.568188 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 21 08:18:14.568188 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 21 08:18:14.568188 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 21 08:18:14.568188 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 21 08:18:14.568188 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jul 21 08:18:14.568188 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application Jul 21 08:18:14.568188 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: [pid: 80245|app: 0|req: 544/1091] 10.222.0.47 () {56 vars in 1093 bytes} [Thu Jul 21 08:18:14 2022] GET /identity/v3/users/nova => generated 82 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 21 08:18:14.572483 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fb2e88b-3a61-4cf1-acaf-c3c79f3d4b0e None admin] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:18:14.670931 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fb2e88b-3a61-4cf1-acaf-c3c79f3d4b0e None admin] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:18:14.673234 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG dbcounter [-] [80244] Writing DB stats keystone:SELECT=100 {{(pid=80244) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:18:14.705442 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67828489-60af-445f-912d-eb63e0a1bb1a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:18:14.706107 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67828489-60af-445f-912d-eb63e0a1bb1a None admin] REQUEST_METHOD: `GET` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:18:14.706391 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67828489-60af-445f-912d-eb63e0a1bb1a None admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:18:14.706665 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67828489-60af-445f-912d-eb63e0a1bb1a None admin] PATH_INFO: `/v3/users` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:18:14.707529 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67828489-60af-445f-912d-eb63e0a1bb1a None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80244) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 08:18:14.707838 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67828489-60af-445f-912d-eb63e0a1bb1a None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:18:14.709269 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67828489-60af-445f-912d-eb63e0a1bb1a None admin] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:18:14.726652 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 548/1092] 10.222.0.47 () {56 vars in 1133 bytes} [Thu Jul 21 08:18:14 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 08:18:14.732361 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d1fed34-3b7f-4fd8-adcd-d833af04e3a7 None admin] Authenticating user token {{(pid=80245) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:18:14.828990 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d1fed34-3b7f-4fd8-adcd-d833af04e3a7 None admin] Validating token access rules against request {{(pid=80245) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:18:14.860463 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a960d1db-8845-47c2-98bf-a831911c7c23 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80245) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:18:14.861112 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a960d1db-8845-47c2-98bf-a831911c7c23 None admin] REQUEST_METHOD: `GET` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:18:14.861392 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a960d1db-8845-47c2-98bf-a831911c7c23 None admin] SCRIPT_NAME: `/identity` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:18:14.862191 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a960d1db-8845-47c2-98bf-a831911c7c23 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:18:14.925397 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG dbcounter [-] [80245] Writing DB stats keystone:SELECT=100 {{(pid=80245) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:18:14.928155 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a960d1db-8845-47c2-98bf-a831911c7c23 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:18:14.930157 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a960d1db-8845-47c2-98bf-a831911c7c23 None admin] RBAC: Authorization granted {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:18:15.030992 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: [pid: 80245|app: 0|req: 545/1093] 10.222.0.47 () {58 vars in 1282 bytes} [Thu Jul 21 08:18:14 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 299 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Jul 21 08:18:15.037497 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67828489-60af-445f-912d-eb63e0a1bb1a None admin] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:18:15.135442 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67828489-60af-445f-912d-eb63e0a1bb1a None admin] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:18:15.169453 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6626bbac-e554-43ce-8230-e87451360197 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:18:15.170242 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6626bbac-e554-43ce-8230-e87451360197 None admin] REQUEST_METHOD: `GET` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:18:15.170519 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6626bbac-e554-43ce-8230-e87451360197 None admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:18:15.170797 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6626bbac-e554-43ce-8230-e87451360197 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:18:15.181392 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6626bbac-e554-43ce-8230-e87451360197 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:18:15.182833 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6626bbac-e554-43ce-8230-e87451360197 None admin] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:18:15.199749 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application [None req-6626bbac-e554-43ce-8230-e87451360197 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 08:18:15.199749 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 08:18:15.199749 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 08:18:15.199749 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 08:18:15.199749 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 08:18:15.199749 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 08:18:15.199749 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 08:18:15.199749 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 08:18:15.199749 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 08:18:15.199749 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 08:18:15.199749 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 08:18:15.199749 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 08:18:15.199749 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 21 08:18:15.199749 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 21 08:18:15.199749 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 21 08:18:15.200857 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 21 08:18:15.200857 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 08:18:15.200857 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 08:18:15.200857 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 08:18:15.200857 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 08:18:15.200857 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 08:18:15.200857 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 08:18:15.200857 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 08:18:15.200857 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application with Lock( Jul 21 08:18:15.200857 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 08:18:15.200857 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self._enter() Jul 21 08:18:15.200857 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 08:18:15.200857 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 08:18:15.200857 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 08:18:15.200857 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self.creator() Jul 21 08:18:15.200857 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 08:18:15.200857 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application created_value = creator( Jul 21 08:18:15.206744 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 21 08:18:15.206744 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 21 08:18:15.206744 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 08:18:15.206744 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application Jul 21 08:18:15.206744 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 549/1094] 10.222.0.47 () {56 vars in 1108 bytes} [Thu Jul 21 08:18:15 2022] GET /identity/v3/domains/Default => generated 87 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 08:18:15.207033 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a960d1db-8845-47c2-98bf-a831911c7c23 None admin] Authenticating user token {{(pid=80245) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:18:15.301304 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG dbcounter [-] [80245] Writing DB stats keystone:SELECT=100 {{(pid=80245) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:18:15.304304 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a960d1db-8845-47c2-98bf-a831911c7c23 None admin] Validating token access rules against request {{(pid=80245) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:18:15.338021 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c1556f8-16fb-488b-961e-a52e7b73d781 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80245) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:18:15.338692 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c1556f8-16fb-488b-961e-a52e7b73d781 None admin] REQUEST_METHOD: `GET` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:18:15.338973 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c1556f8-16fb-488b-961e-a52e7b73d781 None admin] SCRIPT_NAME: `/identity` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:18:15.339253 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c1556f8-16fb-488b-961e-a52e7b73d781 None admin] PATH_INFO: `/v3/domains` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:18:15.340010 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c1556f8-16fb-488b-961e-a52e7b73d781 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80245) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 08:18:15.340311 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c1556f8-16fb-488b-961e-a52e7b73d781 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:18:15.341682 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c1556f8-16fb-488b-961e-a52e7b73d781 None admin] RBAC: Authorization granted {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:18:15.354525 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: [pid: 80245|app: 0|req: 546/1095] 10.222.0.47 () {56 vars in 1109 bytes} [Thu Jul 21 08:18:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 08:18:15.360649 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6626bbac-e554-43ce-8230-e87451360197 None admin] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:18:15.378362 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG dbcounter [-] [80244] Writing DB stats keystone:SELECT=100 {{(pid=80244) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:18:15.463726 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6626bbac-e554-43ce-8230-e87451360197 None admin] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:18:15.499034 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f65f951-fc09-4ca6-a3d5-53f032f677e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:18:15.499862 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f65f951-fc09-4ca6-a3d5-53f032f677e4 None admin] REQUEST_METHOD: `GET` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:18:15.500144 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f65f951-fc09-4ca6-a3d5-53f032f677e4 None admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:18:15.500417 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f65f951-fc09-4ca6-a3d5-53f032f677e4 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:18:15.507102 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f65f951-fc09-4ca6-a3d5-53f032f677e4 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:18:15.508822 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f65f951-fc09-4ca6-a3d5-53f032f677e4 None admin] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:18:15.522789 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application [None req-9f65f951-fc09-4ca6-a3d5-53f032f677e4 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 21 08:18:15.522789 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 08:18:15.522789 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 08:18:15.522789 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 08:18:15.522789 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 08:18:15.522789 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 08:18:15.522789 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 08:18:15.522789 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 08:18:15.522789 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 08:18:15.522789 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 08:18:15.522789 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 08:18:15.522789 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 08:18:15.522789 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 21 08:18:15.522789 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 21 08:18:15.522789 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 21 08:18:15.523963 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 21 08:18:15.523963 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 08:18:15.523963 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 08:18:15.523963 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 08:18:15.523963 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 08:18:15.523963 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 08:18:15.523963 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 08:18:15.523963 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 08:18:15.523963 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application with Lock( Jul 21 08:18:15.523963 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 08:18:15.523963 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self._enter() Jul 21 08:18:15.523963 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 08:18:15.523963 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 08:18:15.523963 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 08:18:15.523963 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self.creator() Jul 21 08:18:15.523963 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 08:18:15.523963 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application created_value = creator( Jul 21 08:18:15.524862 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 21 08:18:15.524862 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 21 08:18:15.524862 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 21 08:18:15.524862 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 21 08:18:15.524862 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 21 08:18:15.524862 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 21 08:18:15.524862 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 21 08:18:15.524862 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application Jul 21 08:18:15.524862 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 550/1096] 10.222.0.47 () {56 vars in 1111 bytes} [Thu Jul 21 08:18:15 2022] GET /identity/v3/projects/service => generated 88 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 08:18:15.530127 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c1556f8-16fb-488b-961e-a52e7b73d781 None admin] Authenticating user token {{(pid=80245) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:18:15.629278 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c1556f8-16fb-488b-961e-a52e7b73d781 None admin] Validating token access rules against request {{(pid=80245) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:18:15.663605 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a68b122-e632-47f2-b8e3-03bbe64e0e54 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80245) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:18:15.664225 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a68b122-e632-47f2-b8e3-03bbe64e0e54 None admin] REQUEST_METHOD: `GET` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:18:15.664507 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a68b122-e632-47f2-b8e3-03bbe64e0e54 None admin] SCRIPT_NAME: `/identity` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:18:15.664781 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a68b122-e632-47f2-b8e3-03bbe64e0e54 None admin] PATH_INFO: `/v3/projects` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:18:15.665551 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a68b122-e632-47f2-b8e3-03bbe64e0e54 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80245) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 08:18:15.665866 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a68b122-e632-47f2-b8e3-03bbe64e0e54 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:18:15.667272 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a68b122-e632-47f2-b8e3-03bbe64e0e54 None admin] RBAC: Authorization granted {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:18:15.679195 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: [pid: 80245|app: 0|req: 547/1097] 10.222.0.47 () {56 vars in 1148 bytes} [Thu Jul 21 08:18:15 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 08:18:15.684983 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f65f951-fc09-4ca6-a3d5-53f032f677e4 None admin] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:18:15.759695 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG dbcounter [-] [80244] Writing DB stats keystone:SELECT=100 {{(pid=80244) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:18:15.785405 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f65f951-fc09-4ca6-a3d5-53f032f677e4 None admin] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:18:15.818973 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b98db28b-a7a5-4c82-ba70-0c7c453f0b5b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:18:15.819646 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b98db28b-a7a5-4c82-ba70-0c7c453f0b5b None admin] REQUEST_METHOD: `GET` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:18:15.819927 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b98db28b-a7a5-4c82-ba70-0c7c453f0b5b None admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:18:15.820196 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b98db28b-a7a5-4c82-ba70-0c7c453f0b5b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:18:15.821019 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b98db28b-a7a5-4c82-ba70-0c7c453f0b5b None admin] RBAC: Adding query filter params (role.id=4b8dff36c32b49618b303451de1d1f69, scope.project.id=4faef13aa50e4e36b6c265fb1398fa14, user.id=ac2b5b99a36442148fe743269054e991) {{(pid=80244) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 08:18:15.821319 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b98db28b-a7a5-4c82-ba70-0c7c453f0b5b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:18:15.822744 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b98db28b-a7a5-4c82-ba70-0c7c453f0b5b None admin] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:18:15.828552 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 551/1098] 10.222.0.47 () {56 vars in 1374 bytes} [Thu Jul 21 08:18:15 2022] GET /identity/v3/role_assignments?user.id=ac2b5b99a36442148fe743269054e991&scope.project.id=4faef13aa50e4e36b6c265fb1398fa14&role.id=4b8dff36c32b49618b303451de1d1f69 => generated 608 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 08:18:15.945779 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-977c3e29-195f-4485-8621-c38cc18b5dc0 None None] REQUEST_METHOD: `GET` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:18:15.946112 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-977c3e29-195f-4485-8621-c38cc18b5dc0 None None] SCRIPT_NAME: `/identity` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:18:15.946397 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-977c3e29-195f-4485-8621-c38cc18b5dc0 None None] PATH_INFO: `/` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:18:15.947838 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: [pid: 80245|app: 0|req: 548/1099] 10.222.0.47 () {54 vars in 926 bytes} [Thu Jul 21 08:18:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 08:18:15.953467 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b04d0517-15bb-4ccf-b7d5-e59511e84a48 None None] REQUEST_METHOD: `POST` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:18:15.953866 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b04d0517-15bb-4ccf-b7d5-e59511e84a48 None None] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:18:15.954153 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b04d0517-15bb-4ccf-b7d5-e59511e84a48 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:18:15.972298 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.auth.core [None req-b04d0517-15bb-4ccf-b7d5-e59511e84a48 None None] MFA Rules not processed for user `2dce40b8d74c41ec94cc36e6ff05d412`. Rule list: `[]` (Enabled: `True`). {{(pid=80244) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 08:18:15.973513 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.fernet_utils [None req-b04d0517-15bb-4ccf-b7d5-e59511e84a48 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80244) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 08:18:16.047841 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 552/1100] 10.222.0.47 () {58 vars in 1023 bytes} [Thu Jul 21 08:18:15 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 21 08:18:16.071691 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-977c3e29-195f-4485-8621-c38cc18b5dc0 None None] Authenticating user token {{(pid=80245) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:18:16.094387 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG dbcounter [-] [80245] Writing DB stats keystone:SELECT=100 {{(pid=80245) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:18:16.178612 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-977c3e29-195f-4485-8621-c38cc18b5dc0 None None] Validating token access rules against request {{(pid=80245) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:18:16.213639 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a0bb045-fbde-4ede-b8c9-0368b7d5aaf8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80245) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:18:16.214449 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a0bb045-fbde-4ede-b8c9-0368b7d5aaf8 None admin] REQUEST_METHOD: `GET` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:18:16.214733 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a0bb045-fbde-4ede-b8c9-0368b7d5aaf8 None admin] SCRIPT_NAME: `/identity` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:18:16.215010 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a0bb045-fbde-4ede-b8c9-0368b7d5aaf8 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:18:16.225480 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a0bb045-fbde-4ede-b8c9-0368b7d5aaf8 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 21 08:18:16.233186 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application [None req-2a0bb045-fbde-4ede-b8c9-0368b7d5aaf8 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 21 08:18:16.233186 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 08:18:16.233186 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 08:18:16.233186 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 08:18:16.233186 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 08:18:16.233186 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 08:18:16.233186 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 08:18:16.233186 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 08:18:16.233186 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 08:18:16.233186 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 08:18:16.233186 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 08:18:16.233186 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 08:18:16.233186 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 21 08:18:16.233186 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 21 08:18:16.233186 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 21 08:18:16.233186 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 21 08:18:16.234360 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 21 08:18:16.234360 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 21 08:18:16.234360 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 21 08:18:16.234360 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 21 08:18:16.234360 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 08:18:16.234360 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 08:18:16.234360 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 08:18:16.234360 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 08:18:16.234360 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 08:18:16.234360 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 08:18:16.234360 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 08:18:16.234360 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application with Lock( Jul 21 08:18:16.234360 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 08:18:16.234360 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application return self._enter() Jul 21 08:18:16.234360 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 08:18:16.234360 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 08:18:16.235671 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 08:18:16.235671 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application return self.creator() Jul 21 08:18:16.235671 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 08:18:16.235671 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application created_value = creator( Jul 21 08:18:16.235671 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 21 08:18:16.235671 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 21 08:18:16.235671 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 21 08:18:16.235671 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 21 08:18:16.235671 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 21 08:18:16.235671 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 21 08:18:16.235671 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 21 08:18:16.235671 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application Jul 21 08:18:16.235671 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: [pid: 80245|app: 0|req: 549/1101] 10.222.0.47 () {56 vars in 1096 bytes} [Thu Jul 21 08:18:16 2022] GET /identity/v3/roles/admin => generated 83 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 08:18:16.241054 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b04d0517-15bb-4ccf-b7d5-e59511e84a48 None None] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:18:16.313869 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG dbcounter [-] [80244] Writing DB stats keystone:SELECT=100 {{(pid=80244) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:18:16.351138 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b04d0517-15bb-4ccf-b7d5-e59511e84a48 None None] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:18:16.390915 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc98cff8-cf5c-4aa1-aac9-bd996518d98e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:18:16.391677 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc98cff8-cf5c-4aa1-aac9-bd996518d98e None admin] REQUEST_METHOD: `GET` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:18:16.391902 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc98cff8-cf5c-4aa1-aac9-bd996518d98e None admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:18:16.392174 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc98cff8-cf5c-4aa1-aac9-bd996518d98e None admin] PATH_INFO: `/v3/roles` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:18:16.393039 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc98cff8-cf5c-4aa1-aac9-bd996518d98e None admin] RBAC: Adding query filter params (name=admin) {{(pid=80244) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 08:18:16.393340 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc98cff8-cf5c-4aa1-aac9-bd996518d98e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:18:16.394775 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc98cff8-cf5c-4aa1-aac9-bd996518d98e None admin] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:18:16.404058 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 553/1102] 10.222.0.47 () {56 vars in 1099 bytes} [Thu Jul 21 08:18:16 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 08:18:16.410043 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a0bb045-fbde-4ede-b8c9-0368b7d5aaf8 None admin] Authenticating user token {{(pid=80245) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:18:16.499388 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG dbcounter [-] [80245] Writing DB stats keystone:SELECT=100 {{(pid=80245) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:18:16.519549 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a0bb045-fbde-4ede-b8c9-0368b7d5aaf8 None admin] Validating token access rules against request {{(pid=80245) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:18:16.555121 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc622813-76b4-4e75-83f9-04c76e9d3b07 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80245) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:18:16.556056 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc622813-76b4-4e75-83f9-04c76e9d3b07 None admin] REQUEST_METHOD: `GET` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:18:16.556389 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc622813-76b4-4e75-83f9-04c76e9d3b07 None admin] SCRIPT_NAME: `/identity` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:18:16.556662 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc622813-76b4-4e75-83f9-04c76e9d3b07 None admin] PATH_INFO: `/v3/users/20ce84c6ca164b98ab3d21058729f19f` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:18:16.557883 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc622813-76b4-4e75-83f9-04c76e9d3b07 None admin] RBAC: Authorizing `identity:get_user(user_id=20ce84c6ca164b98ab3d21058729f19f)` {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:18:16.559305 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc622813-76b4-4e75-83f9-04c76e9d3b07 None admin] RBAC: Authorization granted {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:18:16.560972 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: [pid: 80245|app: 0|req: 550/1103] 10.222.0.47 () {56 vars in 1177 bytes} [Thu Jul 21 08:18:16 2022] GET /identity/v3/users/20ce84c6ca164b98ab3d21058729f19f => generated 285 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 08:18:16.567043 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc98cff8-cf5c-4aa1-aac9-bd996518d98e None admin] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:18:16.670553 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc98cff8-cf5c-4aa1-aac9-bd996518d98e None admin] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:18:16.701004 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG dbcounter [-] [80244] Writing DB stats keystone:SELECT=100 {{(pid=80244) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:18:16.708676 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db3ac212-a968-4cdb-9a5c-5c2b7c81b3d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:18:16.709868 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db3ac212-a968-4cdb-9a5c-5c2b7c81b3d6 None admin] REQUEST_METHOD: `GET` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:18:16.710193 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db3ac212-a968-4cdb-9a5c-5c2b7c81b3d6 None admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:18:16.710698 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db3ac212-a968-4cdb-9a5c-5c2b7c81b3d6 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:18:16.777454 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db3ac212-a968-4cdb-9a5c-5c2b7c81b3d6 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:18:16.778899 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db3ac212-a968-4cdb-9a5c-5c2b7c81b3d6 None admin] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:18:16.874800 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 554/1104] 10.222.0.47 () {58 vars in 1282 bytes} [Thu Jul 21 08:18:16 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 308 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 21 08:18:16.880619 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc622813-76b4-4e75-83f9-04c76e9d3b07 None admin] Authenticating user token {{(pid=80245) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:18:16.977789 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc622813-76b4-4e75-83f9-04c76e9d3b07 None admin] Validating token access rules against request {{(pid=80245) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:18:17.011272 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6d81566-012c-4d0f-a460-a440ad27cfe8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80245) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:18:17.012158 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6d81566-012c-4d0f-a460-a440ad27cfe8 None admin] REQUEST_METHOD: `GET` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:18:17.012438 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6d81566-012c-4d0f-a460-a440ad27cfe8 None admin] SCRIPT_NAME: `/identity` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:18:17.012711 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6d81566-012c-4d0f-a460-a440ad27cfe8 None admin] PATH_INFO: `/v3/projects/057e4c8ef4634149b372875de3e7f3ef` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:18:17.013870 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6d81566-012c-4d0f-a460-a440ad27cfe8 None admin] RBAC: Authorizing `identity:get_project(project_id=057e4c8ef4634149b372875de3e7f3ef)` {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:18:17.015299 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6d81566-012c-4d0f-a460-a440ad27cfe8 None admin] RBAC: Authorization granted {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:18:17.017019 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: [pid: 80245|app: 0|req: 551/1105] 10.222.0.47 () {56 vars in 1186 bytes} [Thu Jul 21 08:18:16 2022] GET /identity/v3/projects/057e4c8ef4634149b372875de3e7f3ef => generated 311 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 08:18:17.023659 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db3ac212-a968-4cdb-9a5c-5c2b7c81b3d6 None admin] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:18:17.053220 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG dbcounter [-] [80244] Writing DB stats keystone:SELECT=100 {{(pid=80244) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:18:17.121864 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db3ac212-a968-4cdb-9a5c-5c2b7c81b3d6 None admin] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:18:17.155823 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a02584a2-4123-4fce-9fe5-2fb9459963b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:18:17.156478 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a02584a2-4123-4fce-9fe5-2fb9459963b4 None admin] REQUEST_METHOD: `GET` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:18:17.156761 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a02584a2-4123-4fce-9fe5-2fb9459963b4 None admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:18:17.157081 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a02584a2-4123-4fce-9fe5-2fb9459963b4 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:18:17.157888 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a02584a2-4123-4fce-9fe5-2fb9459963b4 None admin] RBAC: Adding query filter params (role.id=e29870f6e9e2411a9db0d5bf45f32ca5, scope.project.id=057e4c8ef4634149b372875de3e7f3ef, user.id=20ce84c6ca164b98ab3d21058729f19f) {{(pid=80244) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 08:18:17.158171 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a02584a2-4123-4fce-9fe5-2fb9459963b4 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:18:17.160160 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a02584a2-4123-4fce-9fe5-2fb9459963b4 None admin] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:18:17.165825 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 555/1106] 10.222.0.47 () {56 vars in 1374 bytes} [Thu Jul 21 08:18:17 2022] GET /identity/v3/role_assignments?user.id=20ce84c6ca164b98ab3d21058729f19f&scope.project.id=057e4c8ef4634149b372875de3e7f3ef&role.id=e29870f6e9e2411a9db0d5bf45f32ca5 => generated 608 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 08:18:19.215733 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10d63ae3-2c74-4dff-89f5-890398065e9c None None] REQUEST_METHOD: `GET` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:18:19.216317 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10d63ae3-2c74-4dff-89f5-890398065e9c None None] SCRIPT_NAME: `/identity` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:18:19.216317 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10d63ae3-2c74-4dff-89f5-890398065e9c None None] PATH_INFO: `/` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:18:19.217644 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: [pid: 80245|app: 0|req: 552/1107] 10.222.0.47 () {54 vars in 926 bytes} [Thu Jul 21 08:18:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 08:18:19.222762 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d009c4d2-ec35-49df-a8c3-1eeb702d047a None None] REQUEST_METHOD: `POST` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:18:19.223202 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d009c4d2-ec35-49df-a8c3-1eeb702d047a None None] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:18:19.223301 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d009c4d2-ec35-49df-a8c3-1eeb702d047a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:18:19.240847 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.auth.core [None req-d009c4d2-ec35-49df-a8c3-1eeb702d047a None None] MFA Rules not processed for user `2dce40b8d74c41ec94cc36e6ff05d412`. Rule list: `[]` (Enabled: `True`). {{(pid=80244) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 08:18:19.242027 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.fernet_utils [None req-d009c4d2-ec35-49df-a8c3-1eeb702d047a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80244) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 08:18:19.310008 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 556/1108] 10.222.0.47 () {58 vars in 1023 bytes} [Thu Jul 21 08:18:19 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 21 08:18:19.335136 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10d63ae3-2c74-4dff-89f5-890398065e9c None None] Authenticating user token {{(pid=80245) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:18:19.353337 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG dbcounter [-] [80245] Writing DB stats keystone:SELECT=100 {{(pid=80245) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:18:19.435851 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10d63ae3-2c74-4dff-89f5-890398065e9c None None] Validating token access rules against request {{(pid=80245) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:18:19.469579 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e350ea49-4088-4148-a695-7d57bcc5c670 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80245) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:18:19.470315 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e350ea49-4088-4148-a695-7d57bcc5c670 None admin] REQUEST_METHOD: `GET` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:18:19.470595 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e350ea49-4088-4148-a695-7d57bcc5c670 None admin] SCRIPT_NAME: `/identity` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:18:19.471483 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e350ea49-4088-4148-a695-7d57bcc5c670 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:18:19.472657 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e350ea49-4088-4148-a695-7d57bcc5c670 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:18:19.474311 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e350ea49-4088-4148-a695-7d57bcc5c670 None admin] RBAC: Authorization granted {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:18:19.476170 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: [pid: 80245|app: 0|req: 553/1109] 10.222.0.47 () {56 vars in 1108 bytes} [Thu Jul 21 08:18:19 2022] GET /identity/v3/domains/default => generated 199 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 08:18:19.481830 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d009c4d2-ec35-49df-a8c3-1eeb702d047a None None] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:18:19.506437 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG dbcounter [-] [80244] Writing DB stats keystone:SELECT=100 {{(pid=80244) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:18:19.579738 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d009c4d2-ec35-49df-a8c3-1eeb702d047a None None] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:18:19.612943 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4933698b-ddd9-4d8e-95b3-1d8ed77f154c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:18:19.613599 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4933698b-ddd9-4d8e-95b3-1d8ed77f154c None admin] REQUEST_METHOD: `POST` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:18:19.613879 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4933698b-ddd9-4d8e-95b3-1d8ed77f154c None admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:18:19.614152 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4933698b-ddd9-4d8e-95b3-1d8ed77f154c None admin] PATH_INFO: `/v3/users` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:18:19.614864 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4933698b-ddd9-4d8e-95b3-1d8ed77f154c None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:18:19.616310 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4933698b-ddd9-4d8e-95b3-1d8ed77f154c None admin] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:18:19.636133 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 557/1110] 10.222.0.47 () {60 vars in 1132 bytes} [Thu Jul 21 08:18:19 2022] POST /identity/v3/users => generated 286 bytes in 155 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 08:18:21.776658 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f426731f-cff6-4531-a918-5f128f5f14af None None] REQUEST_METHOD: `GET` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:18:21.777166 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f426731f-cff6-4531-a918-5f128f5f14af None None] SCRIPT_NAME: `/identity` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:18:21.777166 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f426731f-cff6-4531-a918-5f128f5f14af None None] PATH_INFO: `/` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:18:21.778489 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: [pid: 80245|app: 0|req: 554/1111] 10.222.0.47 () {54 vars in 926 bytes} [Thu Jul 21 08:18:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 08:18:21.783685 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c6c2c1c-a559-4a03-a8b0-7ecc6f3fe30e None None] REQUEST_METHOD: `POST` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:18:21.784113 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c6c2c1c-a559-4a03-a8b0-7ecc6f3fe30e None None] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:18:21.784225 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c6c2c1c-a559-4a03-a8b0-7ecc6f3fe30e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:18:21.800521 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.auth.core [None req-6c6c2c1c-a559-4a03-a8b0-7ecc6f3fe30e None None] MFA Rules not processed for user `2dce40b8d74c41ec94cc36e6ff05d412`. Rule list: `[]` (Enabled: `True`). {{(pid=80244) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 08:18:21.801689 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.fernet_utils [None req-6c6c2c1c-a559-4a03-a8b0-7ecc6f3fe30e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80244) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 08:18:21.865342 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 558/1112] 10.222.0.47 () {58 vars in 1023 bytes} [Thu Jul 21 08:18:21 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 21 08:18:21.888957 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f426731f-cff6-4531-a918-5f128f5f14af None None] Authenticating user token {{(pid=80245) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:18:21.976017 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG dbcounter [-] [80245] Writing DB stats keystone:SELECT=100 {{(pid=80245) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:18:21.992488 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f426731f-cff6-4531-a918-5f128f5f14af None None] Validating token access rules against request {{(pid=80245) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:18:22.025545 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc606aa4-cbb6-4850-8dbc-cfa37baba645 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80245) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:18:22.026389 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc606aa4-cbb6-4850-8dbc-cfa37baba645 None admin] REQUEST_METHOD: `GET` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:18:22.026719 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc606aa4-cbb6-4850-8dbc-cfa37baba645 None admin] SCRIPT_NAME: `/identity` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:18:22.026948 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc606aa4-cbb6-4850-8dbc-cfa37baba645 None admin] PATH_INFO: `/v3/roles/99a638c0aac44b79a1f2858574724495` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:18:22.028425 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc606aa4-cbb6-4850-8dbc-cfa37baba645 None admin] RBAC: Authorizing `identity:get_role(role_id=99a638c0aac44b79a1f2858574724495)` {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:18:22.029788 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc606aa4-cbb6-4850-8dbc-cfa37baba645 None admin] RBAC: Authorization granted {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:18:22.031155 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: [pid: 80245|app: 0|req: 555/1113] 10.222.0.47 () {56 vars in 1177 bytes} [Thu Jul 21 08:18:21 2022] GET /identity/v3/roles/99a638c0aac44b79a1f2858574724495 => generated 223 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 08:18:22.037084 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c6c2c1c-a559-4a03-a8b0-7ecc6f3fe30e None None] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:18:22.048457 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG dbcounter [-] [80244] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80244) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:18:22.133677 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c6c2c1c-a559-4a03-a8b0-7ecc6f3fe30e None None] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:18:22.166747 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69b8bcfd-9508-4c1e-95b9-8dde9234e52a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:18:22.167631 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69b8bcfd-9508-4c1e-95b9-8dde9234e52a None admin] REQUEST_METHOD: `GET` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:18:22.167913 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69b8bcfd-9508-4c1e-95b9-8dde9234e52a None admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:18:22.168182 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69b8bcfd-9508-4c1e-95b9-8dde9234e52a None admin] PATH_INFO: `/v3/users/4ba5d9e54b8e400bb13b0dbb208f8105` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:18:22.181213 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69b8bcfd-9508-4c1e-95b9-8dde9234e52a None admin] RBAC: Authorizing `identity:get_user(user_id=4ba5d9e54b8e400bb13b0dbb208f8105)` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:18:22.182644 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69b8bcfd-9508-4c1e-95b9-8dde9234e52a None admin] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:18:22.184293 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 559/1114] 10.222.0.47 () {56 vars in 1177 bytes} [Thu Jul 21 08:18:22 2022] GET /identity/v3/users/4ba5d9e54b8e400bb13b0dbb208f8105 => generated 286 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 08:18:22.190881 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc606aa4-cbb6-4850-8dbc-cfa37baba645 None admin] Authenticating user token {{(pid=80245) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:18:22.290844 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc606aa4-cbb6-4850-8dbc-cfa37baba645 None admin] Validating token access rules against request {{(pid=80245) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:18:22.324172 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5aa139d-afb4-4967-99c8-56fc58d2356d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80245) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:18:22.324819 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5aa139d-afb4-4967-99c8-56fc58d2356d None admin] REQUEST_METHOD: `GET` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:18:22.325105 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5aa139d-afb4-4967-99c8-56fc58d2356d None admin] SCRIPT_NAME: `/identity` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:18:22.325385 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5aa139d-afb4-4967-99c8-56fc58d2356d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:18:22.345902 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG dbcounter [-] [80245] Writing DB stats keystone:SELECT=100 {{(pid=80245) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:18:22.391232 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5aa139d-afb4-4967-99c8-56fc58d2356d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:18:22.392750 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5aa139d-afb4-4967-99c8-56fc58d2356d None admin] RBAC: Authorization granted {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:18:22.489691 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: [pid: 80245|app: 0|req: 556/1115] 10.222.0.47 () {58 vars in 1282 bytes} [Thu Jul 21 08:18:22 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 300 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 21 08:18:22.495557 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69b8bcfd-9508-4c1e-95b9-8dde9234e52a None admin] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:18:22.560526 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG dbcounter [-] [80244] Writing DB stats keystone:SELECT=100 {{(pid=80244) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:18:22.593156 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69b8bcfd-9508-4c1e-95b9-8dde9234e52a None admin] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:18:22.626974 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-035f0cd7-62d7-4743-bd0b-b7b4e2c6a324 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:18:22.627857 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-035f0cd7-62d7-4743-bd0b-b7b4e2c6a324 None admin] REQUEST_METHOD: `GET` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:18:22.628142 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-035f0cd7-62d7-4743-bd0b-b7b4e2c6a324 None admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:18:22.628419 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-035f0cd7-62d7-4743-bd0b-b7b4e2c6a324 None admin] PATH_INFO: `/v3/projects/057e4c8ef4634149b372875de3e7f3ef` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:18:22.629584 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-035f0cd7-62d7-4743-bd0b-b7b4e2c6a324 None admin] RBAC: Authorizing `identity:get_project(project_id=057e4c8ef4634149b372875de3e7f3ef)` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:18:22.631017 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-035f0cd7-62d7-4743-bd0b-b7b4e2c6a324 None admin] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:18:22.632733 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 560/1116] 10.222.0.47 () {56 vars in 1186 bytes} [Thu Jul 21 08:18:22 2022] GET /identity/v3/projects/057e4c8ef4634149b372875de3e7f3ef => generated 311 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 08:18:22.639163 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5aa139d-afb4-4967-99c8-56fc58d2356d None admin] Authenticating user token {{(pid=80245) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:18:22.689675 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG dbcounter [-] [80245] Writing DB stats keystone:SELECT=100 {{(pid=80245) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:18:22.735499 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5aa139d-afb4-4967-99c8-56fc58d2356d None admin] Validating token access rules against request {{(pid=80245) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:18:22.768932 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e63cdc29-88a7-4e7c-b9ef-8fc78819130a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80245) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:18:22.769588 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e63cdc29-88a7-4e7c-b9ef-8fc78819130a None admin] REQUEST_METHOD: `GET` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:18:22.769871 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e63cdc29-88a7-4e7c-b9ef-8fc78819130a None admin] SCRIPT_NAME: `/identity` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:18:22.770143 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e63cdc29-88a7-4e7c-b9ef-8fc78819130a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:18:22.770955 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e63cdc29-88a7-4e7c-b9ef-8fc78819130a None admin] RBAC: Adding query filter params (role.id=99a638c0aac44b79a1f2858574724495, scope.project.id=057e4c8ef4634149b372875de3e7f3ef, user.id=4ba5d9e54b8e400bb13b0dbb208f8105) {{(pid=80245) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 08:18:22.771263 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e63cdc29-88a7-4e7c-b9ef-8fc78819130a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:18:22.772703 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e63cdc29-88a7-4e7c-b9ef-8fc78819130a None admin] RBAC: Authorization granted {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:18:22.778057 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: [pid: 80245|app: 0|req: 557/1117] 10.222.0.47 () {56 vars in 1374 bytes} [Thu Jul 21 08:18:22 2022] GET /identity/v3/role_assignments?user.id=4ba5d9e54b8e400bb13b0dbb208f8105&scope.project.id=057e4c8ef4634149b372875de3e7f3ef&role.id=99a638c0aac44b79a1f2858574724495 => generated 259 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 08:18:24.773092 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-021ffcab-99e2-440a-a512-fff1b0cdb325 None None] REQUEST_METHOD: `GET` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:18:24.773593 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-021ffcab-99e2-440a-a512-fff1b0cdb325 None None] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:18:24.773593 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-021ffcab-99e2-440a-a512-fff1b0cdb325 None None] PATH_INFO: `/` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:18:24.774937 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 561/1118] 10.222.0.47 () {54 vars in 926 bytes} [Thu Jul 21 08:18:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 21 08:18:24.780236 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e86e134-01c5-46ae-9dce-2fe9546945bc None None] REQUEST_METHOD: `POST` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:18:24.780643 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e86e134-01c5-46ae-9dce-2fe9546945bc None None] SCRIPT_NAME: `/identity` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:18:24.780798 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e86e134-01c5-46ae-9dce-2fe9546945bc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:18:24.797269 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.auth.core [None req-0e86e134-01c5-46ae-9dce-2fe9546945bc None None] MFA Rules not processed for user `2dce40b8d74c41ec94cc36e6ff05d412`. Rule list: `[]` (Enabled: `True`). {{(pid=80245) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 08:18:24.798449 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.fernet_utils [None req-0e86e134-01c5-46ae-9dce-2fe9546945bc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80245) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 08:18:24.862049 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: [pid: 80245|app: 0|req: 558/1119] 10.222.0.47 () {58 vars in 1023 bytes} [Thu Jul 21 08:18:24 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 21 08:18:24.885944 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-021ffcab-99e2-440a-a512-fff1b0cdb325 None None] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:18:24.985158 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-021ffcab-99e2-440a-a512-fff1b0cdb325 None None] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:18:25.020596 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0176242a-8bfc-4506-b3a9-481124ecad63 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:18:25.021455 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0176242a-8bfc-4506-b3a9-481124ecad63 None admin] REQUEST_METHOD: `GET` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:18:25.021731 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0176242a-8bfc-4506-b3a9-481124ecad63 None admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:18:25.022004 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0176242a-8bfc-4506-b3a9-481124ecad63 None admin] PATH_INFO: `/v3/roles/99a638c0aac44b79a1f2858574724495` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:18:25.023553 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0176242a-8bfc-4506-b3a9-481124ecad63 None admin] RBAC: Authorizing `identity:get_role(role_id=99a638c0aac44b79a1f2858574724495)` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:18:25.024861 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0176242a-8bfc-4506-b3a9-481124ecad63 None admin] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:18:25.026237 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 562/1120] 10.222.0.47 () {56 vars in 1177 bytes} [Thu Jul 21 08:18:24 2022] GET /identity/v3/roles/99a638c0aac44b79a1f2858574724495 => generated 223 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 08:18:25.032100 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e86e134-01c5-46ae-9dce-2fe9546945bc None None] Authenticating user token {{(pid=80245) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:18:25.075928 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG dbcounter [-] [80245] Writing DB stats keystone:SELECT=100 {{(pid=80245) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:18:25.126459 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e86e134-01c5-46ae-9dce-2fe9546945bc None None] Validating token access rules against request {{(pid=80245) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:18:25.159484 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb3f87e2-7daa-45ae-9e01-411fa710283b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80245) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:18:25.160354 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb3f87e2-7daa-45ae-9e01-411fa710283b None admin] REQUEST_METHOD: `GET` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:18:25.160642 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb3f87e2-7daa-45ae-9e01-411fa710283b None admin] SCRIPT_NAME: `/identity` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:18:25.160916 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb3f87e2-7daa-45ae-9e01-411fa710283b None admin] PATH_INFO: `/v3/users/4ba5d9e54b8e400bb13b0dbb208f8105` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:18:25.162119 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb3f87e2-7daa-45ae-9e01-411fa710283b None admin] RBAC: Authorizing `identity:get_user(user_id=4ba5d9e54b8e400bb13b0dbb208f8105)` {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:18:25.163576 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb3f87e2-7daa-45ae-9e01-411fa710283b None admin] RBAC: Authorization granted {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:18:25.165206 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: [pid: 80245|app: 0|req: 559/1121] 10.222.0.47 () {56 vars in 1177 bytes} [Thu Jul 21 08:18:25 2022] GET /identity/v3/users/4ba5d9e54b8e400bb13b0dbb208f8105 => generated 286 bytes in 134 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 08:18:25.170970 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0176242a-8bfc-4506-b3a9-481124ecad63 None admin] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:18:25.175154 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG dbcounter [-] [80244] Writing DB stats keystone:SELECT=100 {{(pid=80244) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:18:25.268158 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0176242a-8bfc-4506-b3a9-481124ecad63 None admin] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:18:25.301209 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01d3bda6-a076-4dac-a811-3f5ea8693bbb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:18:25.301996 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01d3bda6-a076-4dac-a811-3f5ea8693bbb None admin] REQUEST_METHOD: `GET` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:18:25.302276 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01d3bda6-a076-4dac-a811-3f5ea8693bbb None admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:18:25.302549 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01d3bda6-a076-4dac-a811-3f5ea8693bbb None admin] PATH_INFO: `/v3/projects/057e4c8ef4634149b372875de3e7f3ef` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:18:25.303763 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01d3bda6-a076-4dac-a811-3f5ea8693bbb None admin] RBAC: Authorizing `identity:get_project(project_id=057e4c8ef4634149b372875de3e7f3ef)` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:18:25.305190 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01d3bda6-a076-4dac-a811-3f5ea8693bbb None admin] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:18:25.306871 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 563/1122] 10.222.0.47 () {56 vars in 1186 bytes} [Thu Jul 21 08:18:25 2022] GET /identity/v3/projects/057e4c8ef4634149b372875de3e7f3ef => generated 311 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 08:18:25.312901 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb3f87e2-7daa-45ae-9e01-411fa710283b None admin] Authenticating user token {{(pid=80245) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:18:25.409168 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb3f87e2-7daa-45ae-9e01-411fa710283b None admin] Validating token access rules against request {{(pid=80245) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:18:25.422546 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG dbcounter [-] [80245] Writing DB stats keystone:SELECT=100 {{(pid=80245) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:18:25.443587 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6d53ffd-6d78-41f6-9260-d6786cf8aa99 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80245) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:18:25.444470 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6d53ffd-6d78-41f6-9260-d6786cf8aa99 None admin] REQUEST_METHOD: `PUT` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:18:25.444749 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6d53ffd-6d78-41f6-9260-d6786cf8aa99 None admin] SCRIPT_NAME: `/identity` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:18:25.445020 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6d53ffd-6d78-41f6-9260-d6786cf8aa99 None admin] PATH_INFO: `/v3/projects/057e4c8ef4634149b372875de3e7f3ef/users/4ba5d9e54b8e400bb13b0dbb208f8105/roles/99a638c0aac44b79a1f2858574724495` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:18:25.447180 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6d53ffd-6d78-41f6-9260-d6786cf8aa99 None admin] RBAC: Authorizing `identity:create_grant(project_id=057e4c8ef4634149b372875de3e7f3ef, user_id=4ba5d9e54b8e400bb13b0dbb208f8105, role_id=99a638c0aac44b79a1f2858574724495)` {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:18:25.448633 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6d53ffd-6d78-41f6-9260-d6786cf8aa99 None admin] RBAC: Authorization granted {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:18:25.461947 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: [pid: 80245|app: 0|req: 560/1123] 10.222.0.47 () {58 vars in 1439 bytes} [Thu Jul 21 08:18:25 2022] PUT /identity/v3/projects/057e4c8ef4634149b372875de3e7f3ef/users/4ba5d9e54b8e400bb13b0dbb208f8105/roles/99a638c0aac44b79a1f2858574724495 => generated 0 bytes in 149 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 08:18:27.479763 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f715cc7-2e64-4816-a37b-28da352ce488 None None] REQUEST_METHOD: `GET` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:18:27.480294 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f715cc7-2e64-4816-a37b-28da352ce488 None None] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:18:27.480294 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f715cc7-2e64-4816-a37b-28da352ce488 None None] PATH_INFO: `/` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:18:27.481676 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 564/1124] 10.222.0.47 () {54 vars in 926 bytes} [Thu Jul 21 08:18:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 08:18:27.487458 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44d56ed6-32ec-4dbc-a2bb-90f157f3d0dc None None] REQUEST_METHOD: `POST` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:18:27.487872 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44d56ed6-32ec-4dbc-a2bb-90f157f3d0dc None None] SCRIPT_NAME: `/identity` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:18:27.488004 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44d56ed6-32ec-4dbc-a2bb-90f157f3d0dc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:18:27.504146 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.auth.core [None req-44d56ed6-32ec-4dbc-a2bb-90f157f3d0dc None None] MFA Rules not processed for user `2dce40b8d74c41ec94cc36e6ff05d412`. Rule list: `[]` (Enabled: `True`). {{(pid=80245) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 08:18:27.505321 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.fernet_utils [None req-44d56ed6-32ec-4dbc-a2bb-90f157f3d0dc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80245) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 08:18:27.569905 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: [pid: 80245|app: 0|req: 561/1125] 10.222.0.47 () {58 vars in 1023 bytes} [Thu Jul 21 08:18:27 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 21 08:18:27.593866 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f715cc7-2e64-4816-a37b-28da352ce488 None None] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:18:27.664959 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG dbcounter [-] [80244] Writing DB stats keystone:SELECT=100 {{(pid=80244) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:18:27.694756 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f715cc7-2e64-4816-a37b-28da352ce488 None None] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:18:27.728621 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caeccdbc-1542-4735-908d-6cb2b8fdf691 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:18:27.729469 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caeccdbc-1542-4735-908d-6cb2b8fdf691 None admin] REQUEST_METHOD: `GET` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:18:27.729756 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caeccdbc-1542-4735-908d-6cb2b8fdf691 None admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:18:27.730079 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caeccdbc-1542-4735-908d-6cb2b8fdf691 None admin] PATH_INFO: `/v3/roles/99a638c0aac44b79a1f2858574724495` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:18:27.731535 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caeccdbc-1542-4735-908d-6cb2b8fdf691 None admin] RBAC: Authorizing `identity:get_role(role_id=99a638c0aac44b79a1f2858574724495)` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:18:27.732908 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caeccdbc-1542-4735-908d-6cb2b8fdf691 None admin] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:18:27.734259 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 565/1126] 10.222.0.47 () {56 vars in 1177 bytes} [Thu Jul 21 08:18:27 2022] GET /identity/v3/roles/99a638c0aac44b79a1f2858574724495 => generated 223 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 08:18:27.740123 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44d56ed6-32ec-4dbc-a2bb-90f157f3d0dc None None] Authenticating user token {{(pid=80245) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:18:27.837256 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44d56ed6-32ec-4dbc-a2bb-90f157f3d0dc None None] Validating token access rules against request {{(pid=80245) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:18:27.839546 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG dbcounter [-] [80245] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80245) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:18:27.872544 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94d151f1-0d56-40c1-9ae6-0100d1a4d450 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80245) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:18:27.873420 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94d151f1-0d56-40c1-9ae6-0100d1a4d450 None admin] REQUEST_METHOD: `GET` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:18:27.873700 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94d151f1-0d56-40c1-9ae6-0100d1a4d450 None admin] SCRIPT_NAME: `/identity` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:18:27.873968 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94d151f1-0d56-40c1-9ae6-0100d1a4d450 None admin] PATH_INFO: `/v3/users/4ba5d9e54b8e400bb13b0dbb208f8105` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:18:27.875161 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94d151f1-0d56-40c1-9ae6-0100d1a4d450 None admin] RBAC: Authorizing `identity:get_user(user_id=4ba5d9e54b8e400bb13b0dbb208f8105)` {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:18:27.877099 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94d151f1-0d56-40c1-9ae6-0100d1a4d450 None admin] RBAC: Authorization granted {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:18:27.878723 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: [pid: 80245|app: 0|req: 562/1127] 10.222.0.47 () {56 vars in 1177 bytes} [Thu Jul 21 08:18:27 2022] GET /identity/v3/users/4ba5d9e54b8e400bb13b0dbb208f8105 => generated 286 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 08:18:27.884811 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caeccdbc-1542-4735-908d-6cb2b8fdf691 None admin] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:18:27.983422 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caeccdbc-1542-4735-908d-6cb2b8fdf691 None admin] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:18:28.019721 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ccb6362-0a9e-415d-b524-627846c82c46 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:18:28.020371 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ccb6362-0a9e-415d-b524-627846c82c46 None admin] REQUEST_METHOD: `GET` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:18:28.020658 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ccb6362-0a9e-415d-b524-627846c82c46 None admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:18:28.020936 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ccb6362-0a9e-415d-b524-627846c82c46 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:18:28.031236 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG dbcounter [-] [80244] Writing DB stats keystone:SELECT=100 {{(pid=80244) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:18:28.087902 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ccb6362-0a9e-415d-b524-627846c82c46 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:18:28.089349 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ccb6362-0a9e-415d-b524-627846c82c46 None admin] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:18:28.184199 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 566/1128] 10.222.0.47 () {58 vars in 1282 bytes} [Thu Jul 21 08:18:27 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 300 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 21 08:18:28.190749 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94d151f1-0d56-40c1-9ae6-0100d1a4d450 None admin] Authenticating user token {{(pid=80245) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:18:28.289486 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94d151f1-0d56-40c1-9ae6-0100d1a4d450 None admin] Validating token access rules against request {{(pid=80245) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:18:28.323656 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36cba3b3-191f-40d4-aefa-4bf616304567 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80245) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:18:28.324510 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36cba3b3-191f-40d4-aefa-4bf616304567 None admin] REQUEST_METHOD: `GET` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:18:28.324794 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36cba3b3-191f-40d4-aefa-4bf616304567 None admin] SCRIPT_NAME: `/identity` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:18:28.325066 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36cba3b3-191f-40d4-aefa-4bf616304567 None admin] PATH_INFO: `/v3/projects/057e4c8ef4634149b372875de3e7f3ef` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:18:28.326237 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36cba3b3-191f-40d4-aefa-4bf616304567 None admin] RBAC: Authorizing `identity:get_project(project_id=057e4c8ef4634149b372875de3e7f3ef)` {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:18:28.327689 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36cba3b3-191f-40d4-aefa-4bf616304567 None admin] RBAC: Authorization granted {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:18:28.329376 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: [pid: 80245|app: 0|req: 563/1129] 10.222.0.47 () {56 vars in 1186 bytes} [Thu Jul 21 08:18:28 2022] GET /identity/v3/projects/057e4c8ef4634149b372875de3e7f3ef => generated 311 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 21 08:18:28.335562 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ccb6362-0a9e-415d-b524-627846c82c46 None admin] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:18:28.375487 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG dbcounter [-] [80244] Writing DB stats keystone:SELECT=100 {{(pid=80244) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:18:28.436667 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ccb6362-0a9e-415d-b524-627846c82c46 None admin] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:18:28.469290 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4afcb30-569e-4c3f-aa9e-649ef560df78 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:18:28.469929 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4afcb30-569e-4c3f-aa9e-649ef560df78 None admin] REQUEST_METHOD: `GET` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:18:28.470209 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4afcb30-569e-4c3f-aa9e-649ef560df78 None admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:18:28.470974 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4afcb30-569e-4c3f-aa9e-649ef560df78 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:18:28.471839 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4afcb30-569e-4c3f-aa9e-649ef560df78 None admin] RBAC: Adding query filter params (role.id=99a638c0aac44b79a1f2858574724495, scope.project.id=057e4c8ef4634149b372875de3e7f3ef, user.id=4ba5d9e54b8e400bb13b0dbb208f8105) {{(pid=80244) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 08:18:28.472149 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4afcb30-569e-4c3f-aa9e-649ef560df78 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:18:28.473593 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4afcb30-569e-4c3f-aa9e-649ef560df78 None admin] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:18:28.479428 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 567/1130] 10.222.0.47 () {56 vars in 1374 bytes} [Thu Jul 21 08:18:28 2022] GET /identity/v3/role_assignments?user.id=4ba5d9e54b8e400bb13b0dbb208f8105&scope.project.id=057e4c8ef4634149b372875de3e7f3ef&role.id=99a638c0aac44b79a1f2858574724495 => generated 608 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 08:18:30.515643 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-460fbcf7-dec6-4675-a435-a36444f63c59 None None] REQUEST_METHOD: `GET` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:18:30.516097 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-460fbcf7-dec6-4675-a435-a36444f63c59 None None] SCRIPT_NAME: `/identity` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:18:30.516211 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-460fbcf7-dec6-4675-a435-a36444f63c59 None None] PATH_INFO: `/` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:18:30.518078 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: [pid: 80245|app: 0|req: 564/1131] 10.222.0.47 () {54 vars in 926 bytes} [Thu Jul 21 08:18:30 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 08:18:30.523340 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c94f72f1-2f2f-4f01-a219-dbade430d1ec None None] REQUEST_METHOD: `POST` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:18:30.523743 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c94f72f1-2f2f-4f01-a219-dbade430d1ec None None] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:18:30.523905 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c94f72f1-2f2f-4f01-a219-dbade430d1ec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:18:30.541665 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.auth.core [None req-c94f72f1-2f2f-4f01-a219-dbade430d1ec None None] MFA Rules not processed for user `2dce40b8d74c41ec94cc36e6ff05d412`. Rule list: `[]` (Enabled: `True`). {{(pid=80244) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 08:18:30.542840 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.fernet_utils [None req-c94f72f1-2f2f-4f01-a219-dbade430d1ec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80244) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 08:18:30.608770 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 568/1132] 10.222.0.47 () {58 vars in 1023 bytes} [Thu Jul 21 08:18:30 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 21 08:18:30.632050 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-460fbcf7-dec6-4675-a435-a36444f63c59 None None] Authenticating user token {{(pid=80245) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:18:30.677817 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG dbcounter [-] [80245] Writing DB stats keystone:SELECT=100 {{(pid=80245) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:18:30.734967 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-460fbcf7-dec6-4675-a435-a36444f63c59 None None] Validating token access rules against request {{(pid=80245) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:18:30.768712 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e4ac12b-fc26-429f-af40-50ae50117957 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80245) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:18:30.769441 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e4ac12b-fc26-429f-af40-50ae50117957 None admin] REQUEST_METHOD: `GET` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:18:30.769724 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e4ac12b-fc26-429f-af40-50ae50117957 None admin] SCRIPT_NAME: `/identity` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:18:30.770001 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e4ac12b-fc26-429f-af40-50ae50117957 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:18:30.771162 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e4ac12b-fc26-429f-af40-50ae50117957 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:18:30.772617 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e4ac12b-fc26-429f-af40-50ae50117957 None admin] RBAC: Authorization granted {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:18:30.774231 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: [pid: 80245|app: 0|req: 565/1133] 10.222.0.47 () {56 vars in 1108 bytes} [Thu Jul 21 08:18:30 2022] GET /identity/v3/domains/default => generated 199 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 08:18:30.780155 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c94f72f1-2f2f-4f01-a219-dbade430d1ec None None] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:18:30.812288 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG dbcounter [-] [80244] Writing DB stats keystone:SELECT=100 {{(pid=80244) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:18:30.882797 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c94f72f1-2f2f-4f01-a219-dbade430d1ec None None] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:18:30.918019 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33022ba1-198e-4db0-9d77-456fee15f918 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:18:30.918719 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33022ba1-198e-4db0-9d77-456fee15f918 None admin] REQUEST_METHOD: `POST` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:18:30.919005 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33022ba1-198e-4db0-9d77-456fee15f918 None admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:18:30.919282 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33022ba1-198e-4db0-9d77-456fee15f918 None admin] PATH_INFO: `/v3/projects` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:18:30.920041 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33022ba1-198e-4db0-9d77-456fee15f918 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:18:30.921483 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33022ba1-198e-4db0-9d77-456fee15f918 None admin] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:18:30.947211 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 569/1134] 10.222.0.47 () {60 vars in 1141 bytes} [Thu Jul 21 08:18:30 2022] POST /identity/v3/projects => generated 311 bytes in 167 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 08:18:33.043271 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-718953a0-1124-4516-8acc-1e598dceb993 None None] REQUEST_METHOD: `GET` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:18:33.043830 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-718953a0-1124-4516-8acc-1e598dceb993 None None] SCRIPT_NAME: `/identity` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:18:33.043830 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-718953a0-1124-4516-8acc-1e598dceb993 None None] PATH_INFO: `/` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:18:33.045199 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: [pid: 80245|app: 0|req: 566/1135] 10.222.0.47 () {54 vars in 926 bytes} [Thu Jul 21 08:18:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 08:18:33.050998 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-428d7d68-1f00-4a80-b2f7-3ef3fc9b977e None None] REQUEST_METHOD: `POST` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:18:33.051391 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-428d7d68-1f00-4a80-b2f7-3ef3fc9b977e None None] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:18:33.051591 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-428d7d68-1f00-4a80-b2f7-3ef3fc9b977e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:18:33.068056 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.auth.core [None req-428d7d68-1f00-4a80-b2f7-3ef3fc9b977e None None] MFA Rules not processed for user `2dce40b8d74c41ec94cc36e6ff05d412`. Rule list: `[]` (Enabled: `True`). {{(pid=80244) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 08:18:33.069235 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.fernet_utils [None req-428d7d68-1f00-4a80-b2f7-3ef3fc9b977e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80244) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 08:18:33.133630 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 570/1136] 10.222.0.47 () {58 vars in 1023 bytes} [Thu Jul 21 08:18:33 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 21 08:18:33.158563 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-718953a0-1124-4516-8acc-1e598dceb993 None None] Authenticating user token {{(pid=80245) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:18:33.258878 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-718953a0-1124-4516-8acc-1e598dceb993 None None] Validating token access rules against request {{(pid=80245) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:18:33.268030 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG dbcounter [-] [80245] Writing DB stats keystone:SELECT=100 {{(pid=80245) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:18:33.292400 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac458ca0-84c2-49ee-80ae-38e9d19138c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80245) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:18:33.293194 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac458ca0-84c2-49ee-80ae-38e9d19138c6 None admin] REQUEST_METHOD: `GET` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:18:33.293528 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac458ca0-84c2-49ee-80ae-38e9d19138c6 None admin] SCRIPT_NAME: `/identity` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:18:33.293753 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac458ca0-84c2-49ee-80ae-38e9d19138c6 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:18:33.294916 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac458ca0-84c2-49ee-80ae-38e9d19138c6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:18:33.296432 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac458ca0-84c2-49ee-80ae-38e9d19138c6 None admin] RBAC: Authorization granted {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:18:33.298073 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: [pid: 80245|app: 0|req: 567/1137] 10.222.0.47 () {56 vars in 1108 bytes} [Thu Jul 21 08:18:33 2022] GET /identity/v3/domains/default => generated 199 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 08:18:33.304451 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-428d7d68-1f00-4a80-b2f7-3ef3fc9b977e None None] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:18:33.321299 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG dbcounter [-] [80244] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80244) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:18:33.401894 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-428d7d68-1f00-4a80-b2f7-3ef3fc9b977e None None] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:18:33.435291 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b00e072-af43-4666-aa50-9818cb26c4db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:18:33.435997 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b00e072-af43-4666-aa50-9818cb26c4db None admin] REQUEST_METHOD: `POST` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:18:33.436282 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b00e072-af43-4666-aa50-9818cb26c4db None admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:18:33.436561 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b00e072-af43-4666-aa50-9818cb26c4db None admin] PATH_INFO: `/v3/users` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:18:33.437266 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b00e072-af43-4666-aa50-9818cb26c4db None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:18:33.438702 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b00e072-af43-4666-aa50-9818cb26c4db None admin] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:18:33.458775 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 571/1138] 10.222.0.47 () {60 vars in 1132 bytes} [Thu Jul 21 08:18:33 2022] POST /identity/v3/users => generated 286 bytes in 155 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 08:18:35.546898 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef18979d-3ab9-4bb2-8e61-28315f0ef1a7 None None] REQUEST_METHOD: `GET` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:18:35.547462 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef18979d-3ab9-4bb2-8e61-28315f0ef1a7 None None] SCRIPT_NAME: `/identity` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:18:35.547462 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef18979d-3ab9-4bb2-8e61-28315f0ef1a7 None None] PATH_INFO: `/` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:18:35.548885 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: [pid: 80245|app: 0|req: 568/1139] 10.222.0.47 () {54 vars in 926 bytes} [Thu Jul 21 08:18:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 08:18:35.554181 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-591b245b-11d1-4995-9a6c-5dd80772e36f None None] REQUEST_METHOD: `POST` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:18:35.554598 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-591b245b-11d1-4995-9a6c-5dd80772e36f None None] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:18:35.554747 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-591b245b-11d1-4995-9a6c-5dd80772e36f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:18:35.571850 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.auth.core [None req-591b245b-11d1-4995-9a6c-5dd80772e36f None None] MFA Rules not processed for user `2dce40b8d74c41ec94cc36e6ff05d412`. Rule list: `[]` (Enabled: `True`). {{(pid=80244) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 08:18:35.573036 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.fernet_utils [None req-591b245b-11d1-4995-9a6c-5dd80772e36f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80244) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 08:18:35.639162 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 572/1140] 10.222.0.47 () {58 vars in 1023 bytes} [Thu Jul 21 08:18:35 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 21 08:18:35.667308 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef18979d-3ab9-4bb2-8e61-28315f0ef1a7 None None] Authenticating user token {{(pid=80245) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:18:35.768799 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef18979d-3ab9-4bb2-8e61-28315f0ef1a7 None None] Validating token access rules against request {{(pid=80245) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:18:35.801498 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d16721bd-e3ae-41d3-84a4-158a219a0bec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80245) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:18:35.802293 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d16721bd-e3ae-41d3-84a4-158a219a0bec None admin] REQUEST_METHOD: `GET` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:18:35.802578 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d16721bd-e3ae-41d3-84a4-158a219a0bec None admin] SCRIPT_NAME: `/identity` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:18:35.802853 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d16721bd-e3ae-41d3-84a4-158a219a0bec None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:18:35.812691 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d16721bd-e3ae-41d3-84a4-158a219a0bec None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 21 08:18:35.820864 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application [None req-d16721bd-e3ae-41d3-84a4-158a219a0bec None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 21 08:18:35.820864 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 08:18:35.820864 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 08:18:35.820864 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 08:18:35.820864 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 08:18:35.820864 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 08:18:35.820864 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 08:18:35.820864 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 08:18:35.820864 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 08:18:35.820864 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 08:18:35.820864 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 08:18:35.820864 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 08:18:35.820864 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 21 08:18:35.820864 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 21 08:18:35.820864 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 21 08:18:35.820864 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 21 08:18:35.822021 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 21 08:18:35.822021 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 21 08:18:35.822021 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 21 08:18:35.822021 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 21 08:18:35.822021 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 08:18:35.822021 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 08:18:35.822021 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 08:18:35.822021 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 08:18:35.822021 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 08:18:35.822021 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 08:18:35.822021 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 08:18:35.822021 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application with Lock( Jul 21 08:18:35.822021 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 08:18:35.822021 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application return self._enter() Jul 21 08:18:35.822021 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 08:18:35.822021 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 08:18:35.823098 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 08:18:35.823098 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application return self.creator() Jul 21 08:18:35.823098 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 08:18:35.823098 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application created_value = creator( Jul 21 08:18:35.823098 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 21 08:18:35.823098 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 21 08:18:35.823098 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 21 08:18:35.823098 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 21 08:18:35.823098 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 21 08:18:35.823098 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 21 08:18:35.823098 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 21 08:18:35.823098 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application Jul 21 08:18:35.823098 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: [pid: 80245|app: 0|req: 569/1141] 10.222.0.47 () {56 vars in 1096 bytes} [Thu Jul 21 08:18:35 2022] GET /identity/v3/roles/admin => generated 83 bytes in 155 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 08:18:35.828262 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-591b245b-11d1-4995-9a6c-5dd80772e36f None None] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:18:35.830506 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG dbcounter [-] [80244] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80244) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:18:35.928243 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-591b245b-11d1-4995-9a6c-5dd80772e36f None None] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:18:35.961495 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-722589ae-e2bc-4b59-be90-968add7d1d17 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:18:35.962164 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-722589ae-e2bc-4b59-be90-968add7d1d17 None admin] REQUEST_METHOD: `GET` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:18:35.962449 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-722589ae-e2bc-4b59-be90-968add7d1d17 None admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:18:35.962725 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-722589ae-e2bc-4b59-be90-968add7d1d17 None admin] PATH_INFO: `/v3/roles` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:18:35.963503 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-722589ae-e2bc-4b59-be90-968add7d1d17 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80244) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 08:18:35.963811 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-722589ae-e2bc-4b59-be90-968add7d1d17 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:18:35.965179 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-722589ae-e2bc-4b59-be90-968add7d1d17 None admin] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:18:35.973852 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 573/1142] 10.222.0.47 () {56 vars in 1099 bytes} [Thu Jul 21 08:18:35 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 08:18:35.979763 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d16721bd-e3ae-41d3-84a4-158a219a0bec None admin] Authenticating user token {{(pid=80245) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:18:36.015775 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG dbcounter [-] [80245] Writing DB stats keystone:SELECT=100 {{(pid=80245) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:18:36.079668 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d16721bd-e3ae-41d3-84a4-158a219a0bec None admin] Validating token access rules against request {{(pid=80245) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:18:36.112570 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b7e425a-a7fd-4772-a926-c4efa3100d9b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80245) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:18:36.113427 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b7e425a-a7fd-4772-a926-c4efa3100d9b None admin] REQUEST_METHOD: `GET` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:18:36.113714 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b7e425a-a7fd-4772-a926-c4efa3100d9b None admin] SCRIPT_NAME: `/identity` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:18:36.113991 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b7e425a-a7fd-4772-a926-c4efa3100d9b None admin] PATH_INFO: `/v3/users/ee5d1231ff6f47579386b94ee3ea85e2` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:18:36.127441 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b7e425a-a7fd-4772-a926-c4efa3100d9b None admin] RBAC: Authorizing `identity:get_user(user_id=ee5d1231ff6f47579386b94ee3ea85e2)` {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:18:36.128879 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b7e425a-a7fd-4772-a926-c4efa3100d9b None admin] RBAC: Authorization granted {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:18:36.130527 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: [pid: 80245|app: 0|req: 570/1143] 10.222.0.47 () {56 vars in 1177 bytes} [Thu Jul 21 08:18:35 2022] GET /identity/v3/users/ee5d1231ff6f47579386b94ee3ea85e2 => generated 286 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 08:18:36.136573 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-722589ae-e2bc-4b59-be90-968add7d1d17 None admin] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:18:36.200120 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG dbcounter [-] [80244] Writing DB stats keystone:SELECT=100 {{(pid=80244) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:18:36.237999 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-722589ae-e2bc-4b59-be90-968add7d1d17 None admin] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:18:36.270410 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ece369fe-8602-483e-b5d9-eed09eb8c301 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:18:36.271057 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ece369fe-8602-483e-b5d9-eed09eb8c301 None admin] REQUEST_METHOD: `GET` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:18:36.271365 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ece369fe-8602-483e-b5d9-eed09eb8c301 None admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:18:36.271643 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ece369fe-8602-483e-b5d9-eed09eb8c301 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:18:36.343231 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ece369fe-8602-483e-b5d9-eed09eb8c301 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:18:36.344650 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ece369fe-8602-483e-b5d9-eed09eb8c301 None admin] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:18:36.406306 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG dbcounter [-] [80244] Writing DB stats keystone:SELECT=100 {{(pid=80244) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:18:36.444261 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 574/1144] 10.222.0.47 () {58 vars in 1282 bytes} [Thu Jul 21 08:18:36 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 308 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 21 08:18:36.450177 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b7e425a-a7fd-4772-a926-c4efa3100d9b None admin] Authenticating user token {{(pid=80245) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:18:36.542599 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG dbcounter [-] [80245] Writing DB stats keystone:SELECT=100 {{(pid=80245) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:18:36.549448 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b7e425a-a7fd-4772-a926-c4efa3100d9b None admin] Validating token access rules against request {{(pid=80245) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:18:36.582084 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03b959b6-e922-4bbb-a0d6-7a042732e739 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80245) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:18:36.582932 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03b959b6-e922-4bbb-a0d6-7a042732e739 None admin] REQUEST_METHOD: `GET` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:18:36.583212 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03b959b6-e922-4bbb-a0d6-7a042732e739 None admin] SCRIPT_NAME: `/identity` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:18:36.583510 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03b959b6-e922-4bbb-a0d6-7a042732e739 None admin] PATH_INFO: `/v3/projects/bfdfc7fc244d44638ad2bbc82978a6d5` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:18:36.584658 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03b959b6-e922-4bbb-a0d6-7a042732e739 None admin] RBAC: Authorizing `identity:get_project(project_id=bfdfc7fc244d44638ad2bbc82978a6d5)` {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:18:36.586080 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03b959b6-e922-4bbb-a0d6-7a042732e739 None admin] RBAC: Authorization granted {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:18:36.587804 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: [pid: 80245|app: 0|req: 571/1145] 10.222.0.47 () {56 vars in 1186 bytes} [Thu Jul 21 08:18:36 2022] GET /identity/v3/projects/bfdfc7fc244d44638ad2bbc82978a6d5 => generated 311 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 08:18:36.594008 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ece369fe-8602-483e-b5d9-eed09eb8c301 None admin] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:18:36.694946 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ece369fe-8602-483e-b5d9-eed09eb8c301 None admin] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:18:36.729377 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-469ff874-4077-4f52-893f-f114f12be0a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:18:36.730031 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-469ff874-4077-4f52-893f-f114f12be0a7 None admin] REQUEST_METHOD: `GET` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:18:36.730315 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-469ff874-4077-4f52-893f-f114f12be0a7 None admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:18:36.730592 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-469ff874-4077-4f52-893f-f114f12be0a7 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:18:36.731448 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-469ff874-4077-4f52-893f-f114f12be0a7 None admin] RBAC: Adding query filter params (role.id=e29870f6e9e2411a9db0d5bf45f32ca5, scope.project.id=bfdfc7fc244d44638ad2bbc82978a6d5, user.id=ee5d1231ff6f47579386b94ee3ea85e2) {{(pid=80244) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 08:18:36.731763 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-469ff874-4077-4f52-893f-f114f12be0a7 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:18:36.733195 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-469ff874-4077-4f52-893f-f114f12be0a7 None admin] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:18:36.738661 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 575/1146] 10.222.0.47 () {56 vars in 1374 bytes} [Thu Jul 21 08:18:36 2022] GET /identity/v3/role_assignments?user.id=ee5d1231ff6f47579386b94ee3ea85e2&scope.project.id=bfdfc7fc244d44638ad2bbc82978a6d5&role.id=e29870f6e9e2411a9db0d5bf45f32ca5 => generated 259 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 08:18:38.759386 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52eca279-8118-41f0-9251-1d3b3a34e45b None None] REQUEST_METHOD: `GET` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:18:38.759896 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52eca279-8118-41f0-9251-1d3b3a34e45b None None] SCRIPT_NAME: `/identity` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:18:38.759896 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52eca279-8118-41f0-9251-1d3b3a34e45b None None] PATH_INFO: `/` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:18:38.761255 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: [pid: 80245|app: 0|req: 572/1147] 10.222.0.47 () {54 vars in 926 bytes} [Thu Jul 21 08:18:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 08:18:38.766447 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9af1bbb1-653f-457b-b8fe-95451b36d323 None None] REQUEST_METHOD: `POST` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:18:38.766866 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9af1bbb1-653f-457b-b8fe-95451b36d323 None None] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:18:38.767048 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9af1bbb1-653f-457b-b8fe-95451b36d323 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:18:38.783284 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.auth.core [None req-9af1bbb1-653f-457b-b8fe-95451b36d323 None None] MFA Rules not processed for user `2dce40b8d74c41ec94cc36e6ff05d412`. Rule list: `[]` (Enabled: `True`). {{(pid=80244) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 08:18:38.784474 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.fernet_utils [None req-9af1bbb1-653f-457b-b8fe-95451b36d323 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80244) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 08:18:38.806532 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG dbcounter [-] [80244] Writing DB stats keystone:SELECT=100 {{(pid=80244) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:18:38.848912 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 576/1148] 10.222.0.47 () {58 vars in 1023 bytes} [Thu Jul 21 08:18:38 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 21 08:18:38.871446 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52eca279-8118-41f0-9251-1d3b3a34e45b None None] Authenticating user token {{(pid=80245) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:18:38.971133 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52eca279-8118-41f0-9251-1d3b3a34e45b None None] Validating token access rules against request {{(pid=80245) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:18:39.003822 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b93bb5a-1112-4af1-a302-5ddb6fa51c42 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80245) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:18:39.004607 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b93bb5a-1112-4af1-a302-5ddb6fa51c42 None admin] REQUEST_METHOD: `GET` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:18:39.004892 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b93bb5a-1112-4af1-a302-5ddb6fa51c42 None admin] SCRIPT_NAME: `/identity` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:18:39.005170 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b93bb5a-1112-4af1-a302-5ddb6fa51c42 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:18:39.015316 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b93bb5a-1112-4af1-a302-5ddb6fa51c42 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 21 08:18:39.022858 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application [None req-1b93bb5a-1112-4af1-a302-5ddb6fa51c42 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 21 08:18:39.022858 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 08:18:39.022858 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 08:18:39.022858 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 08:18:39.022858 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 08:18:39.022858 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 08:18:39.022858 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 08:18:39.022858 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 08:18:39.022858 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 08:18:39.022858 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 08:18:39.022858 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 08:18:39.022858 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 08:18:39.022858 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 21 08:18:39.022858 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 21 08:18:39.022858 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 21 08:18:39.022858 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 21 08:18:39.023968 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 21 08:18:39.023968 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 21 08:18:39.023968 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 21 08:18:39.023968 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 21 08:18:39.023968 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 08:18:39.023968 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 08:18:39.023968 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 08:18:39.023968 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 08:18:39.023968 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 08:18:39.023968 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 08:18:39.023968 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 08:18:39.023968 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application with Lock( Jul 21 08:18:39.023968 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 08:18:39.023968 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application return self._enter() Jul 21 08:18:39.023968 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 08:18:39.023968 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 08:18:39.024998 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 08:18:39.024998 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application return self.creator() Jul 21 08:18:39.024998 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 08:18:39.024998 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application created_value = creator( Jul 21 08:18:39.024998 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 21 08:18:39.024998 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 21 08:18:39.024998 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 21 08:18:39.024998 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 21 08:18:39.024998 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 21 08:18:39.024998 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 21 08:18:39.024998 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 21 08:18:39.024998 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application Jul 21 08:18:39.024998 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: [pid: 80245|app: 0|req: 573/1149] 10.222.0.47 () {56 vars in 1096 bytes} [Thu Jul 21 08:18:38 2022] GET /identity/v3/roles/admin => generated 83 bytes in 153 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 08:18:39.030320 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9af1bbb1-653f-457b-b8fe-95451b36d323 None None] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:18:39.127411 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9af1bbb1-653f-457b-b8fe-95451b36d323 None None] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:18:39.161131 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32a59283-6101-4b67-90d9-c2365e4ad366 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:18:39.161797 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32a59283-6101-4b67-90d9-c2365e4ad366 None admin] REQUEST_METHOD: `GET` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:18:39.162082 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32a59283-6101-4b67-90d9-c2365e4ad366 None admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:18:39.162438 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32a59283-6101-4b67-90d9-c2365e4ad366 None admin] PATH_INFO: `/v3/roles` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:18:39.163104 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32a59283-6101-4b67-90d9-c2365e4ad366 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80244) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 08:18:39.163442 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32a59283-6101-4b67-90d9-c2365e4ad366 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:18:39.164809 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32a59283-6101-4b67-90d9-c2365e4ad366 None admin] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:18:39.172755 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 577/1150] 10.222.0.47 () {56 vars in 1099 bytes} [Thu Jul 21 08:18:39 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 08:18:39.179498 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b93bb5a-1112-4af1-a302-5ddb6fa51c42 None admin] Authenticating user token {{(pid=80245) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:18:39.203303 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG dbcounter [-] [80245] Writing DB stats keystone:SELECT=100 {{(pid=80245) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:18:39.276658 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b93bb5a-1112-4af1-a302-5ddb6fa51c42 None admin] Validating token access rules against request {{(pid=80245) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:18:39.309470 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54481004-9d51-4fa2-9c86-4717362bd07e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80245) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:18:39.310324 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54481004-9d51-4fa2-9c86-4717362bd07e None admin] REQUEST_METHOD: `GET` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:18:39.310603 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54481004-9d51-4fa2-9c86-4717362bd07e None admin] SCRIPT_NAME: `/identity` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:18:39.310874 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54481004-9d51-4fa2-9c86-4717362bd07e None admin] PATH_INFO: `/v3/users/ee5d1231ff6f47579386b94ee3ea85e2` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:18:39.312051 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54481004-9d51-4fa2-9c86-4717362bd07e None admin] RBAC: Authorizing `identity:get_user(user_id=ee5d1231ff6f47579386b94ee3ea85e2)` {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:18:39.313464 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54481004-9d51-4fa2-9c86-4717362bd07e None admin] RBAC: Authorization granted {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:18:39.315096 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: [pid: 80245|app: 0|req: 574/1151] 10.222.0.47 () {56 vars in 1177 bytes} [Thu Jul 21 08:18:39 2022] GET /identity/v3/users/ee5d1231ff6f47579386b94ee3ea85e2 => generated 286 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 08:18:39.321428 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32a59283-6101-4b67-90d9-c2365e4ad366 None admin] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:18:39.343069 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG dbcounter [-] [80244] Writing DB stats keystone:SELECT=100 {{(pid=80244) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:18:39.421391 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32a59283-6101-4b67-90d9-c2365e4ad366 None admin] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:18:39.453542 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7203fa4-766d-4385-ae97-441513475725 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:18:39.454893 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7203fa4-766d-4385-ae97-441513475725 None admin] REQUEST_METHOD: `GET` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:18:39.455172 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7203fa4-766d-4385-ae97-441513475725 None admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:18:39.455480 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7203fa4-766d-4385-ae97-441513475725 None admin] PATH_INFO: `/v3/projects/bfdfc7fc244d44638ad2bbc82978a6d5` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:18:39.456650 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7203fa4-766d-4385-ae97-441513475725 None admin] RBAC: Authorizing `identity:get_project(project_id=bfdfc7fc244d44638ad2bbc82978a6d5)` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:18:39.458070 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7203fa4-766d-4385-ae97-441513475725 None admin] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:18:39.459774 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 578/1152] 10.222.0.47 () {56 vars in 1186 bytes} [Thu Jul 21 08:18:39 2022] GET /identity/v3/projects/bfdfc7fc244d44638ad2bbc82978a6d5 => generated 311 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 08:18:39.466075 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54481004-9d51-4fa2-9c86-4717362bd07e None admin] Authenticating user token {{(pid=80245) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:18:39.556555 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG dbcounter [-] [80245] Writing DB stats keystone:SELECT=100 {{(pid=80245) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:18:39.566917 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54481004-9d51-4fa2-9c86-4717362bd07e None admin] Validating token access rules against request {{(pid=80245) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:18:39.600741 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da6faa16-8947-4988-9c18-1ee664cea580 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80245) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:18:39.601631 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da6faa16-8947-4988-9c18-1ee664cea580 None admin] REQUEST_METHOD: `PUT` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:18:39.601919 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da6faa16-8947-4988-9c18-1ee664cea580 None admin] SCRIPT_NAME: `/identity` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:18:39.602198 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da6faa16-8947-4988-9c18-1ee664cea580 None admin] PATH_INFO: `/v3/projects/bfdfc7fc244d44638ad2bbc82978a6d5/users/ee5d1231ff6f47579386b94ee3ea85e2/roles/e29870f6e9e2411a9db0d5bf45f32ca5` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:18:39.604205 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da6faa16-8947-4988-9c18-1ee664cea580 None admin] RBAC: Authorizing `identity:create_grant(project_id=bfdfc7fc244d44638ad2bbc82978a6d5, user_id=ee5d1231ff6f47579386b94ee3ea85e2, role_id=e29870f6e9e2411a9db0d5bf45f32ca5)` {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:18:39.605636 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da6faa16-8947-4988-9c18-1ee664cea580 None admin] RBAC: Authorization granted {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:18:39.618607 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: [pid: 80245|app: 0|req: 575/1153] 10.222.0.47 () {58 vars in 1439 bytes} [Thu Jul 21 08:18:39 2022] PUT /identity/v3/projects/bfdfc7fc244d44638ad2bbc82978a6d5/users/ee5d1231ff6f47579386b94ee3ea85e2/roles/e29870f6e9e2411a9db0d5bf45f32ca5 => generated 0 bytes in 153 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 08:18:41.637929 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e4b2464-ee07-4354-8da6-741123eaeaca None None] REQUEST_METHOD: `GET` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:18:41.638472 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e4b2464-ee07-4354-8da6-741123eaeaca None None] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:18:41.638565 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e4b2464-ee07-4354-8da6-741123eaeaca None None] PATH_INFO: `/` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:18:41.640006 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 579/1154] 10.222.0.47 () {54 vars in 926 bytes} [Thu Jul 21 08:18:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 08:18:41.645259 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d5a56d2-3a4e-4594-8518-29ab50210cee None None] REQUEST_METHOD: `POST` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:18:41.645684 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d5a56d2-3a4e-4594-8518-29ab50210cee None None] SCRIPT_NAME: `/identity` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:18:41.645807 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d5a56d2-3a4e-4594-8518-29ab50210cee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:18:41.662200 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.auth.core [None req-3d5a56d2-3a4e-4594-8518-29ab50210cee None None] MFA Rules not processed for user `2dce40b8d74c41ec94cc36e6ff05d412`. Rule list: `[]` (Enabled: `True`). {{(pid=80245) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 08:18:41.663413 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.fernet_utils [None req-3d5a56d2-3a4e-4594-8518-29ab50210cee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80245) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 08:18:41.728626 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: [pid: 80245|app: 0|req: 576/1155] 10.222.0.47 () {58 vars in 1023 bytes} [Thu Jul 21 08:18:41 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 21 08:18:41.753307 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e4b2464-ee07-4354-8da6-741123eaeaca None None] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:18:41.839238 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG dbcounter [-] [80244] Writing DB stats keystone:SELECT=100 {{(pid=80244) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:18:41.852113 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e4b2464-ee07-4354-8da6-741123eaeaca None None] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:18:41.885975 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ac88fcc-a90e-4359-9d0e-9aec7a0f7b84 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:18:41.886770 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ac88fcc-a90e-4359-9d0e-9aec7a0f7b84 None admin] REQUEST_METHOD: `GET` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:18:41.887052 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ac88fcc-a90e-4359-9d0e-9aec7a0f7b84 None admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:18:41.887347 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ac88fcc-a90e-4359-9d0e-9aec7a0f7b84 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:18:41.897824 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ac88fcc-a90e-4359-9d0e-9aec7a0f7b84 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 21 08:18:41.905418 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application [None req-5ac88fcc-a90e-4359-9d0e-9aec7a0f7b84 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 21 08:18:41.905418 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 08:18:41.905418 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 08:18:41.905418 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 08:18:41.905418 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 08:18:41.905418 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 08:18:41.905418 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 08:18:41.905418 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 08:18:41.905418 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 08:18:41.905418 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 08:18:41.905418 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 08:18:41.905418 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 08:18:41.905418 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 21 08:18:41.905418 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 21 08:18:41.905418 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 21 08:18:41.905418 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 21 08:18:41.906496 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 21 08:18:41.906496 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 21 08:18:41.906496 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 21 08:18:41.906496 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 21 08:18:41.906496 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 08:18:41.906496 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 08:18:41.906496 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 08:18:41.906496 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 08:18:41.906496 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 08:18:41.906496 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 08:18:41.906496 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 08:18:41.906496 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application with Lock( Jul 21 08:18:41.906496 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 08:18:41.906496 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self._enter() Jul 21 08:18:41.906496 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 08:18:41.906496 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 08:18:41.907585 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 08:18:41.907585 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self.creator() Jul 21 08:18:41.907585 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 08:18:41.907585 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application created_value = creator( Jul 21 08:18:41.907585 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 21 08:18:41.907585 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 21 08:18:41.907585 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 21 08:18:41.907585 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 21 08:18:41.907585 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 21 08:18:41.907585 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 21 08:18:41.907585 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 21 08:18:41.907585 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application Jul 21 08:18:41.907585 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 580/1156] 10.222.0.47 () {56 vars in 1096 bytes} [Thu Jul 21 08:18:41 2022] GET /identity/v3/roles/admin => generated 83 bytes in 154 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 08:18:41.912866 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d5a56d2-3a4e-4594-8518-29ab50210cee None None] Authenticating user token {{(pid=80245) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:18:41.993115 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG dbcounter [-] [80245] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80245) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:18:42.013421 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d5a56d2-3a4e-4594-8518-29ab50210cee None None] Validating token access rules against request {{(pid=80245) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:18:42.047861 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c17033a0-b1b9-4161-ad9a-78100a527f6e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80245) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:18:42.049142 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c17033a0-b1b9-4161-ad9a-78100a527f6e None admin] REQUEST_METHOD: `GET` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:18:42.049418 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c17033a0-b1b9-4161-ad9a-78100a527f6e None admin] SCRIPT_NAME: `/identity` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:18:42.049689 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c17033a0-b1b9-4161-ad9a-78100a527f6e None admin] PATH_INFO: `/v3/roles` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:18:42.050425 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c17033a0-b1b9-4161-ad9a-78100a527f6e None admin] RBAC: Adding query filter params (name=admin) {{(pid=80245) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 08:18:42.050724 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c17033a0-b1b9-4161-ad9a-78100a527f6e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:18:42.052101 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c17033a0-b1b9-4161-ad9a-78100a527f6e None admin] RBAC: Authorization granted {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:18:42.059870 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: [pid: 80245|app: 0|req: 577/1157] 10.222.0.47 () {56 vars in 1099 bytes} [Thu Jul 21 08:18:41 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 08:18:42.065809 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ac88fcc-a90e-4359-9d0e-9aec7a0f7b84 None admin] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:18:42.163733 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ac88fcc-a90e-4359-9d0e-9aec7a0f7b84 None admin] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:18:42.196206 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-226cfc80-2406-42ba-b0b7-88d39b6ac3c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:18:42.197072 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-226cfc80-2406-42ba-b0b7-88d39b6ac3c7 None admin] REQUEST_METHOD: `GET` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:18:42.197354 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-226cfc80-2406-42ba-b0b7-88d39b6ac3c7 None admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:18:42.197627 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-226cfc80-2406-42ba-b0b7-88d39b6ac3c7 None admin] PATH_INFO: `/v3/users/ee5d1231ff6f47579386b94ee3ea85e2` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:18:42.198796 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-226cfc80-2406-42ba-b0b7-88d39b6ac3c7 None admin] RBAC: Authorizing `identity:get_user(user_id=ee5d1231ff6f47579386b94ee3ea85e2)` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:18:42.200256 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-226cfc80-2406-42ba-b0b7-88d39b6ac3c7 None admin] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:18:42.201880 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 581/1158] 10.222.0.47 () {56 vars in 1177 bytes} [Thu Jul 21 08:18:42 2022] GET /identity/v3/users/ee5d1231ff6f47579386b94ee3ea85e2 => generated 286 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 08:18:42.207729 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c17033a0-b1b9-4161-ad9a-78100a527f6e None admin] Authenticating user token {{(pid=80245) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:18:42.304431 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c17033a0-b1b9-4161-ad9a-78100a527f6e None admin] Validating token access rules against request {{(pid=80245) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:18:42.337478 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-684ee3a2-27ad-441e-a66e-05552bbc0a43 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80245) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:18:42.338136 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-684ee3a2-27ad-441e-a66e-05552bbc0a43 None admin] REQUEST_METHOD: `GET` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:18:42.338416 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-684ee3a2-27ad-441e-a66e-05552bbc0a43 None admin] SCRIPT_NAME: `/identity` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:18:42.338692 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-684ee3a2-27ad-441e-a66e-05552bbc0a43 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:18:42.353863 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG dbcounter [-] [80245] Writing DB stats keystone:SELECT=100 {{(pid=80245) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:18:42.408716 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-684ee3a2-27ad-441e-a66e-05552bbc0a43 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:18:42.410152 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-684ee3a2-27ad-441e-a66e-05552bbc0a43 None admin] RBAC: Authorization granted {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:18:42.504081 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: [pid: 80245|app: 0|req: 578/1159] 10.222.0.47 () {58 vars in 1282 bytes} [Thu Jul 21 08:18:42 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 297 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 21 08:18:42.510174 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-226cfc80-2406-42ba-b0b7-88d39b6ac3c7 None admin] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:18:42.528711 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG dbcounter [-] [80244] Writing DB stats keystone:SELECT=100 {{(pid=80244) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:18:42.609011 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-226cfc80-2406-42ba-b0b7-88d39b6ac3c7 None admin] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:18:42.644076 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1580a8e4-bca5-4703-bbee-4ecbfca8d908 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:18:42.644937 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1580a8e4-bca5-4703-bbee-4ecbfca8d908 None admin] REQUEST_METHOD: `GET` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:18:42.645226 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1580a8e4-bca5-4703-bbee-4ecbfca8d908 None admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:18:42.645503 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1580a8e4-bca5-4703-bbee-4ecbfca8d908 None admin] PATH_INFO: `/v3/projects/bfdfc7fc244d44638ad2bbc82978a6d5` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:18:42.646705 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1580a8e4-bca5-4703-bbee-4ecbfca8d908 None admin] RBAC: Authorizing `identity:get_project(project_id=bfdfc7fc244d44638ad2bbc82978a6d5)` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:18:42.648191 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1580a8e4-bca5-4703-bbee-4ecbfca8d908 None admin] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:18:42.649906 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 582/1160] 10.222.0.47 () {56 vars in 1186 bytes} [Thu Jul 21 08:18:42 2022] GET /identity/v3/projects/bfdfc7fc244d44638ad2bbc82978a6d5 => generated 311 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 08:18:42.656120 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-684ee3a2-27ad-441e-a66e-05552bbc0a43 None admin] Authenticating user token {{(pid=80245) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:18:42.697706 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG dbcounter [-] [80245] Writing DB stats keystone:SELECT=100 {{(pid=80245) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:18:42.753576 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-684ee3a2-27ad-441e-a66e-05552bbc0a43 None admin] Validating token access rules against request {{(pid=80245) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:18:42.787060 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac531f00-e99a-41c6-803e-a75cbad7337a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80245) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:18:42.787739 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac531f00-e99a-41c6-803e-a75cbad7337a None admin] REQUEST_METHOD: `GET` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:18:42.788020 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac531f00-e99a-41c6-803e-a75cbad7337a None admin] SCRIPT_NAME: `/identity` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:18:42.788290 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac531f00-e99a-41c6-803e-a75cbad7337a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:18:42.789097 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac531f00-e99a-41c6-803e-a75cbad7337a None admin] RBAC: Adding query filter params (role.id=e29870f6e9e2411a9db0d5bf45f32ca5, scope.project.id=bfdfc7fc244d44638ad2bbc82978a6d5, user.id=ee5d1231ff6f47579386b94ee3ea85e2) {{(pid=80245) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 08:18:42.789402 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac531f00-e99a-41c6-803e-a75cbad7337a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:18:42.790809 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac531f00-e99a-41c6-803e-a75cbad7337a None admin] RBAC: Authorization granted {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:18:42.796551 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: [pid: 80245|app: 0|req: 579/1161] 10.222.0.47 () {56 vars in 1374 bytes} [Thu Jul 21 08:18:42 2022] GET /identity/v3/role_assignments?user.id=ee5d1231ff6f47579386b94ee3ea85e2&scope.project.id=bfdfc7fc244d44638ad2bbc82978a6d5&role.id=e29870f6e9e2411a9db0d5bf45f32ca5 => generated 608 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 08:18:44.868571 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf6cdd8d-ffc4-4f8e-807f-4b58c2d07bb2 None None] REQUEST_METHOD: `GET` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:18:44.869071 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf6cdd8d-ffc4-4f8e-807f-4b58c2d07bb2 None None] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:18:44.869071 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf6cdd8d-ffc4-4f8e-807f-4b58c2d07bb2 None None] PATH_INFO: `/` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:18:44.870405 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 583/1162] 10.222.0.47 () {54 vars in 926 bytes} [Thu Jul 21 08:18:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 08:18:44.875722 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a689595-7f8b-4de5-a76a-71abf531d2b9 None None] REQUEST_METHOD: `POST` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:18:44.876132 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a689595-7f8b-4de5-a76a-71abf531d2b9 None None] SCRIPT_NAME: `/identity` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:18:44.876273 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a689595-7f8b-4de5-a76a-71abf531d2b9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:18:44.893620 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.auth.core [None req-7a689595-7f8b-4de5-a76a-71abf531d2b9 None None] MFA Rules not processed for user `2dce40b8d74c41ec94cc36e6ff05d412`. Rule list: `[]` (Enabled: `True`). {{(pid=80245) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 08:18:44.894805 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.fernet_utils [None req-7a689595-7f8b-4de5-a76a-71abf531d2b9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80245) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 08:18:44.959949 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: [pid: 80245|app: 0|req: 580/1163] 10.222.0.47 () {58 vars in 1023 bytes} [Thu Jul 21 08:18:44 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 21 08:18:44.982572 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf6cdd8d-ffc4-4f8e-807f-4b58c2d07bb2 None None] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:18:45.068637 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG dbcounter [-] [80244] Writing DB stats keystone:SELECT=100 {{(pid=80244) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:18:45.086352 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf6cdd8d-ffc4-4f8e-807f-4b58c2d07bb2 None None] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:18:45.118556 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-776f7646-deba-4fd9-8761-3e9e8651b3a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:18:45.119249 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-776f7646-deba-4fd9-8761-3e9e8651b3a5 None admin] REQUEST_METHOD: `GET` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:18:45.119521 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-776f7646-deba-4fd9-8761-3e9e8651b3a5 None admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:18:45.119803 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-776f7646-deba-4fd9-8761-3e9e8651b3a5 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:18:45.185017 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-776f7646-deba-4fd9-8761-3e9e8651b3a5 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:18:45.186461 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-776f7646-deba-4fd9-8761-3e9e8651b3a5 None admin] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:18:45.268248 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG dbcounter [-] [80244] Writing DB stats keystone:SELECT=100 {{(pid=80244) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:18:45.284686 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 584/1164] 10.222.0.47 () {58 vars in 1282 bytes} [Thu Jul 21 08:18:44 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 303 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 21 08:18:45.291435 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a689595-7f8b-4de5-a76a-71abf531d2b9 None None] Authenticating user token {{(pid=80245) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:18:45.324328 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG dbcounter [-] [80245] Writing DB stats keystone:SELECT=100 {{(pid=80245) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:18:45.390497 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a689595-7f8b-4de5-a76a-71abf531d2b9 None None] Validating token access rules against request {{(pid=80245) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:18:45.423307 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a048bf10-18de-4fb7-8a8a-a1ca59a681e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80245) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:18:45.424129 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a048bf10-18de-4fb7-8a8a-a1ca59a681e3 None admin] REQUEST_METHOD: `GET` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:18:45.424414 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a048bf10-18de-4fb7-8a8a-a1ca59a681e3 None admin] SCRIPT_NAME: `/identity` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:18:45.424681 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a048bf10-18de-4fb7-8a8a-a1ca59a681e3 None admin] PATH_INFO: `/v3/domains/swift_test` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:18:45.431130 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a048bf10-18de-4fb7-8a8a-a1ca59a681e3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=swift_test)` {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:18:45.432581 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a048bf10-18de-4fb7-8a8a-a1ca59a681e3 None admin] RBAC: Authorization granted {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:18:45.446773 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application [None req-a048bf10-18de-4fb7-8a8a-a1ca59a681e3 None admin] Could not find domain: swift_test.: keystone.exception.DomainNotFound: Could not find domain: swift_test. Jul 21 08:18:45.446773 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 08:18:45.446773 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 712, in get_domain Jul 21 08:18:45.446773 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application project = self.driver.get_project(domain_id) Jul 21 08:18:45.446773 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 21 08:18:45.446773 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 21 08:18:45.446773 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 21 08:18:45.446773 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 21 08:18:45.446773 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: swift_test. Jul 21 08:18:45.446773 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application Jul 21 08:18:45.446773 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 21 08:18:45.446773 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application Jul 21 08:18:45.446773 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 08:18:45.446773 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 08:18:45.446773 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 08:18:45.446773 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 08:18:45.446773 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 08:18:45.448017 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 08:18:45.448017 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 08:18:45.448017 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 08:18:45.448017 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 08:18:45.448017 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 08:18:45.448017 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 08:18:45.448017 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 21 08:18:45.448017 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 21 08:18:45.448017 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 21 08:18:45.448017 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 21 08:18:45.448017 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 08:18:45.448017 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 08:18:45.448017 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 08:18:45.448017 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 08:18:45.448017 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 08:18:45.448017 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 08:18:45.448897 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 08:18:45.448897 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application with Lock( Jul 21 08:18:45.448897 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 08:18:45.448897 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application return self._enter() Jul 21 08:18:45.448897 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 08:18:45.448897 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 08:18:45.448897 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 08:18:45.448897 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application return self.creator() Jul 21 08:18:45.448897 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 08:18:45.448897 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application created_value = creator( Jul 21 08:18:45.448897 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 718, in get_domain Jul 21 08:18:45.448897 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 21 08:18:45.448897 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: swift_test. Jul 21 08:18:45.448897 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application Jul 21 08:18:45.448897 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: [pid: 80245|app: 0|req: 581/1165] 10.222.0.47 () {56 vars in 1117 bytes} [Thu Jul 21 08:18:45 2022] GET /identity/v3/domains/swift_test => generated 90 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 08:18:45.454017 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-776f7646-deba-4fd9-8761-3e9e8651b3a5 None admin] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:18:45.554386 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-776f7646-deba-4fd9-8761-3e9e8651b3a5 None admin] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:18:45.587270 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff15752c-970d-4a3c-a52a-3777b4733331 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:18:45.587953 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff15752c-970d-4a3c-a52a-3777b4733331 None admin] REQUEST_METHOD: `GET` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:18:45.588234 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff15752c-970d-4a3c-a52a-3777b4733331 None admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:18:45.588508 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff15752c-970d-4a3c-a52a-3777b4733331 None admin] PATH_INFO: `/v3/domains` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:18:45.589496 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff15752c-970d-4a3c-a52a-3777b4733331 None admin] RBAC: Adding query filter params (name=swift_test) {{(pid=80244) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 08:18:45.589496 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff15752c-970d-4a3c-a52a-3777b4733331 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:18:45.590864 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff15752c-970d-4a3c-a52a-3777b4733331 None admin] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:18:45.597868 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 585/1166] 10.222.0.47 () {56 vars in 1115 bytes} [Thu Jul 21 08:18:45 2022] GET /identity/v3/domains?name=swift_test => generated 125 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 08:18:47.593659 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9432f7d-5865-422a-bf94-45fff4754c5f None None] REQUEST_METHOD: `GET` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:18:47.594182 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9432f7d-5865-422a-bf94-45fff4754c5f None None] SCRIPT_NAME: `/identity` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:18:47.594294 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9432f7d-5865-422a-bf94-45fff4754c5f None None] PATH_INFO: `/` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:18:47.596118 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: [pid: 80245|app: 0|req: 582/1167] 10.222.0.47 () {54 vars in 926 bytes} [Thu Jul 21 08:18:47 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 08:18:47.601383 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-175a234d-5818-4259-9651-fea6b1e404f3 None None] REQUEST_METHOD: `POST` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:18:47.601799 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-175a234d-5818-4259-9651-fea6b1e404f3 None None] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:18:47.601944 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-175a234d-5818-4259-9651-fea6b1e404f3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:18:47.619040 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.auth.core [None req-175a234d-5818-4259-9651-fea6b1e404f3 None None] MFA Rules not processed for user `2dce40b8d74c41ec94cc36e6ff05d412`. Rule list: `[]` (Enabled: `True`). {{(pid=80244) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 08:18:47.620263 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.fernet_utils [None req-175a234d-5818-4259-9651-fea6b1e404f3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80244) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 08:18:47.663386 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG dbcounter [-] [80244] Writing DB stats keystone:SELECT=100 {{(pid=80244) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:18:47.687006 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 586/1168] 10.222.0.47 () {58 vars in 1023 bytes} [Thu Jul 21 08:18:47 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 21 08:18:47.710010 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9432f7d-5865-422a-bf94-45fff4754c5f None None] Authenticating user token {{(pid=80245) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:18:47.802787 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG dbcounter [-] [80245] Writing DB stats keystone:SELECT=100 {{(pid=80245) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:18:47.809684 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9432f7d-5865-422a-bf94-45fff4754c5f None None] Validating token access rules against request {{(pid=80245) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:18:47.842341 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80926dbc-3679-4b97-9fd1-46e78bc91057 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80245) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:18:47.842999 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80926dbc-3679-4b97-9fd1-46e78bc91057 None admin] REQUEST_METHOD: `POST` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:18:47.843289 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80926dbc-3679-4b97-9fd1-46e78bc91057 None admin] SCRIPT_NAME: `/identity` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:18:47.843599 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80926dbc-3679-4b97-9fd1-46e78bc91057 None admin] PATH_INFO: `/v3/domains` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:18:47.844334 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80926dbc-3679-4b97-9fd1-46e78bc91057 None admin] RBAC: Authorizing `identity:create_domain()` {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:18:47.846287 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80926dbc-3679-4b97-9fd1-46e78bc91057 None admin] RBAC: Authorization granted {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:18:47.868257 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: [pid: 80245|app: 0|req: 583/1169] 10.222.0.47 () {60 vars in 1138 bytes} [Thu Jul 21 08:18:47 2022] POST /identity/v3/domains => generated 267 bytes in 159 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 08:18:49.911661 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5db3f8bf-d0a6-4881-a1bd-82a4cb13936e None None] REQUEST_METHOD: `GET` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:18:49.912243 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5db3f8bf-d0a6-4881-a1bd-82a4cb13936e None None] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:18:49.912243 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5db3f8bf-d0a6-4881-a1bd-82a4cb13936e None None] PATH_INFO: `/` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:18:49.913593 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 587/1170] 10.222.0.47 () {54 vars in 926 bytes} [Thu Jul 21 08:18:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 08:18:49.918947 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d40f1547-cff2-43ba-aae3-5e223665ae64 None None] REQUEST_METHOD: `POST` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:18:49.919359 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d40f1547-cff2-43ba-aae3-5e223665ae64 None None] SCRIPT_NAME: `/identity` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:18:49.919548 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d40f1547-cff2-43ba-aae3-5e223665ae64 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:18:49.936318 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.auth.core [None req-d40f1547-cff2-43ba-aae3-5e223665ae64 None None] MFA Rules not processed for user `2dce40b8d74c41ec94cc36e6ff05d412`. Rule list: `[]` (Enabled: `True`). {{(pid=80245) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 08:18:49.937492 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.fernet_utils [None req-d40f1547-cff2-43ba-aae3-5e223665ae64 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80245) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 08:18:50.004783 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: [pid: 80245|app: 0|req: 584/1171] 10.222.0.47 () {58 vars in 1023 bytes} [Thu Jul 21 08:18:49 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 21 08:18:50.028181 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5db3f8bf-d0a6-4881-a1bd-82a4cb13936e None None] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:18:50.127966 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5db3f8bf-d0a6-4881-a1bd-82a4cb13936e None None] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:18:50.160396 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67a57bc1-6ac1-4e2e-b9ab-8d556ff70eb1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:18:50.161250 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67a57bc1-6ac1-4e2e-b9ab-8d556ff70eb1 None admin] REQUEST_METHOD: `GET` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:18:50.161534 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67a57bc1-6ac1-4e2e-b9ab-8d556ff70eb1 None admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:18:50.161858 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67a57bc1-6ac1-4e2e-b9ab-8d556ff70eb1 None admin] PATH_INFO: `/v3/domains/6764629114d7452dbfeb95a2920ee178` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:18:50.172088 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67a57bc1-6ac1-4e2e-b9ab-8d556ff70eb1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=6764629114d7452dbfeb95a2920ee178)` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:18:50.174045 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67a57bc1-6ac1-4e2e-b9ab-8d556ff70eb1 None admin] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:18:50.175697 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 588/1172] 10.222.0.47 () {56 vars in 1183 bytes} [Thu Jul 21 08:18:50 2022] GET /identity/v3/domains/6764629114d7452dbfeb95a2920ee178 => generated 267 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 08:18:50.181705 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d40f1547-cff2-43ba-aae3-5e223665ae64 None None] Authenticating user token {{(pid=80245) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:18:50.263972 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG dbcounter [-] [80245] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80245) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:18:50.280005 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d40f1547-cff2-43ba-aae3-5e223665ae64 None None] Validating token access rules against request {{(pid=80245) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:18:50.313450 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ce28ae6-cd64-4cfd-b311-f8d8d6d51815 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80245) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:18:50.314128 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ce28ae6-cd64-4cfd-b311-f8d8d6d51815 None admin] REQUEST_METHOD: `POST` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:18:50.314396 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ce28ae6-cd64-4cfd-b311-f8d8d6d51815 None admin] SCRIPT_NAME: `/identity` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:18:50.314716 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ce28ae6-cd64-4cfd-b311-f8d8d6d51815 None admin] PATH_INFO: `/v3/projects` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:18:50.315561 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ce28ae6-cd64-4cfd-b311-f8d8d6d51815 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:18:50.316986 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ce28ae6-cd64-4cfd-b311-f8d8d6d51815 None admin] RBAC: Authorization granted {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:18:50.340813 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: [pid: 80245|app: 0|req: 585/1173] 10.222.0.47 () {60 vars in 1141 bytes} [Thu Jul 21 08:18:50 2022] POST /identity/v3/projects => generated 361 bytes in 160 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 08:18:52.409840 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea7cce32-5b36-4c84-9605-4efd51c30d53 None None] REQUEST_METHOD: `GET` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:18:52.410403 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea7cce32-5b36-4c84-9605-4efd51c30d53 None None] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:18:52.410403 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea7cce32-5b36-4c84-9605-4efd51c30d53 None None] PATH_INFO: `/` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:18:52.411807 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 589/1174] 10.222.0.47 () {54 vars in 926 bytes} [Thu Jul 21 08:18:52 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 08:18:52.417012 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d8eba51-668b-40e0-be0f-f59653fe43a1 None None] REQUEST_METHOD: `POST` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:18:52.417420 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d8eba51-668b-40e0-be0f-f59653fe43a1 None None] SCRIPT_NAME: `/identity` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:18:52.417579 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d8eba51-668b-40e0-be0f-f59653fe43a1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:18:52.433935 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.auth.core [None req-9d8eba51-668b-40e0-be0f-f59653fe43a1 None None] MFA Rules not processed for user `2dce40b8d74c41ec94cc36e6ff05d412`. Rule list: `[]` (Enabled: `True`). {{(pid=80245) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 08:18:52.435111 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.fernet_utils [None req-9d8eba51-668b-40e0-be0f-f59653fe43a1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80245) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 08:18:52.504062 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: [pid: 80245|app: 0|req: 586/1175] 10.222.0.47 () {58 vars in 1023 bytes} [Thu Jul 21 08:18:52 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 21 08:18:52.528893 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea7cce32-5b36-4c84-9605-4efd51c30d53 None None] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:18:52.569265 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG dbcounter [-] [80244] Writing DB stats keystone:SELECT=100 {{(pid=80244) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:18:52.628912 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea7cce32-5b36-4c84-9605-4efd51c30d53 None None] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:18:52.664881 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21fe52e3-b791-4fc5-9347-574217f066c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:18:52.665674 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21fe52e3-b791-4fc5-9347-574217f066c2 None admin] REQUEST_METHOD: `GET` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:18:52.665959 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21fe52e3-b791-4fc5-9347-574217f066c2 None admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:18:52.666239 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21fe52e3-b791-4fc5-9347-574217f066c2 None admin] PATH_INFO: `/v3/domains/6764629114d7452dbfeb95a2920ee178` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:18:52.667461 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21fe52e3-b791-4fc5-9347-574217f066c2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=6764629114d7452dbfeb95a2920ee178)` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:18:52.668903 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21fe52e3-b791-4fc5-9347-574217f066c2 None admin] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:18:52.670529 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 590/1176] 10.222.0.47 () {56 vars in 1183 bytes} [Thu Jul 21 08:18:52 2022] GET /identity/v3/domains/6764629114d7452dbfeb95a2920ee178 => generated 267 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 08:18:52.676443 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d8eba51-668b-40e0-be0f-f59653fe43a1 None None] Authenticating user token {{(pid=80245) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:18:52.739497 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG dbcounter [-] [80245] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80245) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:18:52.773846 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d8eba51-668b-40e0-be0f-f59653fe43a1 None None] Validating token access rules against request {{(pid=80245) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:18:52.805500 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b34696cd-1910-471d-bec8-bfa1142e3f5f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80245) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:18:52.806159 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b34696cd-1910-471d-bec8-bfa1142e3f5f None admin] REQUEST_METHOD: `POST` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:18:52.806441 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b34696cd-1910-471d-bec8-bfa1142e3f5f None admin] SCRIPT_NAME: `/identity` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:18:52.806716 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b34696cd-1910-471d-bec8-bfa1142e3f5f None admin] PATH_INFO: `/v3/users` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:18:52.807435 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b34696cd-1910-471d-bec8-bfa1142e3f5f None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:18:52.808858 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b34696cd-1910-471d-bec8-bfa1142e3f5f None admin] RBAC: Authorization granted {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:18:52.828999 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: [pid: 80245|app: 0|req: 587/1177] 10.222.0.47 () {60 vars in 1132 bytes} [Thu Jul 21 08:18:52 2022] POST /identity/v3/users => generated 311 bytes in 153 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 08:18:54.896128 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7779794e-0236-426c-b0d0-05d91d8213d1 None None] REQUEST_METHOD: `GET` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:18:54.896659 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7779794e-0236-426c-b0d0-05d91d8213d1 None None] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:18:54.896659 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7779794e-0236-426c-b0d0-05d91d8213d1 None None] PATH_INFO: `/` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:18:54.898037 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 591/1178] 10.222.0.47 () {54 vars in 926 bytes} [Thu Jul 21 08:18:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 08:18:54.903455 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6565328a-d9b4-485c-ab58-b9021f4b4e7b None None] REQUEST_METHOD: `POST` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:18:54.903841 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6565328a-d9b4-485c-ab58-b9021f4b4e7b None None] SCRIPT_NAME: `/identity` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:18:54.904023 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6565328a-d9b4-485c-ab58-b9021f4b4e7b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:18:54.920891 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.auth.core [None req-6565328a-d9b4-485c-ab58-b9021f4b4e7b None None] MFA Rules not processed for user `2dce40b8d74c41ec94cc36e6ff05d412`. Rule list: `[]` (Enabled: `True`). {{(pid=80245) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 08:18:54.922079 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.fernet_utils [None req-6565328a-d9b4-485c-ab58-b9021f4b4e7b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80245) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 08:18:54.987125 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: [pid: 80245|app: 0|req: 588/1179] 10.222.0.47 () {58 vars in 1023 bytes} [Thu Jul 21 08:18:54 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 21 08:18:55.011645 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7779794e-0236-426c-b0d0-05d91d8213d1 None None] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:18:55.112078 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7779794e-0236-426c-b0d0-05d91d8213d1 None None] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:18:55.120068 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG dbcounter [-] [80244] Writing DB stats keystone:SELECT=100 {{(pid=80244) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:18:55.147434 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bb1fb20-a6b0-4221-82b3-22ace2546956 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:18:55.148168 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bb1fb20-a6b0-4221-82b3-22ace2546956 None admin] REQUEST_METHOD: `GET` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:18:55.148448 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bb1fb20-a6b0-4221-82b3-22ace2546956 None admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:18:55.148721 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bb1fb20-a6b0-4221-82b3-22ace2546956 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:18:55.159786 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bb1fb20-a6b0-4221-82b3-22ace2546956 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 21 08:18:55.167677 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application [None req-0bb1fb20-a6b0-4221-82b3-22ace2546956 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 21 08:18:55.167677 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 08:18:55.167677 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 08:18:55.167677 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 08:18:55.167677 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 08:18:55.167677 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 08:18:55.167677 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 08:18:55.167677 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 08:18:55.167677 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 08:18:55.167677 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 08:18:55.167677 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 08:18:55.167677 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 08:18:55.167677 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 21 08:18:55.167677 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 21 08:18:55.167677 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 21 08:18:55.167677 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 21 08:18:55.168845 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 21 08:18:55.168845 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 21 08:18:55.168845 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 21 08:18:55.168845 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 21 08:18:55.168845 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 08:18:55.168845 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 08:18:55.168845 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 08:18:55.168845 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 08:18:55.168845 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 08:18:55.168845 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 08:18:55.168845 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 08:18:55.168845 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application with Lock( Jul 21 08:18:55.168845 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 08:18:55.168845 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self._enter() Jul 21 08:18:55.168845 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 08:18:55.168845 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 08:18:55.169923 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 08:18:55.169923 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self.creator() Jul 21 08:18:55.169923 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 08:18:55.169923 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application created_value = creator( Jul 21 08:18:55.169923 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 21 08:18:55.169923 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 21 08:18:55.169923 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 21 08:18:55.169923 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 21 08:18:55.169923 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 21 08:18:55.169923 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 21 08:18:55.169923 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 21 08:18:55.169923 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application Jul 21 08:18:55.169923 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 592/1180] 10.222.0.47 () {56 vars in 1096 bytes} [Thu Jul 21 08:18:55 2022] GET /identity/v3/roles/admin => generated 83 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 08:18:55.176976 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6565328a-d9b4-485c-ab58-b9021f4b4e7b None None] Authenticating user token {{(pid=80245) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:18:55.231019 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG dbcounter [-] [80245] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80245) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:18:55.275247 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6565328a-d9b4-485c-ab58-b9021f4b4e7b None None] Validating token access rules against request {{(pid=80245) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:18:55.307775 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-414fb9fb-7f4b-44d3-8938-b93efdb10688 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80245) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:18:55.308443 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-414fb9fb-7f4b-44d3-8938-b93efdb10688 None admin] REQUEST_METHOD: `GET` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:18:55.308727 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-414fb9fb-7f4b-44d3-8938-b93efdb10688 None admin] SCRIPT_NAME: `/identity` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:18:55.309004 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-414fb9fb-7f4b-44d3-8938-b93efdb10688 None admin] PATH_INFO: `/v3/roles` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:18:55.309746 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-414fb9fb-7f4b-44d3-8938-b93efdb10688 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80245) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 08:18:55.310049 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-414fb9fb-7f4b-44d3-8938-b93efdb10688 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:18:55.311431 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-414fb9fb-7f4b-44d3-8938-b93efdb10688 None admin] RBAC: Authorization granted {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:18:55.319285 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: [pid: 80245|app: 0|req: 589/1181] 10.222.0.47 () {56 vars in 1099 bytes} [Thu Jul 21 08:18:55 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 08:18:55.325185 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bb1fb20-a6b0-4221-82b3-22ace2546956 None admin] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:18:55.423854 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bb1fb20-a6b0-4221-82b3-22ace2546956 None admin] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:18:55.456429 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11343abe-3d21-420f-8460-f3051f73552b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:18:55.457293 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11343abe-3d21-420f-8460-f3051f73552b None admin] REQUEST_METHOD: `GET` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:18:55.457576 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11343abe-3d21-420f-8460-f3051f73552b None admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:18:55.457854 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11343abe-3d21-420f-8460-f3051f73552b None admin] PATH_INFO: `/v3/users/b0eb3403d0224d2a953434db9efd2731` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:18:55.471125 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11343abe-3d21-420f-8460-f3051f73552b None admin] RBAC: Authorizing `identity:get_user(user_id=b0eb3403d0224d2a953434db9efd2731)` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:18:55.472618 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11343abe-3d21-420f-8460-f3051f73552b None admin] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:18:55.474293 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 593/1182] 10.222.0.47 () {56 vars in 1177 bytes} [Thu Jul 21 08:18:55 2022] GET /identity/v3/users/b0eb3403d0224d2a953434db9efd2731 => generated 311 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 08:18:55.480104 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-414fb9fb-7f4b-44d3-8938-b93efdb10688 None admin] Authenticating user token {{(pid=80245) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:18:55.580304 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-414fb9fb-7f4b-44d3-8938-b93efdb10688 None admin] Validating token access rules against request {{(pid=80245) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:18:55.596353 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG dbcounter [-] [80245] Writing DB stats keystone:SELECT=100 {{(pid=80245) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:18:55.614563 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b3d6fc3-7729-4d34-a1bd-88f607d73613 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80245) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:18:55.615218 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b3d6fc3-7729-4d34-a1bd-88f607d73613 None admin] REQUEST_METHOD: `GET` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:18:55.615533 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b3d6fc3-7729-4d34-a1bd-88f607d73613 None admin] SCRIPT_NAME: `/identity` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:18:55.615812 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b3d6fc3-7729-4d34-a1bd-88f607d73613 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:18:55.680514 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b3d6fc3-7729-4d34-a1bd-88f607d73613 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:18:55.681886 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b3d6fc3-7729-4d34-a1bd-88f607d73613 None admin] RBAC: Authorization granted {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:18:55.775904 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: [pid: 80245|app: 0|req: 590/1183] 10.222.0.47 () {58 vars in 1282 bytes} [Thu Jul 21 08:18:55 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 296 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 21 08:18:55.781871 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11343abe-3d21-420f-8460-f3051f73552b None admin] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:18:55.807104 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG dbcounter [-] [80244] Writing DB stats keystone:SELECT=100 {{(pid=80244) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:18:55.880774 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11343abe-3d21-420f-8460-f3051f73552b None admin] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:18:55.915306 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f019f8dd-f9ed-46e8-a26a-73e7dd159bc2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:18:55.916198 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f019f8dd-f9ed-46e8-a26a-73e7dd159bc2 None admin] REQUEST_METHOD: `GET` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:18:55.916481 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f019f8dd-f9ed-46e8-a26a-73e7dd159bc2 None admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:18:55.916756 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f019f8dd-f9ed-46e8-a26a-73e7dd159bc2 None admin] PATH_INFO: `/v3/projects/f8e16722c9dd4be29c966f2601948c5a` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:18:55.917920 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f019f8dd-f9ed-46e8-a26a-73e7dd159bc2 None admin] RBAC: Authorizing `identity:get_project(project_id=f8e16722c9dd4be29c966f2601948c5a)` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:18:55.919366 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f019f8dd-f9ed-46e8-a26a-73e7dd159bc2 None admin] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:18:55.921046 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 594/1184] 10.222.0.47 () {56 vars in 1186 bytes} [Thu Jul 21 08:18:55 2022] GET /identity/v3/projects/f8e16722c9dd4be29c966f2601948c5a => generated 361 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 08:18:55.927371 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b3d6fc3-7729-4d34-a1bd-88f607d73613 None admin] Authenticating user token {{(pid=80245) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:18:55.944377 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG dbcounter [-] [80245] Writing DB stats keystone:SELECT=100 {{(pid=80245) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:18:56.024372 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b3d6fc3-7729-4d34-a1bd-88f607d73613 None admin] Validating token access rules against request {{(pid=80245) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:18:56.057193 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f73b291-d84f-47f5-93b8-9264db62628d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80245) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:18:56.057836 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f73b291-d84f-47f5-93b8-9264db62628d None admin] REQUEST_METHOD: `GET` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:18:56.058125 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f73b291-d84f-47f5-93b8-9264db62628d None admin] SCRIPT_NAME: `/identity` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:18:56.058398 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f73b291-d84f-47f5-93b8-9264db62628d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:18:56.059211 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f73b291-d84f-47f5-93b8-9264db62628d None admin] RBAC: Adding query filter params (role.id=e29870f6e9e2411a9db0d5bf45f32ca5, scope.project.id=f8e16722c9dd4be29c966f2601948c5a, user.id=b0eb3403d0224d2a953434db9efd2731) {{(pid=80245) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 08:18:56.059549 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f73b291-d84f-47f5-93b8-9264db62628d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:18:56.060981 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f73b291-d84f-47f5-93b8-9264db62628d None admin] RBAC: Authorization granted {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:18:56.066447 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: [pid: 80245|app: 0|req: 591/1185] 10.222.0.47 () {56 vars in 1374 bytes} [Thu Jul 21 08:18:55 2022] GET /identity/v3/role_assignments?user.id=b0eb3403d0224d2a953434db9efd2731&scope.project.id=f8e16722c9dd4be29c966f2601948c5a&role.id=e29870f6e9e2411a9db0d5bf45f32ca5 => generated 259 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 08:18:58.050873 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdf214dd-be52-43a5-9eea-95d6b10ee12c None None] REQUEST_METHOD: `GET` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:18:58.051448 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdf214dd-be52-43a5-9eea-95d6b10ee12c None None] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:18:58.051448 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdf214dd-be52-43a5-9eea-95d6b10ee12c None None] PATH_INFO: `/` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:18:58.052815 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 595/1186] 10.222.0.47 () {54 vars in 926 bytes} [Thu Jul 21 08:18:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 08:18:58.058046 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62b5bc7e-f4a0-4fe5-8236-9f4621a7e44d None None] REQUEST_METHOD: `POST` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:18:58.058445 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62b5bc7e-f4a0-4fe5-8236-9f4621a7e44d None None] SCRIPT_NAME: `/identity` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:18:58.058602 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62b5bc7e-f4a0-4fe5-8236-9f4621a7e44d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:18:58.075383 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.auth.core [None req-62b5bc7e-f4a0-4fe5-8236-9f4621a7e44d None None] MFA Rules not processed for user `2dce40b8d74c41ec94cc36e6ff05d412`. Rule list: `[]` (Enabled: `True`). {{(pid=80245) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 08:18:58.076561 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.fernet_utils [None req-62b5bc7e-f4a0-4fe5-8236-9f4621a7e44d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80245) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 08:18:58.141693 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: [pid: 80245|app: 0|req: 592/1187] 10.222.0.47 () {58 vars in 1023 bytes} [Thu Jul 21 08:18:58 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 21 08:18:58.164046 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdf214dd-be52-43a5-9eea-95d6b10ee12c None None] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:18:58.255295 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG dbcounter [-] [80244] Writing DB stats keystone:SELECT=100 {{(pid=80244) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:18:58.265498 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdf214dd-be52-43a5-9eea-95d6b10ee12c None None] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:18:58.297805 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91cb9c4e-8e4d-4caa-ac30-4b14e147244e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:18:58.299085 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91cb9c4e-8e4d-4caa-ac30-4b14e147244e None admin] REQUEST_METHOD: `GET` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:18:58.299396 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91cb9c4e-8e4d-4caa-ac30-4b14e147244e None admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:18:58.299668 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91cb9c4e-8e4d-4caa-ac30-4b14e147244e None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:18:58.309514 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91cb9c4e-8e4d-4caa-ac30-4b14e147244e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 21 08:18:58.317477 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application [None req-91cb9c4e-8e4d-4caa-ac30-4b14e147244e None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 21 08:18:58.317477 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 08:18:58.317477 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 08:18:58.317477 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 08:18:58.317477 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 08:18:58.317477 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 08:18:58.317477 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 08:18:58.317477 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 08:18:58.317477 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 08:18:58.317477 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 08:18:58.317477 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 08:18:58.317477 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 08:18:58.317477 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 21 08:18:58.317477 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 21 08:18:58.317477 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 21 08:18:58.317477 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 21 08:18:58.318373 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 21 08:18:58.318373 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 21 08:18:58.318373 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 21 08:18:58.318373 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 21 08:18:58.318373 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 08:18:58.318373 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 08:18:58.318373 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 08:18:58.318373 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 08:18:58.318373 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 08:18:58.318373 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 08:18:58.318373 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 08:18:58.318373 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application with Lock( Jul 21 08:18:58.318373 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 08:18:58.318373 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self._enter() Jul 21 08:18:58.318373 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 08:18:58.318373 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 08:18:58.319447 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 08:18:58.319447 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self.creator() Jul 21 08:18:58.319447 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 08:18:58.319447 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application created_value = creator( Jul 21 08:18:58.319447 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 21 08:18:58.319447 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 21 08:18:58.319447 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 21 08:18:58.319447 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 21 08:18:58.319447 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 21 08:18:58.319447 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 21 08:18:58.319447 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 21 08:18:58.319447 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application Jul 21 08:18:58.319447 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 596/1188] 10.222.0.47 () {56 vars in 1096 bytes} [Thu Jul 21 08:18:58 2022] GET /identity/v3/roles/admin => generated 83 bytes in 155 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 08:18:58.324884 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62b5bc7e-f4a0-4fe5-8236-9f4621a7e44d None None] Authenticating user token {{(pid=80245) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:18:58.336175 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG dbcounter [-] [80245] Writing DB stats keystone:SELECT=100 {{(pid=80245) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:18:58.424145 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62b5bc7e-f4a0-4fe5-8236-9f4621a7e44d None None] Validating token access rules against request {{(pid=80245) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:18:58.458461 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc7b44d5-b2f8-45f6-9c87-e2ddd5372c26 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80245) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:18:58.459137 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc7b44d5-b2f8-45f6-9c87-e2ddd5372c26 None admin] REQUEST_METHOD: `GET` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:18:58.459458 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc7b44d5-b2f8-45f6-9c87-e2ddd5372c26 None admin] SCRIPT_NAME: `/identity` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:18:58.459739 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc7b44d5-b2f8-45f6-9c87-e2ddd5372c26 None admin] PATH_INFO: `/v3/roles` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:18:58.460483 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc7b44d5-b2f8-45f6-9c87-e2ddd5372c26 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80245) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 08:18:58.460790 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc7b44d5-b2f8-45f6-9c87-e2ddd5372c26 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:18:58.462155 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc7b44d5-b2f8-45f6-9c87-e2ddd5372c26 None admin] RBAC: Authorization granted {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:18:58.470847 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: [pid: 80245|app: 0|req: 593/1189] 10.222.0.47 () {56 vars in 1099 bytes} [Thu Jul 21 08:18:58 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 08:18:58.476660 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91cb9c4e-8e4d-4caa-ac30-4b14e147244e None admin] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:18:58.575431 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91cb9c4e-8e4d-4caa-ac30-4b14e147244e None admin] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:18:58.608365 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4604a486-1ebb-4f1d-a175-f6a473b7a7a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:18:58.609230 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4604a486-1ebb-4f1d-a175-f6a473b7a7a0 None admin] REQUEST_METHOD: `GET` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:18:58.609514 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4604a486-1ebb-4f1d-a175-f6a473b7a7a0 None admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:18:58.609789 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4604a486-1ebb-4f1d-a175-f6a473b7a7a0 None admin] PATH_INFO: `/v3/users/b0eb3403d0224d2a953434db9efd2731` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:18:58.611601 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4604a486-1ebb-4f1d-a175-f6a473b7a7a0 None admin] RBAC: Authorizing `identity:get_user(user_id=b0eb3403d0224d2a953434db9efd2731)` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:18:58.613035 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4604a486-1ebb-4f1d-a175-f6a473b7a7a0 None admin] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:18:58.614666 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 597/1190] 10.222.0.47 () {56 vars in 1177 bytes} [Thu Jul 21 08:18:58 2022] GET /identity/v3/users/b0eb3403d0224d2a953434db9efd2731 => generated 311 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 08:18:58.621264 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc7b44d5-b2f8-45f6-9c87-e2ddd5372c26 None admin] Authenticating user token {{(pid=80245) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:18:58.689426 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG dbcounter [-] [80245] Writing DB stats keystone:SELECT=100 {{(pid=80245) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:18:58.719732 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc7b44d5-b2f8-45f6-9c87-e2ddd5372c26 None admin] Validating token access rules against request {{(pid=80245) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:18:58.752848 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b9efb50-232c-4d83-a144-abb17f218ec4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80245) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:18:58.753706 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b9efb50-232c-4d83-a144-abb17f218ec4 None admin] REQUEST_METHOD: `GET` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:18:58.753988 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b9efb50-232c-4d83-a144-abb17f218ec4 None admin] SCRIPT_NAME: `/identity` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:18:58.754258 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b9efb50-232c-4d83-a144-abb17f218ec4 None admin] PATH_INFO: `/v3/projects/f8e16722c9dd4be29c966f2601948c5a` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:18:58.755456 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b9efb50-232c-4d83-a144-abb17f218ec4 None admin] RBAC: Authorizing `identity:get_project(project_id=f8e16722c9dd4be29c966f2601948c5a)` {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:18:58.756877 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b9efb50-232c-4d83-a144-abb17f218ec4 None admin] RBAC: Authorization granted {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:18:58.758950 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: [pid: 80245|app: 0|req: 594/1191] 10.222.0.47 () {56 vars in 1186 bytes} [Thu Jul 21 08:18:58 2022] GET /identity/v3/projects/f8e16722c9dd4be29c966f2601948c5a => generated 361 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 08:18:58.765436 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4604a486-1ebb-4f1d-a175-f6a473b7a7a0 None admin] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:18:58.787571 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG dbcounter [-] [80244] Writing DB stats keystone:SELECT=100 {{(pid=80244) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:18:58.864269 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4604a486-1ebb-4f1d-a175-f6a473b7a7a0 None admin] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:18:58.897489 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbb7b531-3473-46dd-9998-06bc59d7e829 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:18:58.898383 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbb7b531-3473-46dd-9998-06bc59d7e829 None admin] REQUEST_METHOD: `PUT` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:18:58.898664 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbb7b531-3473-46dd-9998-06bc59d7e829 None admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:18:58.898942 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbb7b531-3473-46dd-9998-06bc59d7e829 None admin] PATH_INFO: `/v3/projects/f8e16722c9dd4be29c966f2601948c5a/users/b0eb3403d0224d2a953434db9efd2731/roles/e29870f6e9e2411a9db0d5bf45f32ca5` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:18:58.901126 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbb7b531-3473-46dd-9998-06bc59d7e829 None admin] RBAC: Authorizing `identity:create_grant(project_id=f8e16722c9dd4be29c966f2601948c5a, user_id=b0eb3403d0224d2a953434db9efd2731, role_id=e29870f6e9e2411a9db0d5bf45f32ca5)` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:18:58.902565 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbb7b531-3473-46dd-9998-06bc59d7e829 None admin] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:18:58.916362 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 598/1192] 10.222.0.47 () {58 vars in 1439 bytes} [Thu Jul 21 08:18:58 2022] PUT /identity/v3/projects/f8e16722c9dd4be29c966f2601948c5a/users/b0eb3403d0224d2a953434db9efd2731/roles/e29870f6e9e2411a9db0d5bf45f32ca5 => generated 0 bytes in 151 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 08:19:00.925264 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ab057dd-147a-4d8e-981d-7284bc579250 None None] REQUEST_METHOD: `GET` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:19:00.925820 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ab057dd-147a-4d8e-981d-7284bc579250 None None] SCRIPT_NAME: `/identity` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:19:00.925820 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ab057dd-147a-4d8e-981d-7284bc579250 None None] PATH_INFO: `/` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:19:00.927204 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: [pid: 80245|app: 0|req: 595/1193] 10.222.0.47 () {54 vars in 926 bytes} [Thu Jul 21 08:19:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 08:19:00.932317 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14425cb2-fa3b-43f7-a8b8-1049a6b7db3f None None] REQUEST_METHOD: `POST` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:19:00.932706 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14425cb2-fa3b-43f7-a8b8-1049a6b7db3f None None] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:19:00.932880 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14425cb2-fa3b-43f7-a8b8-1049a6b7db3f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:19:00.950316 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.auth.core [None req-14425cb2-fa3b-43f7-a8b8-1049a6b7db3f None None] MFA Rules not processed for user `2dce40b8d74c41ec94cc36e6ff05d412`. Rule list: `[]` (Enabled: `True`). {{(pid=80244) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 08:19:00.951527 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.fernet_utils [None req-14425cb2-fa3b-43f7-a8b8-1049a6b7db3f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80244) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 08:19:01.018080 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 599/1194] 10.222.0.47 () {58 vars in 1023 bytes} [Thu Jul 21 08:19:00 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 21 08:19:01.041598 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ab057dd-147a-4d8e-981d-7284bc579250 None None] Authenticating user token {{(pid=80245) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:19:01.148880 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ab057dd-147a-4d8e-981d-7284bc579250 None None] Validating token access rules against request {{(pid=80245) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:19:01.182184 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45876cc1-22e7-4cad-b151-b09dd6982b00 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80245) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:19:01.182900 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45876cc1-22e7-4cad-b151-b09dd6982b00 None admin] REQUEST_METHOD: `GET` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:19:01.183184 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45876cc1-22e7-4cad-b151-b09dd6982b00 None admin] SCRIPT_NAME: `/identity` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:19:01.183492 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45876cc1-22e7-4cad-b151-b09dd6982b00 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:19:01.191722 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG dbcounter [-] [80245] Writing DB stats keystone:SELECT=100 {{(pid=80245) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:19:01.195555 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45876cc1-22e7-4cad-b151-b09dd6982b00 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 21 08:19:01.203309 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application [None req-45876cc1-22e7-4cad-b151-b09dd6982b00 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 21 08:19:01.203309 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 08:19:01.203309 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 08:19:01.203309 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 08:19:01.203309 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 08:19:01.203309 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 08:19:01.203309 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 08:19:01.203309 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 08:19:01.203309 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 08:19:01.203309 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 08:19:01.203309 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 08:19:01.203309 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 08:19:01.203309 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 21 08:19:01.203309 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 21 08:19:01.203309 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 21 08:19:01.203309 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 21 08:19:01.204437 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 21 08:19:01.204437 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 21 08:19:01.204437 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 21 08:19:01.204437 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 21 08:19:01.204437 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 08:19:01.204437 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 08:19:01.204437 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 08:19:01.204437 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 08:19:01.204437 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 08:19:01.204437 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 08:19:01.204437 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 08:19:01.204437 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application with Lock( Jul 21 08:19:01.204437 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 08:19:01.204437 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application return self._enter() Jul 21 08:19:01.204437 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 08:19:01.204437 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 08:19:01.205440 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 08:19:01.205440 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application return self.creator() Jul 21 08:19:01.205440 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 08:19:01.205440 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application created_value = creator( Jul 21 08:19:01.205440 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 21 08:19:01.205440 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 21 08:19:01.205440 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 21 08:19:01.205440 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 21 08:19:01.205440 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 21 08:19:01.205440 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 21 08:19:01.205440 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 21 08:19:01.205440 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application Jul 21 08:19:01.205440 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: [pid: 80245|app: 0|req: 596/1195] 10.222.0.47 () {56 vars in 1096 bytes} [Thu Jul 21 08:19:01 2022] GET /identity/v3/roles/admin => generated 83 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 08:19:01.210691 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14425cb2-fa3b-43f7-a8b8-1049a6b7db3f None None] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:19:01.222449 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG dbcounter [-] [80244] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80244) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:19:01.311360 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14425cb2-fa3b-43f7-a8b8-1049a6b7db3f None None] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:19:01.345132 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-820e0e88-9c95-44a4-9b70-242ef45bd201 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:19:01.345804 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-820e0e88-9c95-44a4-9b70-242ef45bd201 None admin] REQUEST_METHOD: `GET` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:19:01.346085 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-820e0e88-9c95-44a4-9b70-242ef45bd201 None admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:19:01.346357 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-820e0e88-9c95-44a4-9b70-242ef45bd201 None admin] PATH_INFO: `/v3/roles` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:19:01.347104 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-820e0e88-9c95-44a4-9b70-242ef45bd201 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80244) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 08:19:01.347443 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-820e0e88-9c95-44a4-9b70-242ef45bd201 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:19:01.348815 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-820e0e88-9c95-44a4-9b70-242ef45bd201 None admin] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:19:01.357561 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 600/1196] 10.222.0.47 () {56 vars in 1099 bytes} [Thu Jul 21 08:19:01 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 08:19:01.363938 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45876cc1-22e7-4cad-b151-b09dd6982b00 None admin] Authenticating user token {{(pid=80245) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:19:01.465660 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45876cc1-22e7-4cad-b151-b09dd6982b00 None admin] Validating token access rules against request {{(pid=80245) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:19:01.500588 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fdf996a-dd6a-4144-81eb-f5fe78dc733a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80245) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:19:01.501476 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fdf996a-dd6a-4144-81eb-f5fe78dc733a None admin] REQUEST_METHOD: `GET` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:19:01.501755 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fdf996a-dd6a-4144-81eb-f5fe78dc733a None admin] SCRIPT_NAME: `/identity` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:19:01.502030 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fdf996a-dd6a-4144-81eb-f5fe78dc733a None admin] PATH_INFO: `/v3/users/b0eb3403d0224d2a953434db9efd2731` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:19:01.503209 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fdf996a-dd6a-4144-81eb-f5fe78dc733a None admin] RBAC: Authorizing `identity:get_user(user_id=b0eb3403d0224d2a953434db9efd2731)` {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:19:01.504663 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fdf996a-dd6a-4144-81eb-f5fe78dc733a None admin] RBAC: Authorization granted {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:19:01.506289 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: [pid: 80245|app: 0|req: 597/1197] 10.222.0.47 () {56 vars in 1177 bytes} [Thu Jul 21 08:19:01 2022] GET /identity/v3/users/b0eb3403d0224d2a953434db9efd2731 => generated 311 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 08:19:01.512334 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-820e0e88-9c95-44a4-9b70-242ef45bd201 None admin] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:19:01.592927 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG dbcounter [-] [80244] Writing DB stats keystone:SELECT=100 {{(pid=80244) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:19:01.623196 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-820e0e88-9c95-44a4-9b70-242ef45bd201 None admin] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:19:01.655425 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6141cb34-f0f1-497d-ab83-d8ad8ab0d680 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:19:01.656082 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6141cb34-f0f1-497d-ab83-d8ad8ab0d680 None admin] REQUEST_METHOD: `GET` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:19:01.656370 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6141cb34-f0f1-497d-ab83-d8ad8ab0d680 None admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:19:01.656638 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6141cb34-f0f1-497d-ab83-d8ad8ab0d680 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:19:01.732930 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6141cb34-f0f1-497d-ab83-d8ad8ab0d680 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:19:01.734322 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6141cb34-f0f1-497d-ab83-d8ad8ab0d680 None admin] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:19:01.805300 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG dbcounter [-] [80244] Writing DB stats keystone:SELECT=100 {{(pid=80244) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:19:01.835617 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 601/1198] 10.222.0.47 () {58 vars in 1282 bytes} [Thu Jul 21 08:19:01 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 324 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 21 08:19:01.841633 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fdf996a-dd6a-4144-81eb-f5fe78dc733a None admin] Authenticating user token {{(pid=80245) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:19:01.915587 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG dbcounter [-] [80245] Writing DB stats keystone:SELECT=100 {{(pid=80245) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:19:01.947799 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fdf996a-dd6a-4144-81eb-f5fe78dc733a None admin] Validating token access rules against request {{(pid=80245) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:19:01.981580 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9a81e5d-f3f0-46a7-9935-db1be149baf7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80245) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:19:01.982428 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9a81e5d-f3f0-46a7-9935-db1be149baf7 None admin] REQUEST_METHOD: `GET` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:19:01.982706 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9a81e5d-f3f0-46a7-9935-db1be149baf7 None admin] SCRIPT_NAME: `/identity` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:19:01.983023 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9a81e5d-f3f0-46a7-9935-db1be149baf7 None admin] PATH_INFO: `/v3/projects/f8e16722c9dd4be29c966f2601948c5a` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:19:01.984159 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9a81e5d-f3f0-46a7-9935-db1be149baf7 None admin] RBAC: Authorizing `identity:get_project(project_id=f8e16722c9dd4be29c966f2601948c5a)` {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:19:01.985606 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9a81e5d-f3f0-46a7-9935-db1be149baf7 None admin] RBAC: Authorization granted {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:19:01.987296 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: [pid: 80245|app: 0|req: 598/1199] 10.222.0.47 () {56 vars in 1186 bytes} [Thu Jul 21 08:19:01 2022] GET /identity/v3/projects/f8e16722c9dd4be29c966f2601948c5a => generated 361 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 08:19:01.993741 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6141cb34-f0f1-497d-ab83-d8ad8ab0d680 None admin] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:19:02.090866 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6141cb34-f0f1-497d-ab83-d8ad8ab0d680 None admin] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:19:02.129432 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f6038d7-172c-49d3-9bb4-c1b415672c11 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:19:02.130090 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f6038d7-172c-49d3-9bb4-c1b415672c11 None admin] REQUEST_METHOD: `GET` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:19:02.130377 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f6038d7-172c-49d3-9bb4-c1b415672c11 None admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:19:02.130656 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f6038d7-172c-49d3-9bb4-c1b415672c11 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:19:02.131522 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f6038d7-172c-49d3-9bb4-c1b415672c11 None admin] RBAC: Adding query filter params (role.id=e29870f6e9e2411a9db0d5bf45f32ca5, scope.project.id=f8e16722c9dd4be29c966f2601948c5a, user.id=b0eb3403d0224d2a953434db9efd2731) {{(pid=80244) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 08:19:02.131841 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f6038d7-172c-49d3-9bb4-c1b415672c11 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:19:02.133537 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f6038d7-172c-49d3-9bb4-c1b415672c11 None admin] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:19:02.139307 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 602/1200] 10.222.0.47 () {56 vars in 1374 bytes} [Thu Jul 21 08:19:01 2022] GET /identity/v3/role_assignments?user.id=b0eb3403d0224d2a953434db9efd2731&scope.project.id=f8e16722c9dd4be29c966f2601948c5a&role.id=e29870f6e9e2411a9db0d5bf45f32ca5 => generated 608 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 08:19:11.977362 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG dbcounter [-] [80245] Writing DB stats keystone:SELECT=32 {{(pid=80245) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:19:12.136701 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG dbcounter [-] [80244] Writing DB stats keystone:SELECT=82 {{(pid=80244) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:19:57.535555 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da370060-1c37-461a-a352-6fd126002543 None None] REQUEST_METHOD: `GET` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:19:57.536119 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da370060-1c37-461a-a352-6fd126002543 None None] SCRIPT_NAME: `/identity` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:19:57.536119 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da370060-1c37-461a-a352-6fd126002543 None None] PATH_INFO: `/` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:19:57.537557 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: [pid: 80245|app: 0|req: 599/1201] 10.222.0.47 () {54 vars in 926 bytes} [Thu Jul 21 08:19:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 21 08:19:57.547061 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a76e6c79-0744-4d76-b1ee-0a18816fbc15 None None] REQUEST_METHOD: `POST` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:19:57.547061 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a76e6c79-0744-4d76-b1ee-0a18816fbc15 None None] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:19:57.547061 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a76e6c79-0744-4d76-b1ee-0a18816fbc15 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:19:57.565520 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.auth.core [None req-a76e6c79-0744-4d76-b1ee-0a18816fbc15 None None] MFA Rules not processed for user `2dce40b8d74c41ec94cc36e6ff05d412`. Rule list: `[]` (Enabled: `True`). {{(pid=80244) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 08:19:57.566914 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.fernet_utils [None req-a76e6c79-0744-4d76-b1ee-0a18816fbc15 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80244) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 08:19:57.657934 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 603/1202] 10.222.0.47 () {58 vars in 1023 bytes} [Thu Jul 21 08:19:57 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 21 08:19:57.685683 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da370060-1c37-461a-a352-6fd126002543 None None] Authenticating user token {{(pid=80245) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:19:57.829381 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da370060-1c37-461a-a352-6fd126002543 None None] Validating token access rules against request {{(pid=80245) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:19:57.884915 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0981f4b-fa36-43ec-8699-3b5ca88c9072 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80245) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:19:57.885833 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0981f4b-fa36-43ec-8699-3b5ca88c9072 None admin] REQUEST_METHOD: `GET` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:19:57.886141 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0981f4b-fa36-43ec-8699-3b5ca88c9072 None admin] SCRIPT_NAME: `/identity` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:19:57.886463 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0981f4b-fa36-43ec-8699-3b5ca88c9072 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:19:57.901971 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0981f4b-fa36-43ec-8699-3b5ca88c9072 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:19:57.903706 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0981f4b-fa36-43ec-8699-3b5ca88c9072 None admin] RBAC: Authorization granted {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:19:57.926033 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application [None req-b0981f4b-fa36-43ec-8699-3b5ca88c9072 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 08:19:57.926033 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 08:19:57.926033 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 08:19:57.926033 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 08:19:57.926033 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 08:19:57.926033 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 08:19:57.926033 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 08:19:57.926033 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 08:19:57.926033 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 08:19:57.926033 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 08:19:57.926033 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 08:19:57.926033 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 08:19:57.926033 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 21 08:19:57.926033 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 21 08:19:57.926033 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 21 08:19:57.927041 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 21 08:19:57.927041 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 08:19:57.927041 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 08:19:57.927041 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 08:19:57.927041 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 08:19:57.927041 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 08:19:57.927041 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 08:19:57.927041 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 08:19:57.927041 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application with Lock( Jul 21 08:19:57.927041 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 08:19:57.927041 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application return self._enter() Jul 21 08:19:57.927041 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 08:19:57.927041 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 08:19:57.927041 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 08:19:57.927041 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application return self.creator() Jul 21 08:19:57.927041 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 08:19:57.927041 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application created_value = creator( Jul 21 08:19:57.937304 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 21 08:19:57.937304 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 21 08:19:57.937304 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 08:19:57.937304 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application Jul 21 08:19:57.937304 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: [pid: 80245|app: 0|req: 600/1203] 10.222.0.47 () {56 vars in 1108 bytes} [Thu Jul 21 08:19:57 2022] GET /identity/v3/domains/Default => generated 87 bytes in 243 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 08:19:57.937683 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a76e6c79-0744-4d76-b1ee-0a18816fbc15 None None] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:19:58.070784 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a76e6c79-0744-4d76-b1ee-0a18816fbc15 None None] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:19:58.114543 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG dbcounter [-] [80244] Writing DB stats keystone:SELECT=100 {{(pid=80244) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:19:58.122157 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-025c0609-4705-468b-b828-b8053f12bf60 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:19:58.122976 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-025c0609-4705-468b-b828-b8053f12bf60 None admin] REQUEST_METHOD: `GET` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:19:58.123358 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-025c0609-4705-468b-b828-b8053f12bf60 None admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:19:58.123671 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-025c0609-4705-468b-b828-b8053f12bf60 None admin] PATH_INFO: `/v3/domains` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:19:58.125124 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-025c0609-4705-468b-b828-b8053f12bf60 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80244) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 08:19:58.125547 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-025c0609-4705-468b-b828-b8053f12bf60 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:19:58.127067 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-025c0609-4705-468b-b828-b8053f12bf60 None admin] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:19:58.141637 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 604/1204] 10.222.0.47 () {56 vars in 1109 bytes} [Thu Jul 21 08:19:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 08:19:58.147538 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0981f4b-fa36-43ec-8699-3b5ca88c9072 None admin] Authenticating user token {{(pid=80245) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:19:58.234674 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG dbcounter [-] [80245] Writing DB stats keystone:SELECT=100 {{(pid=80245) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:19:58.303577 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0981f4b-fa36-43ec-8699-3b5ca88c9072 None admin] Validating token access rules against request {{(pid=80245) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:19:58.345628 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d061ee7-34e9-464a-ad77-01d147ce04c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80245) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:19:58.346336 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d061ee7-34e9-464a-ad77-01d147ce04c4 None admin] REQUEST_METHOD: `POST` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:19:58.346625 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d061ee7-34e9-464a-ad77-01d147ce04c4 None admin] SCRIPT_NAME: `/identity` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:19:58.346905 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d061ee7-34e9-464a-ad77-01d147ce04c4 None admin] PATH_INFO: `/v3/users` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:19:58.347646 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d061ee7-34e9-464a-ad77-01d147ce04c4 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:19:58.349186 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d061ee7-34e9-464a-ad77-01d147ce04c4 None admin] RBAC: Authorization granted {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:19:58.371187 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: [pid: 80245|app: 0|req: 601/1205] 10.222.0.47 () {60 vars in 1132 bytes} [Thu Jul 21 08:19:58 2022] POST /identity/v3/users => generated 251 bytes in 224 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 08:20:00.723440 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3d68a67-fd7d-4882-9e9f-6dfd2ba667fb None None] REQUEST_METHOD: `GET` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:20:00.723996 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3d68a67-fd7d-4882-9e9f-6dfd2ba667fb None None] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:20:00.724087 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3d68a67-fd7d-4882-9e9f-6dfd2ba667fb None None] PATH_INFO: `/` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:20:00.725818 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 605/1206] 10.222.0.47 () {54 vars in 926 bytes} [Thu Jul 21 08:20:00 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 08:20:00.732981 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-977c1e53-aec0-4911-8861-294734c09d70 None None] REQUEST_METHOD: `POST` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:20:00.732981 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-977c1e53-aec0-4911-8861-294734c09d70 None None] SCRIPT_NAME: `/identity` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:20:00.732981 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-977c1e53-aec0-4911-8861-294734c09d70 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:20:00.753317 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.auth.core [None req-977c1e53-aec0-4911-8861-294734c09d70 None None] MFA Rules not processed for user `2dce40b8d74c41ec94cc36e6ff05d412`. Rule list: `[]` (Enabled: `True`). {{(pid=80245) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 08:20:00.754939 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.fernet_utils [None req-977c1e53-aec0-4911-8861-294734c09d70 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80245) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 08:20:00.863381 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: [pid: 80245|app: 0|req: 602/1207] 10.222.0.47 () {58 vars in 1023 bytes} [Thu Jul 21 08:20:00 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 21 08:20:00.887185 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3d68a67-fd7d-4882-9e9f-6dfd2ba667fb None None] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:20:01.024423 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3d68a67-fd7d-4882-9e9f-6dfd2ba667fb None None] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:20:01.061992 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61ad033d-f63a-4267-ae02-efdce2d6cb6f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:20:01.062809 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61ad033d-f63a-4267-ae02-efdce2d6cb6f None admin] REQUEST_METHOD: `GET` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:20:01.063057 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61ad033d-f63a-4267-ae02-efdce2d6cb6f None admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:20:01.063353 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61ad033d-f63a-4267-ae02-efdce2d6cb6f None admin] PATH_INFO: `/v3/roles/service` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:20:01.085311 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61ad033d-f63a-4267-ae02-efdce2d6cb6f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 21 08:20:01.095663 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application [None req-61ad033d-f63a-4267-ae02-efdce2d6cb6f None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 21 08:20:01.095663 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 08:20:01.095663 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 08:20:01.095663 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 08:20:01.095663 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 08:20:01.095663 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 08:20:01.095663 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 08:20:01.095663 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 08:20:01.095663 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 08:20:01.095663 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 08:20:01.095663 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 08:20:01.095663 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 08:20:01.095663 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 21 08:20:01.095663 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 21 08:20:01.095663 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 21 08:20:01.095663 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 21 08:20:01.097327 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 21 08:20:01.097327 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 21 08:20:01.097327 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 21 08:20:01.097327 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 21 08:20:01.097327 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 08:20:01.097327 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 08:20:01.097327 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 08:20:01.097327 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 08:20:01.097327 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 08:20:01.097327 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 08:20:01.097327 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 08:20:01.097327 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application with Lock( Jul 21 08:20:01.097327 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 08:20:01.097327 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self._enter() Jul 21 08:20:01.097327 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 08:20:01.097327 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 08:20:01.098886 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 08:20:01.098886 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self.creator() Jul 21 08:20:01.098886 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 08:20:01.098886 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application created_value = creator( Jul 21 08:20:01.098886 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 21 08:20:01.098886 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 21 08:20:01.098886 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 21 08:20:01.098886 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 21 08:20:01.098886 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 21 08:20:01.098886 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 21 08:20:01.098886 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 21 08:20:01.098886 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application Jul 21 08:20:01.098886 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 606/1208] 10.222.0.47 () {56 vars in 1102 bytes} [Thu Jul 21 08:20:00 2022] GET /identity/v3/roles/service => generated 85 bytes in 211 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 08:20:01.106335 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-977c1e53-aec0-4911-8861-294734c09d70 None None] Authenticating user token {{(pid=80245) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:20:01.153043 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG dbcounter [-] [80245] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80245) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:20:01.230973 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-977c1e53-aec0-4911-8861-294734c09d70 None None] Validating token access rules against request {{(pid=80245) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:20:01.269925 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8355d967-6135-445a-b9bf-72c809897bf3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80245) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:20:01.270728 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8355d967-6135-445a-b9bf-72c809897bf3 None admin] REQUEST_METHOD: `GET` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:20:01.271017 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8355d967-6135-445a-b9bf-72c809897bf3 None admin] SCRIPT_NAME: `/identity` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:20:01.271422 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8355d967-6135-445a-b9bf-72c809897bf3 None admin] PATH_INFO: `/v3/roles` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:20:01.272199 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8355d967-6135-445a-b9bf-72c809897bf3 None admin] RBAC: Adding query filter params (name=service) {{(pid=80245) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 08:20:01.272508 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8355d967-6135-445a-b9bf-72c809897bf3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:20:01.273914 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8355d967-6135-445a-b9bf-72c809897bf3 None admin] RBAC: Authorization granted {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:20:01.284843 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: [pid: 80245|app: 0|req: 603/1209] 10.222.0.47 () {56 vars in 1103 bytes} [Thu Jul 21 08:20:01 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 08:20:01.290621 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61ad033d-f63a-4267-ae02-efdce2d6cb6f None admin] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:20:01.350823 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG dbcounter [-] [80244] Writing DB stats keystone:SELECT=100 {{(pid=80244) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:20:01.420868 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61ad033d-f63a-4267-ae02-efdce2d6cb6f None admin] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:20:01.480830 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1331bffd-553f-4282-acc6-7b9a59266e6c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:20:01.482011 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1331bffd-553f-4282-acc6-7b9a59266e6c None admin] REQUEST_METHOD: `GET` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:20:01.482506 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1331bffd-553f-4282-acc6-7b9a59266e6c None admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:20:01.482977 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1331bffd-553f-4282-acc6-7b9a59266e6c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:20:01.497290 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1331bffd-553f-4282-acc6-7b9a59266e6c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:20:01.499566 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1331bffd-553f-4282-acc6-7b9a59266e6c None admin] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:20:01.519086 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application [None req-1331bffd-553f-4282-acc6-7b9a59266e6c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 08:20:01.519086 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 08:20:01.519086 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 08:20:01.519086 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 08:20:01.519086 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 08:20:01.519086 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 08:20:01.519086 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 08:20:01.519086 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 08:20:01.519086 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 08:20:01.519086 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 08:20:01.519086 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 08:20:01.519086 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 08:20:01.519086 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 21 08:20:01.519086 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 21 08:20:01.519086 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 21 08:20:01.520220 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 21 08:20:01.520220 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 08:20:01.520220 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 08:20:01.520220 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 08:20:01.520220 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 08:20:01.520220 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 08:20:01.520220 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 08:20:01.520220 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 08:20:01.520220 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application with Lock( Jul 21 08:20:01.520220 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 08:20:01.520220 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self._enter() Jul 21 08:20:01.520220 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 08:20:01.520220 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 08:20:01.520220 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 08:20:01.520220 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self.creator() Jul 21 08:20:01.520220 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 08:20:01.520220 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application created_value = creator( Jul 21 08:20:01.521128 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 21 08:20:01.521128 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 21 08:20:01.521128 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 08:20:01.521128 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application Jul 21 08:20:01.521128 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 607/1210] 10.222.0.47 () {56 vars in 1108 bytes} [Thu Jul 21 08:20:01 2022] GET /identity/v3/domains/Default => generated 87 bytes in 231 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 08:20:01.527012 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8355d967-6135-445a-b9bf-72c809897bf3 None admin] Authenticating user token {{(pid=80245) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:20:01.647688 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG dbcounter [-] [80245] Writing DB stats keystone:SELECT=100 {{(pid=80245) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:20:01.651216 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8355d967-6135-445a-b9bf-72c809897bf3 None admin] Validating token access rules against request {{(pid=80245) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:20:01.690976 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1951566d-bdba-49c0-a803-2421d35706f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80245) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:20:01.691677 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1951566d-bdba-49c0-a803-2421d35706f5 None admin] REQUEST_METHOD: `GET` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:20:01.691964 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1951566d-bdba-49c0-a803-2421d35706f5 None admin] SCRIPT_NAME: `/identity` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:20:01.692241 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1951566d-bdba-49c0-a803-2421d35706f5 None admin] PATH_INFO: `/v3/domains` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:20:01.692965 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1951566d-bdba-49c0-a803-2421d35706f5 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80245) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 08:20:01.693270 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1951566d-bdba-49c0-a803-2421d35706f5 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:20:01.694635 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1951566d-bdba-49c0-a803-2421d35706f5 None admin] RBAC: Authorization granted {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:20:01.710898 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: [pid: 80245|app: 0|req: 604/1211] 10.222.0.47 () {56 vars in 1109 bytes} [Thu Jul 21 08:20:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 08:20:01.717307 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1331bffd-553f-4282-acc6-7b9a59266e6c None admin] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:20:01.851155 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1331bffd-553f-4282-acc6-7b9a59266e6c None admin] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:20:01.853061 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG dbcounter [-] [80244] Writing DB stats keystone:SELECT=100 {{(pid=80244) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:20:01.891898 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f86c6b9-1ad6-4ca3-bb65-4dffe6a24b94 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:20:01.892810 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f86c6b9-1ad6-4ca3-bb65-4dffe6a24b94 None admin] REQUEST_METHOD: `GET` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:20:01.893132 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f86c6b9-1ad6-4ca3-bb65-4dffe6a24b94 None admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:20:01.893472 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f86c6b9-1ad6-4ca3-bb65-4dffe6a24b94 None admin] PATH_INFO: `/v3/users/placement` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:20:01.902654 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f86c6b9-1ad6-4ca3-bb65-4dffe6a24b94 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:20:01.904177 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f86c6b9-1ad6-4ca3-bb65-4dffe6a24b94 None admin] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:20:01.922584 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application [None req-5f86c6b9-1ad6-4ca3-bb65-4dffe6a24b94 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Jul 21 08:20:01.922584 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 08:20:01.922584 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 08:20:01.922584 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 08:20:01.922584 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 08:20:01.922584 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 08:20:01.922584 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 08:20:01.922584 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 08:20:01.922584 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 08:20:01.922584 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 08:20:01.922584 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 08:20:01.922584 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 08:20:01.922584 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 21 08:20:01.922584 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 21 08:20:01.922584 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 21 08:20:01.922584 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 21 08:20:01.923651 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 08:20:01.923651 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 08:20:01.923651 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 21 08:20:01.923651 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 21 08:20:01.923651 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 21 08:20:01.923651 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 21 08:20:01.923651 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 08:20:01.923651 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 08:20:01.923651 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 08:20:01.923651 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 08:20:01.923651 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 08:20:01.923651 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application with Lock( Jul 21 08:20:01.923651 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 08:20:01.923651 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self._enter() Jul 21 08:20:01.923651 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 08:20:01.923651 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 08:20:01.923651 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 08:20:01.923651 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self.creator() Jul 21 08:20:01.924684 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 08:20:01.924684 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application created_value = creator( Jul 21 08:20:01.924684 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 21 08:20:01.924684 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 21 08:20:01.924684 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 21 08:20:01.924684 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 21 08:20:01.924684 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 21 08:20:01.924684 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 21 08:20:01.924684 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Jul 21 08:20:01.924684 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application Jul 21 08:20:01.928956 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 608/1212] 10.222.0.47 () {56 vars in 1108 bytes} [Thu Jul 21 08:20:01 2022] GET /identity/v3/users/placement => generated 87 bytes in 212 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 08:20:01.930098 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1951566d-bdba-49c0-a803-2421d35706f5 None admin] Authenticating user token {{(pid=80245) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:20:02.051812 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1951566d-bdba-49c0-a803-2421d35706f5 None admin] Validating token access rules against request {{(pid=80245) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:20:02.096642 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dd87096-1d65-4959-afb8-b3bff41afd85 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80245) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:20:02.097284 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dd87096-1d65-4959-afb8-b3bff41afd85 None admin] REQUEST_METHOD: `GET` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:20:02.097572 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dd87096-1d65-4959-afb8-b3bff41afd85 None admin] SCRIPT_NAME: `/identity` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:20:02.097849 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dd87096-1d65-4959-afb8-b3bff41afd85 None admin] PATH_INFO: `/v3/users` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:20:02.098740 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dd87096-1d65-4959-afb8-b3bff41afd85 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80245) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 08:20:02.099657 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dd87096-1d65-4959-afb8-b3bff41afd85 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:20:02.101222 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dd87096-1d65-4959-afb8-b3bff41afd85 None admin] RBAC: Authorization granted {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:20:02.127675 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: [pid: 80245|app: 0|req: 605/1213] 10.222.0.47 () {56 vars in 1143 bytes} [Thu Jul 21 08:20:01 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 08:20:02.128874 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f86c6b9-1ad6-4ca3-bb65-4dffe6a24b94 None admin] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:20:02.247451 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f86c6b9-1ad6-4ca3-bb65-4dffe6a24b94 None admin] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:20:02.287393 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22045d20-e9b4-4cc0-bfa0-929c3d0dfa63 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:20:02.288060 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22045d20-e9b4-4cc0-bfa0-929c3d0dfa63 None admin] REQUEST_METHOD: `GET` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:20:02.288388 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22045d20-e9b4-4cc0-bfa0-929c3d0dfa63 None admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:20:02.288672 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22045d20-e9b4-4cc0-bfa0-929c3d0dfa63 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:20:02.321897 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG dbcounter [-] [80244] Writing DB stats keystone:SELECT=100 {{(pid=80244) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:20:02.374822 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22045d20-e9b4-4cc0-bfa0-929c3d0dfa63 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:20:02.376658 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22045d20-e9b4-4cc0-bfa0-929c3d0dfa63 None admin] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:20:02.531592 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 609/1214] 10.222.0.47 () {58 vars in 1282 bytes} [Thu Jul 21 08:20:02 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 403 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Jul 21 08:20:02.538179 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dd87096-1d65-4959-afb8-b3bff41afd85 None admin] Authenticating user token {{(pid=80245) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:20:02.562366 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG dbcounter [-] [80245] Writing DB stats keystone:SELECT=100 {{(pid=80245) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:20:02.667522 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dd87096-1d65-4959-afb8-b3bff41afd85 None admin] Validating token access rules against request {{(pid=80245) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:20:02.715898 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30a498d6-c6b5-4a10-bd1e-37e6cd2998f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80245) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:20:02.716756 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30a498d6-c6b5-4a10-bd1e-37e6cd2998f4 None admin] REQUEST_METHOD: `GET` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:20:02.717041 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30a498d6-c6b5-4a10-bd1e-37e6cd2998f4 None admin] SCRIPT_NAME: `/identity` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:20:02.717437 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30a498d6-c6b5-4a10-bd1e-37e6cd2998f4 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:20:02.729902 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30a498d6-c6b5-4a10-bd1e-37e6cd2998f4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:20:02.731373 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30a498d6-c6b5-4a10-bd1e-37e6cd2998f4 None admin] RBAC: Authorization granted {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:20:02.752098 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application [None req-30a498d6-c6b5-4a10-bd1e-37e6cd2998f4 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 08:20:02.752098 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 08:20:02.752098 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 08:20:02.752098 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 08:20:02.752098 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 08:20:02.752098 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 08:20:02.752098 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 08:20:02.752098 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 08:20:02.752098 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 08:20:02.752098 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 08:20:02.752098 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 08:20:02.752098 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 08:20:02.752098 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 21 08:20:02.752098 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 21 08:20:02.752098 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 21 08:20:02.759056 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 21 08:20:02.759056 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 08:20:02.759056 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 08:20:02.759056 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 08:20:02.759056 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 08:20:02.759056 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 08:20:02.759056 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 08:20:02.759056 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 08:20:02.759056 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application with Lock( Jul 21 08:20:02.759056 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 08:20:02.759056 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application return self._enter() Jul 21 08:20:02.759056 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 08:20:02.759056 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 08:20:02.759056 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 08:20:02.759056 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application return self.creator() Jul 21 08:20:02.759056 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 08:20:02.759056 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application created_value = creator( Jul 21 08:20:02.759986 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 21 08:20:02.759986 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 21 08:20:02.759986 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 08:20:02.759986 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application Jul 21 08:20:02.759986 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: [pid: 80245|app: 0|req: 606/1215] 10.222.0.47 () {56 vars in 1108 bytes} [Thu Jul 21 08:20:02 2022] GET /identity/v3/domains/Default => generated 87 bytes in 216 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 08:20:02.760288 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22045d20-e9b4-4cc0-bfa0-929c3d0dfa63 None admin] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:20:02.836375 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG dbcounter [-] [80244] Writing DB stats keystone:SELECT=100 {{(pid=80244) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:20:02.892477 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22045d20-e9b4-4cc0-bfa0-929c3d0dfa63 None admin] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:20:02.943608 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f00e5b0-4ec4-41fe-9451-2080d2a2f952 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:20:02.944307 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f00e5b0-4ec4-41fe-9451-2080d2a2f952 None admin] REQUEST_METHOD: `GET` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:20:02.944593 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f00e5b0-4ec4-41fe-9451-2080d2a2f952 None admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:20:02.944870 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f00e5b0-4ec4-41fe-9451-2080d2a2f952 None admin] PATH_INFO: `/v3/domains` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:20:02.946326 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f00e5b0-4ec4-41fe-9451-2080d2a2f952 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80244) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 08:20:02.946696 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f00e5b0-4ec4-41fe-9451-2080d2a2f952 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:20:02.948233 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f00e5b0-4ec4-41fe-9451-2080d2a2f952 None admin] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:20:02.965593 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 610/1216] 10.222.0.47 () {56 vars in 1109 bytes} [Thu Jul 21 08:20:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 08:20:02.972236 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30a498d6-c6b5-4a10-bd1e-37e6cd2998f4 None admin] Authenticating user token {{(pid=80245) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:20:03.063051 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG dbcounter [-] [80245] Writing DB stats keystone:SELECT=100 {{(pid=80245) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:20:03.101933 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30a498d6-c6b5-4a10-bd1e-37e6cd2998f4 None admin] Validating token access rules against request {{(pid=80245) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:20:03.144968 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b8f08d2-3ab1-4ea0-bb48-6750b3bc8968 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80245) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:20:03.146335 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b8f08d2-3ab1-4ea0-bb48-6750b3bc8968 None admin] REQUEST_METHOD: `GET` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:20:03.146335 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b8f08d2-3ab1-4ea0-bb48-6750b3bc8968 None admin] SCRIPT_NAME: `/identity` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:20:03.146517 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b8f08d2-3ab1-4ea0-bb48-6750b3bc8968 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:20:03.155513 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b8f08d2-3ab1-4ea0-bb48-6750b3bc8968 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:20:03.158111 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b8f08d2-3ab1-4ea0-bb48-6750b3bc8968 None admin] RBAC: Authorization granted {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:20:03.182522 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application [None req-3b8f08d2-3ab1-4ea0-bb48-6750b3bc8968 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 21 08:20:03.182522 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 08:20:03.182522 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 08:20:03.182522 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 08:20:03.182522 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 08:20:03.182522 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 08:20:03.182522 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 08:20:03.182522 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 08:20:03.182522 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 08:20:03.182522 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 08:20:03.182522 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 08:20:03.182522 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 08:20:03.182522 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 21 08:20:03.182522 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 21 08:20:03.182522 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 21 08:20:03.183853 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 21 08:20:03.183853 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 08:20:03.183853 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 08:20:03.183853 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 08:20:03.183853 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 08:20:03.183853 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 08:20:03.183853 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 08:20:03.183853 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 08:20:03.183853 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application with Lock( Jul 21 08:20:03.183853 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 08:20:03.183853 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application return self._enter() Jul 21 08:20:03.183853 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 08:20:03.183853 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 08:20:03.183853 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 08:20:03.183853 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application return self.creator() Jul 21 08:20:03.183853 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 08:20:03.183853 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application created_value = creator( Jul 21 08:20:03.184864 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 21 08:20:03.184864 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 21 08:20:03.184864 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 21 08:20:03.184864 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 21 08:20:03.184864 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 21 08:20:03.184864 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 21 08:20:03.184864 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 21 08:20:03.184864 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application Jul 21 08:20:03.184864 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: [pid: 80245|app: 0|req: 607/1217] 10.222.0.47 () {56 vars in 1111 bytes} [Thu Jul 21 08:20:02 2022] GET /identity/v3/projects/service => generated 88 bytes in 213 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 08:20:03.191802 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f00e5b0-4ec4-41fe-9451-2080d2a2f952 None admin] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:20:03.318356 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f00e5b0-4ec4-41fe-9451-2080d2a2f952 None admin] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:20:03.345070 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG dbcounter [-] [80244] Writing DB stats keystone:SELECT=100 {{(pid=80244) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:20:03.359051 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-514ef2ca-6f48-41eb-ab15-41fe8a7e7492 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:20:03.360092 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-514ef2ca-6f48-41eb-ab15-41fe8a7e7492 None admin] REQUEST_METHOD: `GET` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:20:03.360577 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-514ef2ca-6f48-41eb-ab15-41fe8a7e7492 None admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:20:03.361080 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-514ef2ca-6f48-41eb-ab15-41fe8a7e7492 None admin] PATH_INFO: `/v3/projects` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:20:03.362086 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-514ef2ca-6f48-41eb-ab15-41fe8a7e7492 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80244) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 08:20:03.362602 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-514ef2ca-6f48-41eb-ab15-41fe8a7e7492 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:20:03.364349 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-514ef2ca-6f48-41eb-ab15-41fe8a7e7492 None admin] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:20:03.384026 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 611/1218] 10.222.0.47 () {56 vars in 1148 bytes} [Thu Jul 21 08:20:03 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 08:20:03.384977 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b8f08d2-3ab1-4ea0-bb48-6750b3bc8968 None admin] Authenticating user token {{(pid=80245) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:20:03.505680 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b8f08d2-3ab1-4ea0-bb48-6750b3bc8968 None admin] Validating token access rules against request {{(pid=80245) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:20:03.537491 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG dbcounter [-] [80245] Writing DB stats keystone:SELECT=100 {{(pid=80245) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:20:03.544548 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2794b82-f3ab-4167-94ab-99dd3192845b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80245) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:20:03.545364 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2794b82-f3ab-4167-94ab-99dd3192845b None admin] REQUEST_METHOD: `GET` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:20:03.546224 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2794b82-f3ab-4167-94ab-99dd3192845b None admin] SCRIPT_NAME: `/identity` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:20:03.546603 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2794b82-f3ab-4167-94ab-99dd3192845b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:20:03.547618 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2794b82-f3ab-4167-94ab-99dd3192845b None admin] RBAC: Adding query filter params (role.id=50d1ec7f947f42acb65c57e07f2269d6, scope.project.id=4faef13aa50e4e36b6c265fb1398fa14, user.id=3379b32543474d0a8e897484e85c7d04) {{(pid=80245) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 08:20:03.548071 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2794b82-f3ab-4167-94ab-99dd3192845b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:20:03.549707 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2794b82-f3ab-4167-94ab-99dd3192845b None admin] RBAC: Authorization granted {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:20:03.556044 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: [pid: 80245|app: 0|req: 608/1219] 10.222.0.47 () {56 vars in 1374 bytes} [Thu Jul 21 08:20:03 2022] GET /identity/v3/role_assignments?user.id=3379b32543474d0a8e897484e85c7d04&scope.project.id=4faef13aa50e4e36b6c265fb1398fa14&role.id=50d1ec7f947f42acb65c57e07f2269d6 => generated 259 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 21 08:20:05.640974 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-719032fd-df41-4a34-91f4-8d41d2bc9a83 None None] REQUEST_METHOD: `GET` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:20:05.641495 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-719032fd-df41-4a34-91f4-8d41d2bc9a83 None None] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:20:05.641495 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-719032fd-df41-4a34-91f4-8d41d2bc9a83 None None] PATH_INFO: `/` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:20:05.642977 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 612/1220] 10.222.0.47 () {54 vars in 926 bytes} [Thu Jul 21 08:20:05 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 08:20:05.648963 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47397d89-3056-4985-baa0-c024f98bd5fd None None] REQUEST_METHOD: `POST` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:20:05.649475 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47397d89-3056-4985-baa0-c024f98bd5fd None None] SCRIPT_NAME: `/identity` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:20:05.649581 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47397d89-3056-4985-baa0-c024f98bd5fd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:20:05.668492 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.auth.core [None req-47397d89-3056-4985-baa0-c024f98bd5fd None None] MFA Rules not processed for user `2dce40b8d74c41ec94cc36e6ff05d412`. Rule list: `[]` (Enabled: `True`). {{(pid=80245) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 08:20:05.669680 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.fernet_utils [None req-47397d89-3056-4985-baa0-c024f98bd5fd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80245) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 08:20:05.746776 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: [pid: 80245|app: 0|req: 609/1221] 10.222.0.47 () {58 vars in 1023 bytes} [Thu Jul 21 08:20:05 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 21 08:20:05.771581 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-719032fd-df41-4a34-91f4-8d41d2bc9a83 None None] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:20:05.879268 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-719032fd-df41-4a34-91f4-8d41d2bc9a83 None None] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:20:05.923871 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89632932-a847-455d-b144-cba4e1c4027c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:20:05.924819 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89632932-a847-455d-b144-cba4e1c4027c None admin] REQUEST_METHOD: `GET` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:20:05.925107 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89632932-a847-455d-b144-cba4e1c4027c None admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:20:05.925381 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89632932-a847-455d-b144-cba4e1c4027c None admin] PATH_INFO: `/v3/roles/service` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:20:05.937386 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89632932-a847-455d-b144-cba4e1c4027c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 21 08:20:05.946248 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application [None req-89632932-a847-455d-b144-cba4e1c4027c None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 21 08:20:05.946248 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 08:20:05.946248 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 08:20:05.946248 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 08:20:05.946248 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 08:20:05.946248 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 08:20:05.946248 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 08:20:05.946248 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 08:20:05.946248 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 08:20:05.946248 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 08:20:05.946248 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 08:20:05.946248 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 08:20:05.946248 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 21 08:20:05.946248 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 21 08:20:05.946248 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 21 08:20:05.946248 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 21 08:20:05.947516 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 21 08:20:05.947516 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 21 08:20:05.947516 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 21 08:20:05.947516 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 21 08:20:05.947516 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 08:20:05.947516 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 08:20:05.947516 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 08:20:05.947516 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 08:20:05.947516 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 08:20:05.947516 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 08:20:05.947516 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 08:20:05.947516 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application with Lock( Jul 21 08:20:05.947516 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 08:20:05.947516 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self._enter() Jul 21 08:20:05.947516 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 08:20:05.947516 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 08:20:05.948457 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 08:20:05.948457 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self.creator() Jul 21 08:20:05.948457 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 08:20:05.948457 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application created_value = creator( Jul 21 08:20:05.948457 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 21 08:20:05.948457 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 21 08:20:05.948457 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 21 08:20:05.948457 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 21 08:20:05.948457 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 21 08:20:05.948457 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 21 08:20:05.948457 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 21 08:20:05.948457 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application Jul 21 08:20:05.948457 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 613/1222] 10.222.0.47 () {56 vars in 1102 bytes} [Thu Jul 21 08:20:05 2022] GET /identity/v3/roles/service => generated 85 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 08:20:05.954357 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47397d89-3056-4985-baa0-c024f98bd5fd None None] Authenticating user token {{(pid=80245) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:20:06.076468 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47397d89-3056-4985-baa0-c024f98bd5fd None None] Validating token access rules against request {{(pid=80245) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:20:06.104720 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG dbcounter [-] [80245] Writing DB stats keystone:SELECT=100 {{(pid=80245) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:20:06.117801 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb630426-f6a9-4f9e-bb10-a3915adbd7a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80245) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:20:06.118479 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb630426-f6a9-4f9e-bb10-a3915adbd7a4 None admin] REQUEST_METHOD: `GET` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:20:06.118762 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb630426-f6a9-4f9e-bb10-a3915adbd7a4 None admin] SCRIPT_NAME: `/identity` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:20:06.119045 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb630426-f6a9-4f9e-bb10-a3915adbd7a4 None admin] PATH_INFO: `/v3/roles` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:20:06.119824 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb630426-f6a9-4f9e-bb10-a3915adbd7a4 None admin] RBAC: Adding query filter params (name=service) {{(pid=80245) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 08:20:06.120130 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb630426-f6a9-4f9e-bb10-a3915adbd7a4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:20:06.121499 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb630426-f6a9-4f9e-bb10-a3915adbd7a4 None admin] RBAC: Authorization granted {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:20:06.130242 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: [pid: 80245|app: 0|req: 610/1223] 10.222.0.47 () {56 vars in 1103 bytes} [Thu Jul 21 08:20:05 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 08:20:06.136842 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89632932-a847-455d-b144-cba4e1c4027c None admin] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:20:06.195482 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG dbcounter [-] [80244] Writing DB stats keystone:SELECT=100 {{(pid=80244) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:20:06.263725 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89632932-a847-455d-b144-cba4e1c4027c None admin] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:20:06.297782 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23118e26-bbe9-48c9-836c-e460c691fa9d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:20:06.298606 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23118e26-bbe9-48c9-836c-e460c691fa9d None admin] REQUEST_METHOD: `GET` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:20:06.298888 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23118e26-bbe9-48c9-836c-e460c691fa9d None admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:20:06.299165 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23118e26-bbe9-48c9-836c-e460c691fa9d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:20:06.310375 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23118e26-bbe9-48c9-836c-e460c691fa9d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:20:06.312535 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23118e26-bbe9-48c9-836c-e460c691fa9d None admin] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:20:06.331612 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application [None req-23118e26-bbe9-48c9-836c-e460c691fa9d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 08:20:06.331612 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 08:20:06.331612 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 08:20:06.331612 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 08:20:06.331612 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 08:20:06.331612 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 08:20:06.331612 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 08:20:06.331612 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 08:20:06.331612 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 08:20:06.331612 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 08:20:06.331612 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 08:20:06.331612 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 08:20:06.331612 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 21 08:20:06.331612 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 21 08:20:06.331612 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 21 08:20:06.332860 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 21 08:20:06.332860 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 08:20:06.332860 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 08:20:06.332860 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 08:20:06.332860 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 08:20:06.332860 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 08:20:06.332860 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 08:20:06.332860 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 08:20:06.332860 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application with Lock( Jul 21 08:20:06.332860 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 08:20:06.332860 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self._enter() Jul 21 08:20:06.332860 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 08:20:06.332860 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 08:20:06.332860 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 08:20:06.332860 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self.creator() Jul 21 08:20:06.332860 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 08:20:06.332860 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application created_value = creator( Jul 21 08:20:06.334782 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 21 08:20:06.334782 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 21 08:20:06.334782 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 08:20:06.334782 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application Jul 21 08:20:06.334782 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 614/1224] 10.222.0.47 () {56 vars in 1108 bytes} [Thu Jul 21 08:20:06 2022] GET /identity/v3/domains/Default => generated 87 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 08:20:06.340794 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb630426-f6a9-4f9e-bb10-a3915adbd7a4 None admin] Authenticating user token {{(pid=80245) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:20:06.451193 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb630426-f6a9-4f9e-bb10-a3915adbd7a4 None admin] Validating token access rules against request {{(pid=80245) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:20:06.494339 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee2d02f2-7321-4333-ba7b-17a6dc7a8587 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80245) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:20:06.495014 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee2d02f2-7321-4333-ba7b-17a6dc7a8587 None admin] REQUEST_METHOD: `GET` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:20:06.495293 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee2d02f2-7321-4333-ba7b-17a6dc7a8587 None admin] SCRIPT_NAME: `/identity` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:20:06.495595 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee2d02f2-7321-4333-ba7b-17a6dc7a8587 None admin] PATH_INFO: `/v3/domains` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:20:06.496313 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee2d02f2-7321-4333-ba7b-17a6dc7a8587 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80245) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 08:20:06.496622 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee2d02f2-7321-4333-ba7b-17a6dc7a8587 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:20:06.497997 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee2d02f2-7321-4333-ba7b-17a6dc7a8587 None admin] RBAC: Authorization granted {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:20:06.516397 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: [pid: 80245|app: 0|req: 611/1225] 10.222.0.47 () {56 vars in 1109 bytes} [Thu Jul 21 08:20:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 08:20:06.517497 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23118e26-bbe9-48c9-836c-e460c691fa9d None admin] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:20:06.617447 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG dbcounter [-] [80244] Writing DB stats keystone:SELECT=100 {{(pid=80244) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:20:06.625753 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23118e26-bbe9-48c9-836c-e460c691fa9d None admin] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:20:06.661638 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-942ecee9-cbb2-4a05-8146-bae815503f59 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:20:06.662582 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-942ecee9-cbb2-4a05-8146-bae815503f59 None admin] REQUEST_METHOD: `GET` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:20:06.662902 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-942ecee9-cbb2-4a05-8146-bae815503f59 None admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:20:06.663181 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-942ecee9-cbb2-4a05-8146-bae815503f59 None admin] PATH_INFO: `/v3/users/placement` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:20:06.672919 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-942ecee9-cbb2-4a05-8146-bae815503f59 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:20:06.674513 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-942ecee9-cbb2-4a05-8146-bae815503f59 None admin] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:20:06.689336 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application [None req-942ecee9-cbb2-4a05-8146-bae815503f59 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Jul 21 08:20:06.689336 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 08:20:06.689336 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 08:20:06.689336 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 08:20:06.689336 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 08:20:06.689336 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 08:20:06.689336 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 08:20:06.689336 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 08:20:06.689336 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 08:20:06.689336 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 08:20:06.689336 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 08:20:06.689336 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 08:20:06.689336 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 21 08:20:06.689336 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 21 08:20:06.689336 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 21 08:20:06.689336 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 21 08:20:06.690561 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 08:20:06.690561 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 08:20:06.690561 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 21 08:20:06.690561 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 21 08:20:06.690561 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 21 08:20:06.690561 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 21 08:20:06.690561 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 08:20:06.690561 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 08:20:06.690561 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 08:20:06.690561 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 08:20:06.690561 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 08:20:06.690561 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application with Lock( Jul 21 08:20:06.690561 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 08:20:06.690561 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self._enter() Jul 21 08:20:06.690561 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 08:20:06.690561 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 08:20:06.690561 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 08:20:06.690561 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self.creator() Jul 21 08:20:06.691929 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 08:20:06.691929 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application created_value = creator( Jul 21 08:20:06.691929 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 21 08:20:06.691929 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 21 08:20:06.691929 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 21 08:20:06.691929 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 21 08:20:06.691929 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 21 08:20:06.691929 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 21 08:20:06.691929 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Jul 21 08:20:06.691929 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application Jul 21 08:20:06.691929 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 615/1226] 10.222.0.47 () {56 vars in 1108 bytes} [Thu Jul 21 08:20:06 2022] GET /identity/v3/users/placement => generated 87 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 21 08:20:06.696462 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee2d02f2-7321-4333-ba7b-17a6dc7a8587 None admin] Authenticating user token {{(pid=80245) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:20:06.748564 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG dbcounter [-] [80245] Writing DB stats keystone:SELECT=100 {{(pid=80245) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:20:06.807082 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee2d02f2-7321-4333-ba7b-17a6dc7a8587 None admin] Validating token access rules against request {{(pid=80245) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:20:06.842468 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25a5a8f1-0a57-49e9-bf97-7de3c9bd958c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80245) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:20:06.843228 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25a5a8f1-0a57-49e9-bf97-7de3c9bd958c None admin] REQUEST_METHOD: `GET` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:20:06.843570 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25a5a8f1-0a57-49e9-bf97-7de3c9bd958c None admin] SCRIPT_NAME: `/identity` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:20:06.843924 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25a5a8f1-0a57-49e9-bf97-7de3c9bd958c None admin] PATH_INFO: `/v3/users` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:20:06.844789 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25a5a8f1-0a57-49e9-bf97-7de3c9bd958c None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80245) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 08:20:06.845132 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25a5a8f1-0a57-49e9-bf97-7de3c9bd958c None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:20:06.846618 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25a5a8f1-0a57-49e9-bf97-7de3c9bd958c None admin] RBAC: Authorization granted {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:20:06.865088 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: [pid: 80245|app: 0|req: 612/1227] 10.222.0.47 () {56 vars in 1143 bytes} [Thu Jul 21 08:20:06 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 08:20:06.871260 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-942ecee9-cbb2-4a05-8146-bae815503f59 None admin] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:20:06.977538 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-942ecee9-cbb2-4a05-8146-bae815503f59 None admin] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:20:07.018663 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ed095bc-c7fe-4c34-8d64-7559648eca12 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:20:07.019484 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ed095bc-c7fe-4c34-8d64-7559648eca12 None admin] REQUEST_METHOD: `GET` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:20:07.019763 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ed095bc-c7fe-4c34-8d64-7559648eca12 None admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:20:07.020033 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ed095bc-c7fe-4c34-8d64-7559648eca12 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:20:07.031745 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ed095bc-c7fe-4c34-8d64-7559648eca12 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:20:07.033195 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ed095bc-c7fe-4c34-8d64-7559648eca12 None admin] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:20:07.053939 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application [None req-1ed095bc-c7fe-4c34-8d64-7559648eca12 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 08:20:07.053939 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 08:20:07.053939 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 08:20:07.053939 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 08:20:07.053939 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 08:20:07.053939 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 08:20:07.053939 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 08:20:07.053939 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 08:20:07.053939 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 08:20:07.053939 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 08:20:07.053939 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 08:20:07.053939 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 08:20:07.053939 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 21 08:20:07.053939 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 21 08:20:07.053939 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 21 08:20:07.054959 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 21 08:20:07.054959 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 08:20:07.054959 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 08:20:07.054959 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 08:20:07.054959 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 08:20:07.054959 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 08:20:07.054959 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 08:20:07.054959 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 08:20:07.054959 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application with Lock( Jul 21 08:20:07.054959 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 08:20:07.054959 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self._enter() Jul 21 08:20:07.054959 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 08:20:07.054959 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 08:20:07.054959 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 08:20:07.054959 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self.creator() Jul 21 08:20:07.054959 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 08:20:07.054959 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application created_value = creator( Jul 21 08:20:07.055951 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 21 08:20:07.055951 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 21 08:20:07.055951 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 08:20:07.055951 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application Jul 21 08:20:07.055951 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 616/1228] 10.222.0.47 () {56 vars in 1108 bytes} [Thu Jul 21 08:20:06 2022] GET /identity/v3/domains/Default => generated 87 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 08:20:07.063062 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25a5a8f1-0a57-49e9-bf97-7de3c9bd958c None admin] Authenticating user token {{(pid=80245) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:20:07.181421 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25a5a8f1-0a57-49e9-bf97-7de3c9bd958c None admin] Validating token access rules against request {{(pid=80245) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:20:07.184068 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG dbcounter [-] [80245] Writing DB stats keystone:SELECT=100 {{(pid=80245) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:20:07.219182 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90de263b-1a67-441a-b055-f77bd14d846e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80245) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:20:07.219883 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90de263b-1a67-441a-b055-f77bd14d846e None admin] REQUEST_METHOD: `GET` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:20:07.220168 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90de263b-1a67-441a-b055-f77bd14d846e None admin] SCRIPT_NAME: `/identity` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:20:07.220445 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90de263b-1a67-441a-b055-f77bd14d846e None admin] PATH_INFO: `/v3/domains` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:20:07.221157 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90de263b-1a67-441a-b055-f77bd14d846e None admin] RBAC: Adding query filter params (name=Default) {{(pid=80245) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 08:20:07.221496 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90de263b-1a67-441a-b055-f77bd14d846e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:20:07.223203 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90de263b-1a67-441a-b055-f77bd14d846e None admin] RBAC: Authorization granted {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:20:07.237250 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: [pid: 80245|app: 0|req: 613/1229] 10.222.0.47 () {56 vars in 1109 bytes} [Thu Jul 21 08:20:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 08:20:07.243259 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ed095bc-c7fe-4c34-8d64-7559648eca12 None admin] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:20:07.254953 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG dbcounter [-] [80244] Writing DB stats keystone:SELECT=100 {{(pid=80244) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:20:07.367211 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ed095bc-c7fe-4c34-8d64-7559648eca12 None admin] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:20:07.404211 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a75a7a1-38d0-4e51-b9fa-1ecfa996fb77 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:20:07.405029 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a75a7a1-38d0-4e51-b9fa-1ecfa996fb77 None admin] REQUEST_METHOD: `GET` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:20:07.405310 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a75a7a1-38d0-4e51-b9fa-1ecfa996fb77 None admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:20:07.405604 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a75a7a1-38d0-4e51-b9fa-1ecfa996fb77 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:20:07.413516 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a75a7a1-38d0-4e51-b9fa-1ecfa996fb77 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:20:07.415037 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a75a7a1-38d0-4e51-b9fa-1ecfa996fb77 None admin] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:20:07.430615 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application [None req-9a75a7a1-38d0-4e51-b9fa-1ecfa996fb77 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 21 08:20:07.430615 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 08:20:07.430615 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 08:20:07.430615 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 08:20:07.430615 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 08:20:07.430615 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 08:20:07.430615 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 08:20:07.430615 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 08:20:07.430615 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 08:20:07.430615 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 08:20:07.430615 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 08:20:07.430615 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 08:20:07.430615 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 21 08:20:07.430615 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 21 08:20:07.430615 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 21 08:20:07.431929 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 21 08:20:07.431929 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 08:20:07.431929 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 08:20:07.431929 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 08:20:07.431929 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 08:20:07.431929 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 08:20:07.431929 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 08:20:07.431929 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 08:20:07.431929 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application with Lock( Jul 21 08:20:07.431929 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 08:20:07.431929 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self._enter() Jul 21 08:20:07.431929 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 08:20:07.431929 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 08:20:07.431929 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 08:20:07.431929 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self.creator() Jul 21 08:20:07.431929 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 08:20:07.431929 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application created_value = creator( Jul 21 08:20:07.432854 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 21 08:20:07.432854 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 21 08:20:07.432854 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 21 08:20:07.432854 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 21 08:20:07.432854 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 21 08:20:07.432854 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 21 08:20:07.432854 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 21 08:20:07.432854 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application Jul 21 08:20:07.432854 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 617/1230] 10.222.0.47 () {56 vars in 1111 bytes} [Thu Jul 21 08:20:07 2022] GET /identity/v3/projects/service => generated 88 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 08:20:07.438421 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90de263b-1a67-441a-b055-f77bd14d846e None admin] Authenticating user token {{(pid=80245) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:20:07.550829 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90de263b-1a67-441a-b055-f77bd14d846e None admin] Validating token access rules against request {{(pid=80245) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:20:07.587283 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93960d01-abff-4887-8e8c-4302f3c412ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80245) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:20:07.587996 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93960d01-abff-4887-8e8c-4302f3c412ed None admin] REQUEST_METHOD: `GET` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:20:07.588284 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93960d01-abff-4887-8e8c-4302f3c412ed None admin] SCRIPT_NAME: `/identity` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:20:07.588579 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93960d01-abff-4887-8e8c-4302f3c412ed None admin] PATH_INFO: `/v3/projects` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:20:07.589450 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93960d01-abff-4887-8e8c-4302f3c412ed None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80245) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 08:20:07.589762 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93960d01-abff-4887-8e8c-4302f3c412ed None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:20:07.591454 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93960d01-abff-4887-8e8c-4302f3c412ed None admin] RBAC: Authorization granted {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:20:07.604347 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: [pid: 80245|app: 0|req: 614/1231] 10.222.0.47 () {56 vars in 1148 bytes} [Thu Jul 21 08:20:07 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 21 08:20:07.611470 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a75a7a1-38d0-4e51-b9fa-1ecfa996fb77 None admin] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:20:07.688114 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG dbcounter [-] [80244] Writing DB stats keystone:SELECT=100 {{(pid=80244) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:20:07.723920 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a75a7a1-38d0-4e51-b9fa-1ecfa996fb77 None admin] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:20:07.760809 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4250daee-2694-4884-a846-10cd74c14553 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:20:07.761715 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4250daee-2694-4884-a846-10cd74c14553 None admin] REQUEST_METHOD: `PUT` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:20:07.762002 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4250daee-2694-4884-a846-10cd74c14553 None admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:20:07.762280 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4250daee-2694-4884-a846-10cd74c14553 None admin] PATH_INFO: `/v3/projects/4faef13aa50e4e36b6c265fb1398fa14/users/3379b32543474d0a8e897484e85c7d04/roles/50d1ec7f947f42acb65c57e07f2269d6` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:20:07.777796 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4250daee-2694-4884-a846-10cd74c14553 None admin] RBAC: Authorizing `identity:create_grant(project_id=4faef13aa50e4e36b6c265fb1398fa14, user_id=3379b32543474d0a8e897484e85c7d04, role_id=50d1ec7f947f42acb65c57e07f2269d6)` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:20:07.779264 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4250daee-2694-4884-a846-10cd74c14553 None admin] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:20:07.792510 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 618/1232] 10.222.0.47 () {58 vars in 1439 bytes} [Thu Jul 21 08:20:07 2022] PUT /identity/v3/projects/4faef13aa50e4e36b6c265fb1398fa14/users/3379b32543474d0a8e897484e85c7d04/roles/50d1ec7f947f42acb65c57e07f2269d6 => generated 0 bytes in 182 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 08:20:10.050040 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3061af3f-637f-4d9d-a2d8-d56a0515ffc5 None None] REQUEST_METHOD: `GET` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:20:10.050583 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3061af3f-637f-4d9d-a2d8-d56a0515ffc5 None None] SCRIPT_NAME: `/identity` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:20:10.050583 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3061af3f-637f-4d9d-a2d8-d56a0515ffc5 None None] PATH_INFO: `/` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:20:10.052059 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: [pid: 80245|app: 0|req: 615/1233] 10.222.0.47 () {54 vars in 926 bytes} [Thu Jul 21 08:20:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 08:20:10.057801 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11387e4b-c78d-4fa8-894b-198e6a1ddb81 None None] REQUEST_METHOD: `POST` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:20:10.058288 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11387e4b-c78d-4fa8-894b-198e6a1ddb81 None None] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:20:10.058426 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11387e4b-c78d-4fa8-894b-198e6a1ddb81 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:20:10.078236 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.auth.core [None req-11387e4b-c78d-4fa8-894b-198e6a1ddb81 None None] MFA Rules not processed for user `2dce40b8d74c41ec94cc36e6ff05d412`. Rule list: `[]` (Enabled: `True`). {{(pid=80244) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 08:20:10.079458 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.fernet_utils [None req-11387e4b-c78d-4fa8-894b-198e6a1ddb81 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80244) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 08:20:10.164235 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 619/1234] 10.222.0.47 () {58 vars in 1023 bytes} [Thu Jul 21 08:20:10 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 21 08:20:10.190143 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3061af3f-637f-4d9d-a2d8-d56a0515ffc5 None None] Authenticating user token {{(pid=80245) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:20:10.219197 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG dbcounter [-] [80245] Writing DB stats keystone:SELECT=100 {{(pid=80245) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:20:10.307434 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3061af3f-637f-4d9d-a2d8-d56a0515ffc5 None None] Validating token access rules against request {{(pid=80245) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:20:10.347987 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f7813b9-e61f-45a0-aebd-36e2ac8c0a61 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80245) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:20:10.348786 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f7813b9-e61f-45a0-aebd-36e2ac8c0a61 None admin] REQUEST_METHOD: `GET` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:20:10.349069 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f7813b9-e61f-45a0-aebd-36e2ac8c0a61 None admin] SCRIPT_NAME: `/identity` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:20:10.349343 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f7813b9-e61f-45a0-aebd-36e2ac8c0a61 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:20:10.360299 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f7813b9-e61f-45a0-aebd-36e2ac8c0a61 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 21 08:20:10.368072 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application [None req-5f7813b9-e61f-45a0-aebd-36e2ac8c0a61 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 21 08:20:10.368072 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 08:20:10.368072 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 08:20:10.368072 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 08:20:10.368072 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 08:20:10.368072 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 08:20:10.368072 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 08:20:10.368072 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 08:20:10.368072 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 08:20:10.368072 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 08:20:10.368072 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 08:20:10.368072 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 08:20:10.368072 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 21 08:20:10.368072 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 21 08:20:10.368072 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 21 08:20:10.368072 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 21 08:20:10.375714 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 21 08:20:10.375714 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 21 08:20:10.375714 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 21 08:20:10.375714 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 21 08:20:10.375714 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 08:20:10.375714 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 08:20:10.375714 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 08:20:10.375714 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 08:20:10.375714 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 08:20:10.375714 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 08:20:10.375714 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 08:20:10.375714 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application with Lock( Jul 21 08:20:10.375714 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 08:20:10.375714 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application return self._enter() Jul 21 08:20:10.375714 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 08:20:10.375714 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 08:20:10.376853 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 08:20:10.376853 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application return self.creator() Jul 21 08:20:10.376853 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 08:20:10.376853 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application created_value = creator( Jul 21 08:20:10.376853 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 21 08:20:10.376853 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 21 08:20:10.376853 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 21 08:20:10.376853 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 21 08:20:10.376853 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 21 08:20:10.376853 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 21 08:20:10.376853 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 21 08:20:10.376853 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application Jul 21 08:20:10.376853 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: [pid: 80245|app: 0|req: 616/1235] 10.222.0.47 () {56 vars in 1102 bytes} [Thu Jul 21 08:20:10 2022] GET /identity/v3/roles/service => generated 85 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 08:20:10.377715 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11387e4b-c78d-4fa8-894b-198e6a1ddb81 None None] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:20:10.427116 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG dbcounter [-] [80244] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80244) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:20:10.494913 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11387e4b-c78d-4fa8-894b-198e6a1ddb81 None None] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:20:10.530802 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e73c5975-a096-42ca-9628-d29ad8b5865a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:20:10.531505 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e73c5975-a096-42ca-9628-d29ad8b5865a None admin] REQUEST_METHOD: `GET` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:20:10.531788 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e73c5975-a096-42ca-9628-d29ad8b5865a None admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:20:10.532061 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e73c5975-a096-42ca-9628-d29ad8b5865a None admin] PATH_INFO: `/v3/roles` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:20:10.532805 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e73c5975-a096-42ca-9628-d29ad8b5865a None admin] RBAC: Adding query filter params (name=service) {{(pid=80244) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 08:20:10.533110 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e73c5975-a096-42ca-9628-d29ad8b5865a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:20:10.534480 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e73c5975-a096-42ca-9628-d29ad8b5865a None admin] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:20:10.542730 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 620/1236] 10.222.0.47 () {56 vars in 1103 bytes} [Thu Jul 21 08:20:10 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 21 08:20:10.550220 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f7813b9-e61f-45a0-aebd-36e2ac8c0a61 None admin] Authenticating user token {{(pid=80245) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:20:10.643554 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG dbcounter [-] [80245] Writing DB stats keystone:SELECT=100 {{(pid=80245) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:20:10.661582 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f7813b9-e61f-45a0-aebd-36e2ac8c0a61 None admin] Validating token access rules against request {{(pid=80245) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:20:10.700365 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74761626-d473-48b7-8e1b-fb893d741f79 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80245) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:20:10.701165 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74761626-d473-48b7-8e1b-fb893d741f79 None admin] REQUEST_METHOD: `GET` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:20:10.701450 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74761626-d473-48b7-8e1b-fb893d741f79 None admin] SCRIPT_NAME: `/identity` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:20:10.701753 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74761626-d473-48b7-8e1b-fb893d741f79 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:20:10.714729 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74761626-d473-48b7-8e1b-fb893d741f79 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:20:10.716699 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74761626-d473-48b7-8e1b-fb893d741f79 None admin] RBAC: Authorization granted {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:20:10.741854 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application [None req-74761626-d473-48b7-8e1b-fb893d741f79 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 08:20:10.741854 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 08:20:10.741854 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 08:20:10.741854 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 08:20:10.741854 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 08:20:10.741854 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 08:20:10.741854 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 08:20:10.741854 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 08:20:10.741854 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 08:20:10.741854 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 08:20:10.741854 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 08:20:10.741854 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 08:20:10.741854 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 21 08:20:10.741854 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 21 08:20:10.741854 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 21 08:20:10.743153 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 21 08:20:10.743153 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 08:20:10.743153 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 08:20:10.743153 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 08:20:10.743153 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 08:20:10.743153 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 08:20:10.743153 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 08:20:10.743153 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 08:20:10.743153 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application with Lock( Jul 21 08:20:10.743153 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 08:20:10.743153 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application return self._enter() Jul 21 08:20:10.743153 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 08:20:10.743153 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 08:20:10.743153 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 08:20:10.743153 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application return self.creator() Jul 21 08:20:10.743153 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 08:20:10.743153 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application created_value = creator( Jul 21 08:20:10.744502 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 21 08:20:10.744502 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 21 08:20:10.744502 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 08:20:10.744502 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application Jul 21 08:20:10.748793 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: [pid: 80245|app: 0|req: 617/1237] 10.222.0.47 () {56 vars in 1108 bytes} [Thu Jul 21 08:20:10 2022] GET /identity/v3/domains/Default => generated 87 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 08:20:10.749642 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e73c5975-a096-42ca-9628-d29ad8b5865a None admin] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:20:10.868151 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e73c5975-a096-42ca-9628-d29ad8b5865a None admin] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:20:10.877687 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG dbcounter [-] [80244] Writing DB stats keystone:SELECT=100 {{(pid=80244) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:20:10.906610 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fe71913-198f-4fc2-8c9b-e6b1d05c497b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:20:10.907301 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fe71913-198f-4fc2-8c9b-e6b1d05c497b None admin] REQUEST_METHOD: `GET` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:20:10.907618 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fe71913-198f-4fc2-8c9b-e6b1d05c497b None admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:20:10.907911 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fe71913-198f-4fc2-8c9b-e6b1d05c497b None admin] PATH_INFO: `/v3/domains` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:20:10.908767 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fe71913-198f-4fc2-8c9b-e6b1d05c497b None admin] RBAC: Adding query filter params (name=Default) {{(pid=80244) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 08:20:10.909130 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fe71913-198f-4fc2-8c9b-e6b1d05c497b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:20:10.910699 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fe71913-198f-4fc2-8c9b-e6b1d05c497b None admin] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:20:10.923647 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 621/1238] 10.222.0.47 () {56 vars in 1109 bytes} [Thu Jul 21 08:20:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 21 08:20:10.929745 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74761626-d473-48b7-8e1b-fb893d741f79 None admin] Authenticating user token {{(pid=80245) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:20:11.050920 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74761626-d473-48b7-8e1b-fb893d741f79 None admin] Validating token access rules against request {{(pid=80245) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:20:11.088049 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aedd5f92-4f86-427d-881e-9202d5f3850a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80245) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:20:11.088956 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aedd5f92-4f86-427d-881e-9202d5f3850a None admin] REQUEST_METHOD: `GET` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:20:11.089242 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aedd5f92-4f86-427d-881e-9202d5f3850a None admin] SCRIPT_NAME: `/identity` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:20:11.089514 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aedd5f92-4f86-427d-881e-9202d5f3850a None admin] PATH_INFO: `/v3/users/placement` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:20:11.093253 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG dbcounter [-] [80245] Writing DB stats keystone:SELECT=100 {{(pid=80245) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:20:11.099589 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aedd5f92-4f86-427d-881e-9202d5f3850a None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:20:11.101048 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aedd5f92-4f86-427d-881e-9202d5f3850a None admin] RBAC: Authorization granted {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:20:11.116068 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application [None req-aedd5f92-4f86-427d-881e-9202d5f3850a None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Jul 21 08:20:11.116068 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 08:20:11.116068 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 08:20:11.116068 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 08:20:11.116068 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 08:20:11.116068 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 08:20:11.116068 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 08:20:11.116068 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 08:20:11.116068 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 08:20:11.116068 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 08:20:11.116068 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 08:20:11.116068 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 08:20:11.116068 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 21 08:20:11.116068 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 21 08:20:11.116068 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 21 08:20:11.116068 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 21 08:20:11.117161 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 08:20:11.117161 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 08:20:11.117161 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 21 08:20:11.117161 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 21 08:20:11.117161 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 21 08:20:11.117161 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 21 08:20:11.117161 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 08:20:11.117161 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 08:20:11.117161 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 08:20:11.117161 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 08:20:11.117161 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 08:20:11.117161 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application with Lock( Jul 21 08:20:11.117161 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 08:20:11.117161 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application return self._enter() Jul 21 08:20:11.117161 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 08:20:11.117161 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 08:20:11.117161 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 08:20:11.117161 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application return self.creator() Jul 21 08:20:11.118118 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 08:20:11.118118 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application created_value = creator( Jul 21 08:20:11.118118 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 21 08:20:11.118118 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 21 08:20:11.118118 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 21 08:20:11.118118 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 21 08:20:11.118118 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 21 08:20:11.118118 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 21 08:20:11.118118 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Jul 21 08:20:11.118118 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application Jul 21 08:20:11.118118 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: [pid: 80245|app: 0|req: 618/1239] 10.222.0.47 () {56 vars in 1108 bytes} [Thu Jul 21 08:20:10 2022] GET /identity/v3/users/placement => generated 87 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 08:20:11.124505 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fe71913-198f-4fc2-8c9b-e6b1d05c497b None admin] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:20:11.235250 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fe71913-198f-4fc2-8c9b-e6b1d05c497b None admin] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:20:11.271824 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6d83a1b-c62c-44e5-a8d8-fb8770c89b4f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:20:11.272484 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6d83a1b-c62c-44e5-a8d8-fb8770c89b4f None admin] REQUEST_METHOD: `GET` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:20:11.272778 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6d83a1b-c62c-44e5-a8d8-fb8770c89b4f None admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:20:11.273036 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6d83a1b-c62c-44e5-a8d8-fb8770c89b4f None admin] PATH_INFO: `/v3/users` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:20:11.273862 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6d83a1b-c62c-44e5-a8d8-fb8770c89b4f None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80244) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 08:20:11.274174 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6d83a1b-c62c-44e5-a8d8-fb8770c89b4f None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:20:11.275608 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6d83a1b-c62c-44e5-a8d8-fb8770c89b4f None admin] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:20:11.298160 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 622/1240] 10.222.0.47 () {56 vars in 1143 bytes} [Thu Jul 21 08:20:11 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 21 08:20:11.299354 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aedd5f92-4f86-427d-881e-9202d5f3850a None admin] Authenticating user token {{(pid=80245) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:20:11.414516 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aedd5f92-4f86-427d-881e-9202d5f3850a None admin] Validating token access rules against request {{(pid=80245) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:20:11.458309 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07973f2c-9769-489e-b703-fb735b3353b9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80245) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:20:11.459086 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07973f2c-9769-489e-b703-fb735b3353b9 None admin] REQUEST_METHOD: `GET` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:20:11.459400 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07973f2c-9769-489e-b703-fb735b3353b9 None admin] SCRIPT_NAME: `/identity` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:20:11.459679 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07973f2c-9769-489e-b703-fb735b3353b9 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:20:11.538927 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG dbcounter [-] [80245] Writing DB stats keystone:SELECT=100 {{(pid=80245) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:20:11.546641 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07973f2c-9769-489e-b703-fb735b3353b9 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:20:11.548381 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07973f2c-9769-489e-b703-fb735b3353b9 None admin] RBAC: Authorization granted {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:20:11.659435 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: [pid: 80245|app: 0|req: 619/1241] 10.222.0.47 () {58 vars in 1282 bytes} [Thu Jul 21 08:20:11 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 361 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 21 08:20:11.665265 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6d83a1b-c62c-44e5-a8d8-fb8770c89b4f None admin] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:20:11.696223 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG dbcounter [-] [80244] Writing DB stats keystone:SELECT=100 {{(pid=80244) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:20:11.776501 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6d83a1b-c62c-44e5-a8d8-fb8770c89b4f None admin] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:20:11.816411 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afceb392-a3d3-45d2-a359-75cc433babfb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:20:11.817354 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afceb392-a3d3-45d2-a359-75cc433babfb None admin] REQUEST_METHOD: `GET` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:20:11.817643 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afceb392-a3d3-45d2-a359-75cc433babfb None admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:20:11.817976 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afceb392-a3d3-45d2-a359-75cc433babfb None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:20:11.829758 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afceb392-a3d3-45d2-a359-75cc433babfb None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:20:11.831406 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afceb392-a3d3-45d2-a359-75cc433babfb None admin] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:20:11.851143 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application [None req-afceb392-a3d3-45d2-a359-75cc433babfb None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 08:20:11.851143 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 08:20:11.851143 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 08:20:11.851143 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 08:20:11.851143 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 08:20:11.851143 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 08:20:11.851143 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 08:20:11.851143 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 08:20:11.851143 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 08:20:11.851143 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 08:20:11.851143 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 08:20:11.851143 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 08:20:11.851143 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 21 08:20:11.851143 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 21 08:20:11.851143 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 21 08:20:11.852160 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 21 08:20:11.852160 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 08:20:11.852160 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 08:20:11.852160 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 08:20:11.852160 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 08:20:11.852160 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 08:20:11.852160 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 08:20:11.852160 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 08:20:11.852160 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application with Lock( Jul 21 08:20:11.852160 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 08:20:11.852160 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self._enter() Jul 21 08:20:11.852160 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 08:20:11.852160 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 08:20:11.852160 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 08:20:11.852160 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self.creator() Jul 21 08:20:11.852160 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 08:20:11.852160 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application created_value = creator( Jul 21 08:20:11.853272 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 21 08:20:11.853272 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 21 08:20:11.853272 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 08:20:11.853272 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application Jul 21 08:20:11.853272 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 623/1242] 10.222.0.47 () {56 vars in 1108 bytes} [Thu Jul 21 08:20:11 2022] GET /identity/v3/domains/Default => generated 87 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 21 08:20:11.858965 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07973f2c-9769-489e-b703-fb735b3353b9 None admin] Authenticating user token {{(pid=80245) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:20:11.970199 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG dbcounter [-] [80245] Writing DB stats keystone:SELECT=100 {{(pid=80245) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:20:11.976798 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07973f2c-9769-489e-b703-fb735b3353b9 None admin] Validating token access rules against request {{(pid=80245) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:20:12.013702 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2244822a-d3e0-4a0b-82c3-87096d2c274a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80245) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:20:12.014799 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2244822a-d3e0-4a0b-82c3-87096d2c274a None admin] REQUEST_METHOD: `GET` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:20:12.015101 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2244822a-d3e0-4a0b-82c3-87096d2c274a None admin] SCRIPT_NAME: `/identity` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:20:12.015483 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2244822a-d3e0-4a0b-82c3-87096d2c274a None admin] PATH_INFO: `/v3/domains` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:20:12.016296 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2244822a-d3e0-4a0b-82c3-87096d2c274a None admin] RBAC: Adding query filter params (name=Default) {{(pid=80245) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 08:20:12.016675 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2244822a-d3e0-4a0b-82c3-87096d2c274a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:20:12.018198 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2244822a-d3e0-4a0b-82c3-87096d2c274a None admin] RBAC: Authorization granted {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:20:12.031998 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: [pid: 80245|app: 0|req: 620/1243] 10.222.0.47 () {56 vars in 1109 bytes} [Thu Jul 21 08:20:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 08:20:12.039400 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afceb392-a3d3-45d2-a359-75cc433babfb None admin] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:20:12.130223 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG dbcounter [-] [80244] Writing DB stats keystone:SELECT=100 {{(pid=80244) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:20:12.159573 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afceb392-a3d3-45d2-a359-75cc433babfb None admin] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:20:12.208884 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b934f3f8-2663-48ae-a022-20b48ac4aa1a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:20:12.209814 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b934f3f8-2663-48ae-a022-20b48ac4aa1a None admin] REQUEST_METHOD: `GET` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:20:12.210203 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b934f3f8-2663-48ae-a022-20b48ac4aa1a None admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:20:12.210557 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b934f3f8-2663-48ae-a022-20b48ac4aa1a None admin] PATH_INFO: `/v3/projects/service` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:20:12.218951 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b934f3f8-2663-48ae-a022-20b48ac4aa1a None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:20:12.220489 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b934f3f8-2663-48ae-a022-20b48ac4aa1a None admin] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:20:12.236500 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application [None req-b934f3f8-2663-48ae-a022-20b48ac4aa1a None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 21 08:20:12.236500 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 08:20:12.236500 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 08:20:12.236500 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 08:20:12.236500 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 08:20:12.236500 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 08:20:12.236500 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 08:20:12.236500 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 08:20:12.236500 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 08:20:12.236500 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 08:20:12.236500 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 08:20:12.236500 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 08:20:12.236500 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 21 08:20:12.236500 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 21 08:20:12.236500 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 21 08:20:12.237630 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 21 08:20:12.237630 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 08:20:12.237630 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 08:20:12.237630 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 08:20:12.237630 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 08:20:12.237630 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 08:20:12.237630 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 08:20:12.237630 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 08:20:12.237630 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application with Lock( Jul 21 08:20:12.237630 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 08:20:12.237630 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self._enter() Jul 21 08:20:12.237630 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 08:20:12.237630 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 08:20:12.237630 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 08:20:12.237630 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self.creator() Jul 21 08:20:12.237630 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 08:20:12.237630 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application created_value = creator( Jul 21 08:20:12.238763 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 21 08:20:12.238763 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 21 08:20:12.238763 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 21 08:20:12.238763 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 21 08:20:12.238763 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 21 08:20:12.238763 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 21 08:20:12.238763 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 21 08:20:12.238763 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application Jul 21 08:20:12.238763 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 624/1244] 10.222.0.47 () {56 vars in 1111 bytes} [Thu Jul 21 08:20:12 2022] GET /identity/v3/projects/service => generated 88 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 21 08:20:12.244304 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2244822a-d3e0-4a0b-82c3-87096d2c274a None admin] Authenticating user token {{(pid=80245) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:20:12.357512 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2244822a-d3e0-4a0b-82c3-87096d2c274a None admin] Validating token access rules against request {{(pid=80245) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:20:12.395850 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e107121-6b52-4514-8ea6-ec1df0bbd9af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80245) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:20:12.396548 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e107121-6b52-4514-8ea6-ec1df0bbd9af None admin] REQUEST_METHOD: `GET` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:20:12.396819 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e107121-6b52-4514-8ea6-ec1df0bbd9af None admin] SCRIPT_NAME: `/identity` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:20:12.397092 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e107121-6b52-4514-8ea6-ec1df0bbd9af None admin] PATH_INFO: `/v3/projects` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:20:12.397875 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e107121-6b52-4514-8ea6-ec1df0bbd9af None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80245) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 08:20:12.398175 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e107121-6b52-4514-8ea6-ec1df0bbd9af None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:20:12.399629 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e107121-6b52-4514-8ea6-ec1df0bbd9af None admin] RBAC: Authorization granted {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:20:12.413732 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: [pid: 80245|app: 0|req: 621/1245] 10.222.0.47 () {56 vars in 1148 bytes} [Thu Jul 21 08:20:12 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 21 08:20:12.420167 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b934f3f8-2663-48ae-a022-20b48ac4aa1a None admin] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:20:12.537133 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b934f3f8-2663-48ae-a022-20b48ac4aa1a None admin] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:20:12.573065 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-968343d2-ad4c-4c53-84cf-0c6d6c79cd9b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:20:12.573727 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-968343d2-ad4c-4c53-84cf-0c6d6c79cd9b None admin] REQUEST_METHOD: `GET` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:20:12.574004 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-968343d2-ad4c-4c53-84cf-0c6d6c79cd9b None admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:20:12.574296 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-968343d2-ad4c-4c53-84cf-0c6d6c79cd9b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:20:12.575104 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-968343d2-ad4c-4c53-84cf-0c6d6c79cd9b None admin] RBAC: Adding query filter params (role.id=50d1ec7f947f42acb65c57e07f2269d6, scope.project.id=4faef13aa50e4e36b6c265fb1398fa14, user.id=3379b32543474d0a8e897484e85c7d04) {{(pid=80244) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 08:20:12.575448 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-968343d2-ad4c-4c53-84cf-0c6d6c79cd9b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:20:12.577404 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-968343d2-ad4c-4c53-84cf-0c6d6c79cd9b None admin] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:20:12.583507 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 625/1246] 10.222.0.47 () {56 vars in 1374 bytes} [Thu Jul 21 08:20:12 2022] GET /identity/v3/role_assignments?user.id=3379b32543474d0a8e897484e85c7d04&scope.project.id=4faef13aa50e4e36b6c265fb1398fa14&role.id=50d1ec7f947f42acb65c57e07f2269d6 => generated 608 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 08:20:14.885998 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db03bd66-53cc-4113-87cd-45582eceb873 None None] REQUEST_METHOD: `GET` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:20:14.885998 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db03bd66-53cc-4113-87cd-45582eceb873 None None] SCRIPT_NAME: `/identity` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:20:14.886780 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db03bd66-53cc-4113-87cd-45582eceb873 None None] PATH_INFO: `/` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:20:14.887726 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: [pid: 80245|app: 0|req: 622/1247] 10.222.0.47 () {54 vars in 926 bytes} [Thu Jul 21 08:20:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 08:20:14.893710 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1c7c9d1-e040-40ea-b4cd-cce1180dd516 None None] REQUEST_METHOD: `POST` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:20:14.894249 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1c7c9d1-e040-40ea-b4cd-cce1180dd516 None None] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:20:14.894403 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1c7c9d1-e040-40ea-b4cd-cce1180dd516 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:20:14.900468 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG dbcounter [-] [80244] Writing DB stats keystone:SELECT=100 {{(pid=80244) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:20:14.915062 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.auth.core [None req-d1c7c9d1-e040-40ea-b4cd-cce1180dd516 None None] MFA Rules not processed for user `2dce40b8d74c41ec94cc36e6ff05d412`. Rule list: `[]` (Enabled: `True`). {{(pid=80244) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 08:20:14.916415 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.fernet_utils [None req-d1c7c9d1-e040-40ea-b4cd-cce1180dd516 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80244) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 08:20:15.000127 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 626/1248] 10.222.0.47 () {58 vars in 1023 bytes} [Thu Jul 21 08:20:14 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 21 08:20:15.024394 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db03bd66-53cc-4113-87cd-45582eceb873 None None] Authenticating user token {{(pid=80245) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:20:15.043785 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG dbcounter [-] [80245] Writing DB stats keystone:SELECT=100 {{(pid=80245) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:20:15.141946 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db03bd66-53cc-4113-87cd-45582eceb873 None None] Validating token access rules against request {{(pid=80245) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:20:15.184516 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b0bbfcb-b61b-43e3-9c91-39a25052d411 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80245) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:20:15.185322 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b0bbfcb-b61b-43e3-9c91-39a25052d411 None admin] REQUEST_METHOD: `GET` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:20:15.185609 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b0bbfcb-b61b-43e3-9c91-39a25052d411 None admin] SCRIPT_NAME: `/identity` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:20:15.185912 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b0bbfcb-b61b-43e3-9c91-39a25052d411 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:20:15.197194 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b0bbfcb-b61b-43e3-9c91-39a25052d411 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 21 08:20:15.205151 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application [None req-0b0bbfcb-b61b-43e3-9c91-39a25052d411 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 21 08:20:15.205151 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 08:20:15.205151 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 08:20:15.205151 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 08:20:15.205151 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 08:20:15.205151 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 08:20:15.205151 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 08:20:15.205151 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 08:20:15.205151 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 08:20:15.205151 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 08:20:15.205151 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 08:20:15.205151 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 08:20:15.205151 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 21 08:20:15.205151 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 21 08:20:15.205151 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 21 08:20:15.205151 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 21 08:20:15.206151 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 21 08:20:15.206151 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 21 08:20:15.206151 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 21 08:20:15.206151 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 21 08:20:15.206151 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 08:20:15.206151 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 08:20:15.206151 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 08:20:15.206151 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 08:20:15.206151 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 08:20:15.206151 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 08:20:15.206151 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 08:20:15.206151 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application with Lock( Jul 21 08:20:15.206151 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 08:20:15.206151 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application return self._enter() Jul 21 08:20:15.206151 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 08:20:15.206151 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 08:20:15.206975 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 08:20:15.206975 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application return self.creator() Jul 21 08:20:15.206975 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 08:20:15.206975 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application created_value = creator( Jul 21 08:20:15.206975 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 21 08:20:15.206975 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 21 08:20:15.206975 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 21 08:20:15.206975 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 21 08:20:15.206975 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 21 08:20:15.206975 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 21 08:20:15.206975 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 21 08:20:15.206975 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application Jul 21 08:20:15.211448 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: [pid: 80245|app: 0|req: 623/1249] 10.222.0.47 () {56 vars in 1096 bytes} [Thu Jul 21 08:20:15 2022] GET /identity/v3/roles/admin => generated 83 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 08:20:15.213058 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1c7c9d1-e040-40ea-b4cd-cce1180dd516 None None] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:20:15.339238 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1c7c9d1-e040-40ea-b4cd-cce1180dd516 None None] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:20:15.387141 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG dbcounter [-] [80244] Writing DB stats keystone:SELECT=100 {{(pid=80244) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:20:15.397965 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-404b8ffa-b711-4b92-a501-67e8af9a28dc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:20:15.398572 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-404b8ffa-b711-4b92-a501-67e8af9a28dc None admin] REQUEST_METHOD: `GET` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:20:15.398869 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-404b8ffa-b711-4b92-a501-67e8af9a28dc None admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:20:15.399146 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-404b8ffa-b711-4b92-a501-67e8af9a28dc None admin] PATH_INFO: `/v3/roles` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:20:15.400026 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-404b8ffa-b711-4b92-a501-67e8af9a28dc None admin] RBAC: Adding query filter params (name=admin) {{(pid=80244) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 08:20:15.400352 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-404b8ffa-b711-4b92-a501-67e8af9a28dc None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:20:15.401811 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-404b8ffa-b711-4b92-a501-67e8af9a28dc None admin] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:20:15.412400 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 627/1250] 10.222.0.47 () {56 vars in 1099 bytes} [Thu Jul 21 08:20:15 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 21 08:20:15.421433 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b0bbfcb-b61b-43e3-9c91-39a25052d411 None admin] Authenticating user token {{(pid=80245) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:20:15.515135 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG dbcounter [-] [80245] Writing DB stats keystone:SELECT=100 {{(pid=80245) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:20:15.540099 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b0bbfcb-b61b-43e3-9c91-39a25052d411 None admin] Validating token access rules against request {{(pid=80245) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:20:15.575029 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cab0710d-3ac3-4159-a171-59b5a124fdb7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80245) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:20:15.575919 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cab0710d-3ac3-4159-a171-59b5a124fdb7 None admin] REQUEST_METHOD: `GET` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:20:15.576132 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cab0710d-3ac3-4159-a171-59b5a124fdb7 None admin] SCRIPT_NAME: `/identity` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:20:15.576413 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cab0710d-3ac3-4159-a171-59b5a124fdb7 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:20:15.588440 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cab0710d-3ac3-4159-a171-59b5a124fdb7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:20:15.589887 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cab0710d-3ac3-4159-a171-59b5a124fdb7 None admin] RBAC: Authorization granted {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:20:15.608314 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application [None req-cab0710d-3ac3-4159-a171-59b5a124fdb7 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 08:20:15.608314 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 08:20:15.608314 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 08:20:15.608314 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 08:20:15.608314 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 08:20:15.608314 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 08:20:15.608314 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 08:20:15.608314 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 08:20:15.608314 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 08:20:15.608314 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 08:20:15.608314 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 08:20:15.608314 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 08:20:15.608314 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 21 08:20:15.608314 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 21 08:20:15.608314 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 21 08:20:15.609667 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 21 08:20:15.609667 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 08:20:15.609667 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 08:20:15.609667 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 08:20:15.609667 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 08:20:15.609667 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 08:20:15.609667 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 08:20:15.609667 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 08:20:15.609667 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application with Lock( Jul 21 08:20:15.609667 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 08:20:15.609667 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application return self._enter() Jul 21 08:20:15.609667 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 08:20:15.609667 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 08:20:15.609667 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 08:20:15.609667 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application return self.creator() Jul 21 08:20:15.609667 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 08:20:15.609667 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application created_value = creator( Jul 21 08:20:15.610955 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 21 08:20:15.610955 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 21 08:20:15.610955 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 08:20:15.610955 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application Jul 21 08:20:15.610955 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: [pid: 80245|app: 0|req: 624/1251] 10.222.0.47 () {56 vars in 1108 bytes} [Thu Jul 21 08:20:15 2022] GET /identity/v3/domains/Default => generated 87 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 21 08:20:15.617421 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-404b8ffa-b711-4b92-a501-67e8af9a28dc None admin] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:20:15.734025 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-404b8ffa-b711-4b92-a501-67e8af9a28dc None admin] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:20:15.774728 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c44b2432-2790-4597-9aa9-a5a2ef56d53a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:20:15.775471 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c44b2432-2790-4597-9aa9-a5a2ef56d53a None admin] REQUEST_METHOD: `GET` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:20:15.775754 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c44b2432-2790-4597-9aa9-a5a2ef56d53a None admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:20:15.776022 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c44b2432-2790-4597-9aa9-a5a2ef56d53a None admin] PATH_INFO: `/v3/domains` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:20:15.776743 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c44b2432-2790-4597-9aa9-a5a2ef56d53a None admin] RBAC: Adding query filter params (name=Default) {{(pid=80244) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 08:20:15.777032 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c44b2432-2790-4597-9aa9-a5a2ef56d53a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:20:15.779206 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c44b2432-2790-4597-9aa9-a5a2ef56d53a None admin] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:20:15.800627 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 628/1252] 10.222.0.47 () {56 vars in 1109 bytes} [Thu Jul 21 08:20:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 08:20:15.806441 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cab0710d-3ac3-4159-a171-59b5a124fdb7 None admin] Authenticating user token {{(pid=80245) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:20:15.923450 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cab0710d-3ac3-4159-a171-59b5a124fdb7 None admin] Validating token access rules against request {{(pid=80245) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:20:15.959256 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG dbcounter [-] [80245] Writing DB stats keystone:SELECT=100 {{(pid=80245) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:20:15.968016 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d31d95d1-14b1-4a4e-a4f5-31e312d5941c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80245) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:20:15.968826 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d31d95d1-14b1-4a4e-a4f5-31e312d5941c None admin] REQUEST_METHOD: `GET` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:20:15.969109 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d31d95d1-14b1-4a4e-a4f5-31e312d5941c None admin] SCRIPT_NAME: `/identity` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:20:15.969383 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d31d95d1-14b1-4a4e-a4f5-31e312d5941c None admin] PATH_INFO: `/v3/users/placement` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:20:15.978788 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d31d95d1-14b1-4a4e-a4f5-31e312d5941c None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:20:15.980279 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d31d95d1-14b1-4a4e-a4f5-31e312d5941c None admin] RBAC: Authorization granted {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:20:15.996774 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application [None req-d31d95d1-14b1-4a4e-a4f5-31e312d5941c None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Jul 21 08:20:15.996774 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 08:20:15.996774 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 08:20:15.996774 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 08:20:15.996774 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 08:20:15.996774 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 08:20:15.996774 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 08:20:15.996774 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 08:20:15.996774 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 08:20:15.996774 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 08:20:15.996774 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 08:20:15.996774 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 08:20:15.996774 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 21 08:20:15.996774 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 21 08:20:15.996774 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 21 08:20:15.996774 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 21 08:20:15.998915 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 08:20:15.998915 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 08:20:15.998915 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 21 08:20:15.998915 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 21 08:20:15.998915 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 21 08:20:15.998915 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 21 08:20:15.998915 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 08:20:15.998915 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 08:20:15.998915 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 08:20:15.998915 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 08:20:15.998915 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 08:20:15.998915 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application with Lock( Jul 21 08:20:15.998915 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 08:20:15.998915 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application return self._enter() Jul 21 08:20:15.998915 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 08:20:15.998915 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 08:20:15.998915 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 08:20:15.998915 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application return self.creator() Jul 21 08:20:15.999965 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 08:20:15.999965 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application created_value = creator( Jul 21 08:20:15.999965 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 21 08:20:15.999965 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 21 08:20:15.999965 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 21 08:20:15.999965 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 21 08:20:15.999965 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 21 08:20:15.999965 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 21 08:20:15.999965 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Jul 21 08:20:15.999965 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application Jul 21 08:20:15.999965 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: [pid: 80245|app: 0|req: 625/1253] 10.222.0.47 () {56 vars in 1108 bytes} [Thu Jul 21 08:20:15 2022] GET /identity/v3/users/placement => generated 87 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 08:20:16.004676 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c44b2432-2790-4597-9aa9-a5a2ef56d53a None admin] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:20:16.069024 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG dbcounter [-] [80244] Writing DB stats keystone:SELECT=100 {{(pid=80244) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:20:16.122568 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c44b2432-2790-4597-9aa9-a5a2ef56d53a None admin] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:20:16.164429 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40e67ee7-9980-46e2-be1d-42ec230a71c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:20:16.165158 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40e67ee7-9980-46e2-be1d-42ec230a71c3 None admin] REQUEST_METHOD: `GET` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:20:16.165482 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40e67ee7-9980-46e2-be1d-42ec230a71c3 None admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:20:16.165717 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40e67ee7-9980-46e2-be1d-42ec230a71c3 None admin] PATH_INFO: `/v3/users` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:20:16.166680 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40e67ee7-9980-46e2-be1d-42ec230a71c3 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80244) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 08:20:16.167192 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40e67ee7-9980-46e2-be1d-42ec230a71c3 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:20:16.168846 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40e67ee7-9980-46e2-be1d-42ec230a71c3 None admin] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:20:16.191688 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 629/1254] 10.222.0.47 () {56 vars in 1143 bytes} [Thu Jul 21 08:20:16 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 08:20:16.197780 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d31d95d1-14b1-4a4e-a4f5-31e312d5941c None admin] Authenticating user token {{(pid=80245) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:20:16.328636 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d31d95d1-14b1-4a4e-a4f5-31e312d5941c None admin] Validating token access rules against request {{(pid=80245) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:20:16.369047 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59b02430-bf63-4574-bdcb-cdbf323ef41d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80245) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:20:16.369704 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59b02430-bf63-4574-bdcb-cdbf323ef41d None admin] REQUEST_METHOD: `GET` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:20:16.369977 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59b02430-bf63-4574-bdcb-cdbf323ef41d None admin] SCRIPT_NAME: `/identity` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:20:16.370293 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59b02430-bf63-4574-bdcb-cdbf323ef41d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:20:16.437251 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG dbcounter [-] [80245] Writing DB stats keystone:SELECT=100 {{(pid=80245) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:20:16.452033 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59b02430-bf63-4574-bdcb-cdbf323ef41d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:20:16.453685 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59b02430-bf63-4574-bdcb-cdbf323ef41d None admin] RBAC: Authorization granted {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:20:16.578785 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: [pid: 80245|app: 0|req: 626/1255] 10.222.0.47 () {58 vars in 1282 bytes} [Thu Jul 21 08:20:16 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 381 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 21 08:20:16.584996 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40e67ee7-9980-46e2-be1d-42ec230a71c3 None admin] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:20:16.703149 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40e67ee7-9980-46e2-be1d-42ec230a71c3 None admin] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:20:16.716161 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG dbcounter [-] [80244] Writing DB stats keystone:SELECT=100 {{(pid=80244) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:20:16.739855 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e9d4475-4d7c-4b83-8e5d-f0cdd4edf7e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:20:16.740661 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e9d4475-4d7c-4b83-8e5d-f0cdd4edf7e3 None admin] REQUEST_METHOD: `GET` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:20:16.740949 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e9d4475-4d7c-4b83-8e5d-f0cdd4edf7e3 None admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:20:16.741229 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e9d4475-4d7c-4b83-8e5d-f0cdd4edf7e3 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:20:16.754186 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e9d4475-4d7c-4b83-8e5d-f0cdd4edf7e3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:20:16.756300 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e9d4475-4d7c-4b83-8e5d-f0cdd4edf7e3 None admin] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:20:16.780322 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application [None req-8e9d4475-4d7c-4b83-8e5d-f0cdd4edf7e3 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 08:20:16.780322 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 08:20:16.780322 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 08:20:16.780322 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 08:20:16.780322 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 08:20:16.780322 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 08:20:16.780322 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 08:20:16.780322 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 08:20:16.780322 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 08:20:16.780322 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 08:20:16.780322 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 08:20:16.780322 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 08:20:16.780322 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 21 08:20:16.780322 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 21 08:20:16.780322 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 21 08:20:16.781406 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 21 08:20:16.781406 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 08:20:16.781406 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 08:20:16.781406 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 08:20:16.781406 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 08:20:16.781406 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 08:20:16.781406 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 08:20:16.781406 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 08:20:16.781406 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application with Lock( Jul 21 08:20:16.781406 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 08:20:16.781406 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self._enter() Jul 21 08:20:16.781406 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 08:20:16.781406 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 08:20:16.781406 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 08:20:16.781406 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self.creator() Jul 21 08:20:16.781406 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 08:20:16.781406 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application created_value = creator( Jul 21 08:20:16.782355 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 21 08:20:16.782355 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 21 08:20:16.782355 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 08:20:16.782355 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application Jul 21 08:20:16.782355 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 630/1256] 10.222.0.47 () {56 vars in 1108 bytes} [Thu Jul 21 08:20:16 2022] GET /identity/v3/domains/Default => generated 87 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 21 08:20:16.788425 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59b02430-bf63-4574-bdcb-cdbf323ef41d None admin] Authenticating user token {{(pid=80245) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:20:16.897027 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG dbcounter [-] [80245] Writing DB stats keystone:SELECT=100 {{(pid=80245) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:20:16.912939 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59b02430-bf63-4574-bdcb-cdbf323ef41d None admin] Validating token access rules against request {{(pid=80245) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:20:16.959170 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8fd11bb-dd04-45b4-a845-0e3f15759495 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80245) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:20:16.959943 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8fd11bb-dd04-45b4-a845-0e3f15759495 None admin] REQUEST_METHOD: `GET` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:20:16.960231 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8fd11bb-dd04-45b4-a845-0e3f15759495 None admin] SCRIPT_NAME: `/identity` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:20:16.960544 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8fd11bb-dd04-45b4-a845-0e3f15759495 None admin] PATH_INFO: `/v3/domains` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:20:16.961306 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8fd11bb-dd04-45b4-a845-0e3f15759495 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80245) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 08:20:16.961557 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8fd11bb-dd04-45b4-a845-0e3f15759495 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:20:16.962922 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8fd11bb-dd04-45b4-a845-0e3f15759495 None admin] RBAC: Authorization granted {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:20:16.975210 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: [pid: 80245|app: 0|req: 627/1257] 10.222.0.47 () {56 vars in 1109 bytes} [Thu Jul 21 08:20:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 08:20:16.981285 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e9d4475-4d7c-4b83-8e5d-f0cdd4edf7e3 None admin] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:20:17.090894 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e9d4475-4d7c-4b83-8e5d-f0cdd4edf7e3 None admin] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:20:17.127111 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afd835ad-a026-49d3-8bd8-df7b14e56829 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:20:17.127944 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afd835ad-a026-49d3-8bd8-df7b14e56829 None admin] REQUEST_METHOD: `GET` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:20:17.128260 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afd835ad-a026-49d3-8bd8-df7b14e56829 None admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:20:17.128557 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afd835ad-a026-49d3-8bd8-df7b14e56829 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:20:17.136165 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afd835ad-a026-49d3-8bd8-df7b14e56829 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:20:17.137620 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afd835ad-a026-49d3-8bd8-df7b14e56829 None admin] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:20:17.152773 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application [None req-afd835ad-a026-49d3-8bd8-df7b14e56829 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 21 08:20:17.152773 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 08:20:17.152773 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 08:20:17.152773 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 08:20:17.152773 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 08:20:17.152773 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 08:20:17.152773 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 08:20:17.152773 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 08:20:17.152773 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 08:20:17.152773 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 08:20:17.152773 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 08:20:17.152773 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 08:20:17.152773 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 21 08:20:17.152773 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 21 08:20:17.152773 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 21 08:20:17.153884 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 21 08:20:17.153884 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 08:20:17.153884 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 08:20:17.153884 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 08:20:17.153884 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 08:20:17.153884 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 08:20:17.153884 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 08:20:17.153884 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 08:20:17.153884 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application with Lock( Jul 21 08:20:17.153884 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 08:20:17.153884 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self._enter() Jul 21 08:20:17.153884 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 08:20:17.153884 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 08:20:17.153884 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 08:20:17.153884 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self.creator() Jul 21 08:20:17.153884 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 08:20:17.153884 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application created_value = creator( Jul 21 08:20:17.160808 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 21 08:20:17.160808 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 21 08:20:17.160808 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 21 08:20:17.160808 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 21 08:20:17.160808 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 21 08:20:17.160808 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 21 08:20:17.160808 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 21 08:20:17.160808 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application Jul 21 08:20:17.160808 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 631/1258] 10.222.0.47 () {56 vars in 1111 bytes} [Thu Jul 21 08:20:16 2022] GET /identity/v3/projects/service => generated 88 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 08:20:17.162562 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8fd11bb-dd04-45b4-a845-0e3f15759495 None admin] Authenticating user token {{(pid=80245) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:20:17.295643 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8fd11bb-dd04-45b4-a845-0e3f15759495 None admin] Validating token access rules against request {{(pid=80245) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:20:17.336144 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d473b45a-5820-4433-bf39-f8c27e4db92f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80245) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:20:17.336970 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d473b45a-5820-4433-bf39-f8c27e4db92f None admin] REQUEST_METHOD: `GET` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:20:17.337289 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d473b45a-5820-4433-bf39-f8c27e4db92f None admin] SCRIPT_NAME: `/identity` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:20:17.337638 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d473b45a-5820-4433-bf39-f8c27e4db92f None admin] PATH_INFO: `/v3/projects` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:20:17.338460 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d473b45a-5820-4433-bf39-f8c27e4db92f None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80245) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 08:20:17.338867 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d473b45a-5820-4433-bf39-f8c27e4db92f None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:20:17.340685 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d473b45a-5820-4433-bf39-f8c27e4db92f None admin] RBAC: Authorization granted {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:20:17.353392 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: [pid: 80245|app: 0|req: 628/1259] 10.222.0.47 () {56 vars in 1148 bytes} [Thu Jul 21 08:20:17 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 08:20:17.359694 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afd835ad-a026-49d3-8bd8-df7b14e56829 None admin] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:20:17.390235 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG dbcounter [-] [80244] Writing DB stats keystone:SELECT=100 {{(pid=80244) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:20:17.470411 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afd835ad-a026-49d3-8bd8-df7b14e56829 None admin] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:20:17.508727 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5202372f-d589-4ef2-8d5e-b1cada3b3555 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:20:17.509390 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5202372f-d589-4ef2-8d5e-b1cada3b3555 None admin] REQUEST_METHOD: `GET` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:20:17.509667 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5202372f-d589-4ef2-8d5e-b1cada3b3555 None admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:20:17.509939 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5202372f-d589-4ef2-8d5e-b1cada3b3555 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:20:17.510769 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5202372f-d589-4ef2-8d5e-b1cada3b3555 None admin] RBAC: Adding query filter params (role.id=e29870f6e9e2411a9db0d5bf45f32ca5, scope.project.id=4faef13aa50e4e36b6c265fb1398fa14, user.id=3379b32543474d0a8e897484e85c7d04) {{(pid=80244) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 08:20:17.511075 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5202372f-d589-4ef2-8d5e-b1cada3b3555 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:20:17.512524 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5202372f-d589-4ef2-8d5e-b1cada3b3555 None admin] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:20:17.518867 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 632/1260] 10.222.0.47 () {56 vars in 1374 bytes} [Thu Jul 21 08:20:17 2022] GET /identity/v3/role_assignments?user.id=3379b32543474d0a8e897484e85c7d04&scope.project.id=4faef13aa50e4e36b6c265fb1398fa14&role.id=e29870f6e9e2411a9db0d5bf45f32ca5 => generated 259 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 08:20:19.578667 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-977f0486-bded-4ea2-a582-982e4188f7ba None None] REQUEST_METHOD: `GET` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:20:19.579179 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-977f0486-bded-4ea2-a582-982e4188f7ba None None] SCRIPT_NAME: `/identity` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:20:19.579282 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-977f0486-bded-4ea2-a582-982e4188f7ba None None] PATH_INFO: `/` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:20:19.580625 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: [pid: 80245|app: 0|req: 629/1261] 10.222.0.47 () {54 vars in 926 bytes} [Thu Jul 21 08:20:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 08:20:19.586040 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e15e8c3-e342-43cd-9778-2e6df31934ab None None] REQUEST_METHOD: `POST` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:20:19.586585 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e15e8c3-e342-43cd-9778-2e6df31934ab None None] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:20:19.587063 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e15e8c3-e342-43cd-9778-2e6df31934ab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:20:19.605135 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.auth.core [None req-9e15e8c3-e342-43cd-9778-2e6df31934ab None None] MFA Rules not processed for user `2dce40b8d74c41ec94cc36e6ff05d412`. Rule list: `[]` (Enabled: `True`). {{(pid=80244) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 08:20:19.606336 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.fernet_utils [None req-9e15e8c3-e342-43cd-9778-2e6df31934ab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80244) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 08:20:19.671418 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 633/1262] 10.222.0.47 () {58 vars in 1023 bytes} [Thu Jul 21 08:20:19 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 21 08:20:19.694482 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-977f0486-bded-4ea2-a582-982e4188f7ba None None] Authenticating user token {{(pid=80245) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:20:19.705405 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG dbcounter [-] [80245] Writing DB stats keystone:SELECT=100 {{(pid=80245) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:20:19.793279 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-977f0486-bded-4ea2-a582-982e4188f7ba None None] Validating token access rules against request {{(pid=80245) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:20:19.826960 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efb6ac46-1fe9-4a48-ab58-89a1352112aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80245) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:20:19.827774 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efb6ac46-1fe9-4a48-ab58-89a1352112aa None admin] REQUEST_METHOD: `GET` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:20:19.828052 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efb6ac46-1fe9-4a48-ab58-89a1352112aa None admin] SCRIPT_NAME: `/identity` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:20:19.828319 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efb6ac46-1fe9-4a48-ab58-89a1352112aa None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:20:19.837978 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efb6ac46-1fe9-4a48-ab58-89a1352112aa None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 21 08:20:19.846075 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application [None req-efb6ac46-1fe9-4a48-ab58-89a1352112aa None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 21 08:20:19.846075 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 08:20:19.846075 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 08:20:19.846075 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 08:20:19.846075 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 08:20:19.846075 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 08:20:19.846075 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 08:20:19.846075 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 08:20:19.846075 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 08:20:19.846075 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 08:20:19.846075 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 08:20:19.846075 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 08:20:19.846075 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 21 08:20:19.846075 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 21 08:20:19.846075 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 21 08:20:19.846075 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 21 08:20:19.847168 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 21 08:20:19.847168 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 21 08:20:19.847168 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 21 08:20:19.847168 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 21 08:20:19.847168 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 08:20:19.847168 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 08:20:19.847168 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 08:20:19.847168 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 08:20:19.847168 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 08:20:19.847168 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 08:20:19.847168 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 08:20:19.847168 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application with Lock( Jul 21 08:20:19.847168 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 08:20:19.847168 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application return self._enter() Jul 21 08:20:19.847168 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 08:20:19.847168 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 08:20:19.848084 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 08:20:19.848084 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application return self.creator() Jul 21 08:20:19.848084 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 08:20:19.848084 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application created_value = creator( Jul 21 08:20:19.848084 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 21 08:20:19.848084 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 21 08:20:19.848084 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 21 08:20:19.848084 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 21 08:20:19.848084 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 21 08:20:19.848084 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 21 08:20:19.848084 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 21 08:20:19.848084 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application Jul 21 08:20:19.848084 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: [pid: 80245|app: 0|req: 630/1263] 10.222.0.47 () {56 vars in 1096 bytes} [Thu Jul 21 08:20:19 2022] GET /identity/v3/roles/admin => generated 83 bytes in 153 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 08:20:19.853203 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e15e8c3-e342-43cd-9778-2e6df31934ab None None] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:20:19.872582 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG dbcounter [-] [80244] Writing DB stats keystone:SELECT=100 {{(pid=80244) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:20:19.953066 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e15e8c3-e342-43cd-9778-2e6df31934ab None None] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:20:19.986068 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e10ed5c9-ca37-490e-9228-c859adf1e842 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:20:19.986736 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e10ed5c9-ca37-490e-9228-c859adf1e842 None admin] REQUEST_METHOD: `GET` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:20:19.987022 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e10ed5c9-ca37-490e-9228-c859adf1e842 None admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:20:19.987299 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e10ed5c9-ca37-490e-9228-c859adf1e842 None admin] PATH_INFO: `/v3/roles` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:20:19.988079 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e10ed5c9-ca37-490e-9228-c859adf1e842 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80244) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 08:20:19.988384 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e10ed5c9-ca37-490e-9228-c859adf1e842 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:20:19.989749 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e10ed5c9-ca37-490e-9228-c859adf1e842 None admin] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:20:19.997578 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 634/1264] 10.222.0.47 () {56 vars in 1099 bytes} [Thu Jul 21 08:20:19 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 08:20:20.003527 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efb6ac46-1fe9-4a48-ab58-89a1352112aa None admin] Authenticating user token {{(pid=80245) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:20:20.068921 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG dbcounter [-] [80245] Writing DB stats keystone:SELECT=100 {{(pid=80245) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:20:20.101895 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efb6ac46-1fe9-4a48-ab58-89a1352112aa None admin] Validating token access rules against request {{(pid=80245) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:20:20.134176 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6aa71a6-9403-4b98-b292-13e3c4803ef1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80245) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:20:20.134969 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6aa71a6-9403-4b98-b292-13e3c4803ef1 None admin] REQUEST_METHOD: `GET` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:20:20.135257 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6aa71a6-9403-4b98-b292-13e3c4803ef1 None admin] SCRIPT_NAME: `/identity` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:20:20.135558 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6aa71a6-9403-4b98-b292-13e3c4803ef1 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:20:20.146392 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6aa71a6-9403-4b98-b292-13e3c4803ef1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:20:20.147938 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6aa71a6-9403-4b98-b292-13e3c4803ef1 None admin] RBAC: Authorization granted {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:20:20.167820 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application [None req-c6aa71a6-9403-4b98-b292-13e3c4803ef1 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 08:20:20.167820 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 08:20:20.167820 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 08:20:20.167820 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 08:20:20.167820 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 08:20:20.167820 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 08:20:20.167820 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 08:20:20.167820 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 08:20:20.167820 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 08:20:20.167820 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 08:20:20.167820 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 08:20:20.167820 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 08:20:20.167820 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 21 08:20:20.167820 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 21 08:20:20.167820 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 21 08:20:20.168899 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 21 08:20:20.168899 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 08:20:20.168899 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 08:20:20.168899 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 08:20:20.168899 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 08:20:20.168899 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 08:20:20.168899 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 08:20:20.168899 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 08:20:20.168899 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application with Lock( Jul 21 08:20:20.168899 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 08:20:20.168899 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application return self._enter() Jul 21 08:20:20.168899 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 08:20:20.168899 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 08:20:20.168899 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 08:20:20.168899 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application return self.creator() Jul 21 08:20:20.168899 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 08:20:20.168899 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application created_value = creator( Jul 21 08:20:20.169858 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 21 08:20:20.169858 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 21 08:20:20.169858 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 08:20:20.169858 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application Jul 21 08:20:20.169858 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: [pid: 80245|app: 0|req: 631/1265] 10.222.0.47 () {56 vars in 1108 bytes} [Thu Jul 21 08:20:20 2022] GET /identity/v3/domains/Default => generated 87 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 08:20:20.175489 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e10ed5c9-ca37-490e-9228-c859adf1e842 None admin] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:20:20.267250 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG dbcounter [-] [80244] Writing DB stats keystone:SELECT=100 {{(pid=80244) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:20:20.286321 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e10ed5c9-ca37-490e-9228-c859adf1e842 None admin] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:20:20.320050 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6a8dcd7-d0f9-46db-9b22-9763a8d3b960 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:20:20.320639 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6a8dcd7-d0f9-46db-9b22-9763a8d3b960 None admin] REQUEST_METHOD: `GET` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:20:20.320927 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6a8dcd7-d0f9-46db-9b22-9763a8d3b960 None admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:20:20.321207 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6a8dcd7-d0f9-46db-9b22-9763a8d3b960 None admin] PATH_INFO: `/v3/domains` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:20:20.321932 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6a8dcd7-d0f9-46db-9b22-9763a8d3b960 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80244) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 08:20:20.322239 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6a8dcd7-d0f9-46db-9b22-9763a8d3b960 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:20:20.323640 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6a8dcd7-d0f9-46db-9b22-9763a8d3b960 None admin] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:20:20.335829 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 635/1266] 10.222.0.47 () {56 vars in 1109 bytes} [Thu Jul 21 08:20:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 08:20:20.341605 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6aa71a6-9403-4b98-b292-13e3c4803ef1 None admin] Authenticating user token {{(pid=80245) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:20:20.439405 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6aa71a6-9403-4b98-b292-13e3c4803ef1 None admin] Validating token access rules against request {{(pid=80245) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:20:20.458629 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG dbcounter [-] [80245] Writing DB stats keystone:SELECT=100 {{(pid=80245) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:20:20.472629 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0f19d72-e999-4ac8-b877-eecf5825b88d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80245) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:20:20.473441 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0f19d72-e999-4ac8-b877-eecf5825b88d None admin] REQUEST_METHOD: `GET` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:20:20.473728 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0f19d72-e999-4ac8-b877-eecf5825b88d None admin] SCRIPT_NAME: `/identity` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:20:20.474006 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0f19d72-e999-4ac8-b877-eecf5825b88d None admin] PATH_INFO: `/v3/users/placement` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:20:20.481825 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0f19d72-e999-4ac8-b877-eecf5825b88d None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:20:20.483716 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0f19d72-e999-4ac8-b877-eecf5825b88d None admin] RBAC: Authorization granted {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:20:20.498728 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application [None req-f0f19d72-e999-4ac8-b877-eecf5825b88d None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Jul 21 08:20:20.498728 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 08:20:20.498728 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 08:20:20.498728 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 08:20:20.498728 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 08:20:20.498728 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 08:20:20.498728 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 08:20:20.498728 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 08:20:20.498728 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 08:20:20.498728 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 08:20:20.498728 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 08:20:20.498728 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 08:20:20.498728 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 21 08:20:20.498728 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 21 08:20:20.498728 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 21 08:20:20.498728 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 21 08:20:20.499821 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 08:20:20.499821 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 08:20:20.499821 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 21 08:20:20.499821 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 21 08:20:20.499821 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 21 08:20:20.499821 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 21 08:20:20.499821 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 08:20:20.499821 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 08:20:20.499821 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 08:20:20.499821 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 08:20:20.499821 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 08:20:20.499821 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application with Lock( Jul 21 08:20:20.499821 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 08:20:20.499821 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application return self._enter() Jul 21 08:20:20.499821 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 08:20:20.499821 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 08:20:20.499821 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 08:20:20.499821 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application return self.creator() Jul 21 08:20:20.500741 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 08:20:20.500741 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application created_value = creator( Jul 21 08:20:20.500741 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 21 08:20:20.500741 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 21 08:20:20.500741 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 21 08:20:20.500741 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 21 08:20:20.500741 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 21 08:20:20.500741 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 21 08:20:20.500741 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Jul 21 08:20:20.500741 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application Jul 21 08:20:20.500741 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: [pid: 80245|app: 0|req: 632/1267] 10.222.0.47 () {56 vars in 1108 bytes} [Thu Jul 21 08:20:20 2022] GET /identity/v3/users/placement => generated 87 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 08:20:20.505711 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6a8dcd7-d0f9-46db-9b22-9763a8d3b960 None admin] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:20:20.602654 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6a8dcd7-d0f9-46db-9b22-9763a8d3b960 None admin] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:20:20.636112 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-635f28f5-5f57-4189-9559-4f6491608f4a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:20:20.636772 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-635f28f5-5f57-4189-9559-4f6491608f4a None admin] REQUEST_METHOD: `GET` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:20:20.637055 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-635f28f5-5f57-4189-9559-4f6491608f4a None admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:20:20.637329 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-635f28f5-5f57-4189-9559-4f6491608f4a None admin] PATH_INFO: `/v3/users` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:20:20.638159 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-635f28f5-5f57-4189-9559-4f6491608f4a None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80244) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 08:20:20.638468 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-635f28f5-5f57-4189-9559-4f6491608f4a None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:20:20.639953 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-635f28f5-5f57-4189-9559-4f6491608f4a None admin] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:20:20.648250 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG dbcounter [-] [80244] Writing DB stats keystone:SELECT=100 {{(pid=80244) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:20:20.657071 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 636/1268] 10.222.0.47 () {56 vars in 1143 bytes} [Thu Jul 21 08:20:20 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 08:20:20.662701 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0f19d72-e999-4ac8-b877-eecf5825b88d None admin] Authenticating user token {{(pid=80245) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:20:20.765424 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0f19d72-e999-4ac8-b877-eecf5825b88d None admin] Validating token access rules against request {{(pid=80245) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:20:20.802016 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e9ac4c4-64e2-4e84-8109-4719e1da0033 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80245) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:20:20.802807 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e9ac4c4-64e2-4e84-8109-4719e1da0033 None admin] REQUEST_METHOD: `GET` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:20:20.803094 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e9ac4c4-64e2-4e84-8109-4719e1da0033 None admin] SCRIPT_NAME: `/identity` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:20:20.803399 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e9ac4c4-64e2-4e84-8109-4719e1da0033 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:20:20.815373 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e9ac4c4-64e2-4e84-8109-4719e1da0033 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:20:20.816825 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e9ac4c4-64e2-4e84-8109-4719e1da0033 None admin] RBAC: Authorization granted {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:20:20.833450 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application [None req-2e9ac4c4-64e2-4e84-8109-4719e1da0033 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 08:20:20.833450 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 08:20:20.833450 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 08:20:20.833450 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 08:20:20.833450 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 08:20:20.833450 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 08:20:20.833450 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 08:20:20.833450 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 08:20:20.833450 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 08:20:20.833450 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 08:20:20.833450 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 08:20:20.833450 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 08:20:20.833450 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 21 08:20:20.833450 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 21 08:20:20.833450 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 21 08:20:20.834479 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 21 08:20:20.834479 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 08:20:20.834479 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 08:20:20.834479 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 08:20:20.834479 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 08:20:20.834479 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 08:20:20.834479 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 08:20:20.834479 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 08:20:20.834479 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application with Lock( Jul 21 08:20:20.834479 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 08:20:20.834479 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application return self._enter() Jul 21 08:20:20.834479 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 08:20:20.834479 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 08:20:20.834479 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 08:20:20.834479 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application return self.creator() Jul 21 08:20:20.834479 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 08:20:20.834479 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application created_value = creator( Jul 21 08:20:20.835464 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 21 08:20:20.835464 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 21 08:20:20.835464 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 08:20:20.835464 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application Jul 21 08:20:20.835464 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: [pid: 80245|app: 0|req: 633/1269] 10.222.0.47 () {56 vars in 1108 bytes} [Thu Jul 21 08:20:20 2022] GET /identity/v3/domains/Default => generated 87 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 08:20:20.840381 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-635f28f5-5f57-4189-9559-4f6491608f4a None admin] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:20:20.939782 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-635f28f5-5f57-4189-9559-4f6491608f4a None admin] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:20:20.972359 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-911dfe35-51a4-4e3f-8ec0-ebd671ba2965 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:20:20.973012 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-911dfe35-51a4-4e3f-8ec0-ebd671ba2965 None admin] REQUEST_METHOD: `GET` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:20:20.973291 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-911dfe35-51a4-4e3f-8ec0-ebd671ba2965 None admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:20:20.973563 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-911dfe35-51a4-4e3f-8ec0-ebd671ba2965 None admin] PATH_INFO: `/v3/domains` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:20:20.974278 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-911dfe35-51a4-4e3f-8ec0-ebd671ba2965 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80244) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 08:20:20.974580 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-911dfe35-51a4-4e3f-8ec0-ebd671ba2965 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:20:20.976041 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-911dfe35-51a4-4e3f-8ec0-ebd671ba2965 None admin] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:20:20.987454 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 637/1270] 10.222.0.47 () {56 vars in 1109 bytes} [Thu Jul 21 08:20:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 21 08:20:20.993212 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e9ac4c4-64e2-4e84-8109-4719e1da0033 None admin] Authenticating user token {{(pid=80245) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:20:21.023836 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG dbcounter [-] [80245] Writing DB stats keystone:SELECT=100 {{(pid=80245) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:20:21.090811 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e9ac4c4-64e2-4e84-8109-4719e1da0033 None admin] Validating token access rules against request {{(pid=80245) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:20:21.124772 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bda5e176-8caa-468d-bb78-01ffa9371d14 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80245) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:20:21.125451 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bda5e176-8caa-468d-bb78-01ffa9371d14 None admin] REQUEST_METHOD: `GET` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:20:21.125732 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bda5e176-8caa-468d-bb78-01ffa9371d14 None admin] SCRIPT_NAME: `/identity` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:20:21.126004 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bda5e176-8caa-468d-bb78-01ffa9371d14 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:20:21.132485 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bda5e176-8caa-468d-bb78-01ffa9371d14 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:20:21.134186 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bda5e176-8caa-468d-bb78-01ffa9371d14 None admin] RBAC: Authorization granted {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:20:21.148717 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application [None req-bda5e176-8caa-468d-bb78-01ffa9371d14 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 21 08:20:21.148717 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 08:20:21.148717 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 08:20:21.148717 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 08:20:21.148717 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 08:20:21.148717 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 08:20:21.148717 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 08:20:21.148717 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 08:20:21.148717 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 08:20:21.148717 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 08:20:21.148717 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 08:20:21.148717 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 08:20:21.148717 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 21 08:20:21.148717 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 21 08:20:21.148717 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 21 08:20:21.149777 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 21 08:20:21.149777 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 08:20:21.149777 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 08:20:21.149777 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 08:20:21.149777 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 08:20:21.149777 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 08:20:21.149777 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 08:20:21.149777 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 08:20:21.149777 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application with Lock( Jul 21 08:20:21.149777 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 08:20:21.149777 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application return self._enter() Jul 21 08:20:21.149777 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 08:20:21.149777 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 08:20:21.149777 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 08:20:21.149777 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application return self.creator() Jul 21 08:20:21.149777 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 08:20:21.149777 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application created_value = creator( Jul 21 08:20:21.150873 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 21 08:20:21.150873 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 21 08:20:21.150873 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 21 08:20:21.150873 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 21 08:20:21.150873 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 21 08:20:21.150873 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 21 08:20:21.150873 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 21 08:20:21.150873 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application Jul 21 08:20:21.150873 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: [pid: 80245|app: 0|req: 634/1271] 10.222.0.47 () {56 vars in 1111 bytes} [Thu Jul 21 08:20:20 2022] GET /identity/v3/projects/service => generated 88 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 08:20:21.156783 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-911dfe35-51a4-4e3f-8ec0-ebd671ba2965 None admin] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:20:21.229140 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG dbcounter [-] [80244] Writing DB stats keystone:SELECT=100 {{(pid=80244) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:20:21.272288 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-911dfe35-51a4-4e3f-8ec0-ebd671ba2965 None admin] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:20:21.305108 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-312208e8-4c04-42f4-b4da-3e9a93d47894 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:20:21.305751 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-312208e8-4c04-42f4-b4da-3e9a93d47894 None admin] REQUEST_METHOD: `GET` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:20:21.306031 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-312208e8-4c04-42f4-b4da-3e9a93d47894 None admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:20:21.306300 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-312208e8-4c04-42f4-b4da-3e9a93d47894 None admin] PATH_INFO: `/v3/projects` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:20:21.307077 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-312208e8-4c04-42f4-b4da-3e9a93d47894 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80244) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 08:20:21.307408 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-312208e8-4c04-42f4-b4da-3e9a93d47894 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:20:21.308829 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-312208e8-4c04-42f4-b4da-3e9a93d47894 None admin] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:20:21.321237 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 638/1272] 10.222.0.47 () {56 vars in 1148 bytes} [Thu Jul 21 08:20:21 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 08:20:21.328006 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bda5e176-8caa-468d-bb78-01ffa9371d14 None admin] Authenticating user token {{(pid=80245) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:20:21.423930 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG dbcounter [-] [80245] Writing DB stats keystone:SELECT=100 {{(pid=80245) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:20:21.432957 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bda5e176-8caa-468d-bb78-01ffa9371d14 None admin] Validating token access rules against request {{(pid=80245) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:20:21.466531 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86aa9302-9f8d-4204-99a3-5febbe749310 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80245) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:20:21.468047 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86aa9302-9f8d-4204-99a3-5febbe749310 None admin] REQUEST_METHOD: `PUT` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:20:21.468332 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86aa9302-9f8d-4204-99a3-5febbe749310 None admin] SCRIPT_NAME: `/identity` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:20:21.468610 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86aa9302-9f8d-4204-99a3-5febbe749310 None admin] PATH_INFO: `/v3/projects/4faef13aa50e4e36b6c265fb1398fa14/users/3379b32543474d0a8e897484e85c7d04/roles/e29870f6e9e2411a9db0d5bf45f32ca5` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:20:21.470559 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86aa9302-9f8d-4204-99a3-5febbe749310 None admin] RBAC: Authorizing `identity:create_grant(project_id=4faef13aa50e4e36b6c265fb1398fa14, user_id=3379b32543474d0a8e897484e85c7d04, role_id=e29870f6e9e2411a9db0d5bf45f32ca5)` {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:20:21.472033 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86aa9302-9f8d-4204-99a3-5febbe749310 None admin] RBAC: Authorization granted {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:20:21.488466 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: [pid: 80245|app: 0|req: 635/1273] 10.222.0.47 () {58 vars in 1439 bytes} [Thu Jul 21 08:20:21 2022] PUT /identity/v3/projects/4faef13aa50e4e36b6c265fb1398fa14/users/3379b32543474d0a8e897484e85c7d04/roles/e29870f6e9e2411a9db0d5bf45f32ca5 => generated 0 bytes in 161 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 08:20:23.502210 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f206e8a-528d-4fcd-ad20-ddbfb4d49b16 None None] REQUEST_METHOD: `GET` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:20:23.502742 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f206e8a-528d-4fcd-ad20-ddbfb4d49b16 None None] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:20:23.502845 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f206e8a-528d-4fcd-ad20-ddbfb4d49b16 None None] PATH_INFO: `/` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:20:23.504357 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 639/1274] 10.222.0.47 () {54 vars in 926 bytes} [Thu Jul 21 08:20:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 08:20:23.510393 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aed5495-a4b5-4f6d-9897-8d2f47083d66 None None] REQUEST_METHOD: `POST` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:20:23.510805 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aed5495-a4b5-4f6d-9897-8d2f47083d66 None None] SCRIPT_NAME: `/identity` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:20:23.511156 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aed5495-a4b5-4f6d-9897-8d2f47083d66 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:20:23.528930 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.auth.core [None req-8aed5495-a4b5-4f6d-9897-8d2f47083d66 None None] MFA Rules not processed for user `2dce40b8d74c41ec94cc36e6ff05d412`. Rule list: `[]` (Enabled: `True`). {{(pid=80245) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 08:20:23.530137 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.fernet_utils [None req-8aed5495-a4b5-4f6d-9897-8d2f47083d66 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80245) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 08:20:23.595943 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: [pid: 80245|app: 0|req: 636/1275] 10.222.0.47 () {58 vars in 1023 bytes} [Thu Jul 21 08:20:23 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 21 08:20:23.619457 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f206e8a-528d-4fcd-ad20-ddbfb4d49b16 None None] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:20:23.716914 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f206e8a-528d-4fcd-ad20-ddbfb4d49b16 None None] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:20:23.733963 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG dbcounter [-] [80244] Writing DB stats keystone:SELECT=100 {{(pid=80244) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:20:23.750991 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13f17f85-9c53-435d-b9fe-30281d323b27 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:20:23.751751 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13f17f85-9c53-435d-b9fe-30281d323b27 None admin] REQUEST_METHOD: `GET` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:20:23.752029 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13f17f85-9c53-435d-b9fe-30281d323b27 None admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:20:23.752302 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13f17f85-9c53-435d-b9fe-30281d323b27 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:20:23.762412 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13f17f85-9c53-435d-b9fe-30281d323b27 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 21 08:20:23.769882 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application [None req-13f17f85-9c53-435d-b9fe-30281d323b27 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 21 08:20:23.769882 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 08:20:23.769882 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 08:20:23.769882 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 08:20:23.769882 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 08:20:23.769882 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 08:20:23.769882 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 08:20:23.769882 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 08:20:23.769882 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 08:20:23.769882 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 08:20:23.769882 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 08:20:23.769882 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 08:20:23.769882 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 21 08:20:23.769882 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 21 08:20:23.769882 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 21 08:20:23.769882 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 21 08:20:23.771065 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 21 08:20:23.771065 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 21 08:20:23.771065 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 21 08:20:23.771065 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 21 08:20:23.771065 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 08:20:23.771065 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 08:20:23.771065 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 08:20:23.771065 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 08:20:23.771065 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 08:20:23.771065 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 08:20:23.771065 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 08:20:23.771065 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application with Lock( Jul 21 08:20:23.771065 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 08:20:23.771065 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self._enter() Jul 21 08:20:23.771065 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 08:20:23.771065 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 08:20:23.772285 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 08:20:23.772285 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self.creator() Jul 21 08:20:23.772285 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 08:20:23.772285 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application created_value = creator( Jul 21 08:20:23.772285 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 21 08:20:23.772285 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 21 08:20:23.772285 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 21 08:20:23.772285 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 21 08:20:23.772285 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 21 08:20:23.772285 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 21 08:20:23.772285 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 21 08:20:23.772285 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application Jul 21 08:20:23.772285 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 640/1276] 10.222.0.47 () {56 vars in 1096 bytes} [Thu Jul 21 08:20:23 2022] GET /identity/v3/roles/admin => generated 83 bytes in 152 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 08:20:23.776966 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aed5495-a4b5-4f6d-9897-8d2f47083d66 None None] Authenticating user token {{(pid=80245) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:20:23.859559 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG dbcounter [-] [80245] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80245) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:20:23.880029 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aed5495-a4b5-4f6d-9897-8d2f47083d66 None None] Validating token access rules against request {{(pid=80245) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:20:23.913613 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2f52be5-0c0a-4746-8aca-3974fed228b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80245) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:20:23.914283 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2f52be5-0c0a-4746-8aca-3974fed228b8 None admin] REQUEST_METHOD: `GET` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:20:23.914567 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2f52be5-0c0a-4746-8aca-3974fed228b8 None admin] SCRIPT_NAME: `/identity` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:20:23.914839 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2f52be5-0c0a-4746-8aca-3974fed228b8 None admin] PATH_INFO: `/v3/roles` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:20:23.915605 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2f52be5-0c0a-4746-8aca-3974fed228b8 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80245) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 08:20:23.915909 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2f52be5-0c0a-4746-8aca-3974fed228b8 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:20:23.917267 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2f52be5-0c0a-4746-8aca-3974fed228b8 None admin] RBAC: Authorization granted {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:20:23.925812 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: [pid: 80245|app: 0|req: 637/1277] 10.222.0.47 () {56 vars in 1099 bytes} [Thu Jul 21 08:20:23 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 08:20:23.931543 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13f17f85-9c53-435d-b9fe-30281d323b27 None admin] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:20:24.030111 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13f17f85-9c53-435d-b9fe-30281d323b27 None admin] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:20:24.062951 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b0a32bc-614d-4d0d-b847-b2d62965d236 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:20:24.063770 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b0a32bc-614d-4d0d-b847-b2d62965d236 None admin] REQUEST_METHOD: `GET` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:20:24.064056 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b0a32bc-614d-4d0d-b847-b2d62965d236 None admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:20:24.064330 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b0a32bc-614d-4d0d-b847-b2d62965d236 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:20:24.074539 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b0a32bc-614d-4d0d-b847-b2d62965d236 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:20:24.076020 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b0a32bc-614d-4d0d-b847-b2d62965d236 None admin] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:20:24.093202 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application [None req-7b0a32bc-614d-4d0d-b847-b2d62965d236 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 08:20:24.093202 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 08:20:24.093202 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 08:20:24.093202 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 08:20:24.093202 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 08:20:24.093202 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 08:20:24.093202 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 08:20:24.093202 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 08:20:24.093202 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 08:20:24.093202 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 08:20:24.093202 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 08:20:24.093202 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 08:20:24.093202 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 21 08:20:24.093202 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 21 08:20:24.093202 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 21 08:20:24.094117 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 21 08:20:24.094117 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 08:20:24.094117 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 08:20:24.094117 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 08:20:24.094117 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 08:20:24.094117 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 08:20:24.094117 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 08:20:24.094117 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 08:20:24.094117 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application with Lock( Jul 21 08:20:24.094117 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 08:20:24.094117 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self._enter() Jul 21 08:20:24.094117 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 08:20:24.094117 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 08:20:24.094117 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 08:20:24.094117 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self.creator() Jul 21 08:20:24.094117 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 08:20:24.094117 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application created_value = creator( Jul 21 08:20:24.095471 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 21 08:20:24.095471 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 21 08:20:24.095471 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 08:20:24.095471 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application Jul 21 08:20:24.095471 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 641/1278] 10.222.0.47 () {56 vars in 1108 bytes} [Thu Jul 21 08:20:23 2022] GET /identity/v3/domains/Default => generated 87 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 08:20:24.100287 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2f52be5-0c0a-4746-8aca-3974fed228b8 None admin] Authenticating user token {{(pid=80245) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:20:24.222482 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2f52be5-0c0a-4746-8aca-3974fed228b8 None admin] Validating token access rules against request {{(pid=80245) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:20:24.255537 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e4fbf74-5d09-4308-bfa8-d2078ab633a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80245) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:20:24.256200 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e4fbf74-5d09-4308-bfa8-d2078ab633a8 None admin] REQUEST_METHOD: `GET` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:20:24.256482 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e4fbf74-5d09-4308-bfa8-d2078ab633a8 None admin] SCRIPT_NAME: `/identity` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:20:24.256767 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e4fbf74-5d09-4308-bfa8-d2078ab633a8 None admin] PATH_INFO: `/v3/domains` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:20:24.257481 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e4fbf74-5d09-4308-bfa8-d2078ab633a8 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80245) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 08:20:24.257790 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e4fbf74-5d09-4308-bfa8-d2078ab633a8 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:20:24.259443 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e4fbf74-5d09-4308-bfa8-d2078ab633a8 None admin] RBAC: Authorization granted {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:20:24.270858 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: [pid: 80245|app: 0|req: 638/1279] 10.222.0.47 () {56 vars in 1109 bytes} [Thu Jul 21 08:20:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 08:20:24.276496 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b0a32bc-614d-4d0d-b847-b2d62965d236 None admin] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:20:24.307133 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG dbcounter [-] [80244] Writing DB stats keystone:SELECT=100 {{(pid=80244) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:20:24.375860 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b0a32bc-614d-4d0d-b847-b2d62965d236 None admin] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:20:24.408713 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22294c97-7472-493e-96ca-d2ea0a45ab0d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:20:24.409547 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22294c97-7472-493e-96ca-d2ea0a45ab0d None admin] REQUEST_METHOD: `GET` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:20:24.409777 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22294c97-7472-493e-96ca-d2ea0a45ab0d None admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:20:24.410050 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22294c97-7472-493e-96ca-d2ea0a45ab0d None admin] PATH_INFO: `/v3/users/placement` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:20:24.417976 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22294c97-7472-493e-96ca-d2ea0a45ab0d None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:20:24.419451 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22294c97-7472-493e-96ca-d2ea0a45ab0d None admin] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:20:24.433254 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application [None req-22294c97-7472-493e-96ca-d2ea0a45ab0d None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Jul 21 08:20:24.433254 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 08:20:24.433254 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 08:20:24.433254 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 08:20:24.433254 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 08:20:24.433254 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 08:20:24.433254 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 08:20:24.433254 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 08:20:24.433254 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 08:20:24.433254 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 08:20:24.433254 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 08:20:24.433254 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 08:20:24.433254 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 21 08:20:24.433254 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 21 08:20:24.433254 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 21 08:20:24.433254 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 21 08:20:24.434404 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 08:20:24.434404 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 08:20:24.434404 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 21 08:20:24.434404 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 21 08:20:24.434404 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 21 08:20:24.434404 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 21 08:20:24.434404 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 08:20:24.434404 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 08:20:24.434404 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 08:20:24.434404 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 08:20:24.434404 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 08:20:24.434404 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application with Lock( Jul 21 08:20:24.434404 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 08:20:24.434404 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self._enter() Jul 21 08:20:24.434404 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 08:20:24.434404 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 08:20:24.434404 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 08:20:24.434404 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self.creator() Jul 21 08:20:24.435425 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 08:20:24.435425 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application created_value = creator( Jul 21 08:20:24.435425 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 21 08:20:24.435425 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 21 08:20:24.435425 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 21 08:20:24.435425 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 21 08:20:24.435425 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 21 08:20:24.435425 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 21 08:20:24.435425 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Jul 21 08:20:24.435425 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application Jul 21 08:20:24.435425 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 642/1280] 10.222.0.47 () {56 vars in 1108 bytes} [Thu Jul 21 08:20:24 2022] GET /identity/v3/users/placement => generated 87 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 08:20:24.440281 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e4fbf74-5d09-4308-bfa8-d2078ab633a8 None admin] Authenticating user token {{(pid=80245) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:20:24.444071 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG dbcounter [-] [80245] Writing DB stats keystone:SELECT=100 {{(pid=80245) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:20:24.556275 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e4fbf74-5d09-4308-bfa8-d2078ab633a8 None admin] Validating token access rules against request {{(pid=80245) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:20:24.589932 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80243e5b-e05a-45e5-b87c-f25592b52e8a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80245) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:20:24.590549 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80243e5b-e05a-45e5-b87c-f25592b52e8a None admin] REQUEST_METHOD: `GET` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:20:24.590833 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80243e5b-e05a-45e5-b87c-f25592b52e8a None admin] SCRIPT_NAME: `/identity` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:20:24.591112 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80243e5b-e05a-45e5-b87c-f25592b52e8a None admin] PATH_INFO: `/v3/users` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:20:24.592584 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80243e5b-e05a-45e5-b87c-f25592b52e8a None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80245) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 08:20:24.592896 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80243e5b-e05a-45e5-b87c-f25592b52e8a None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:20:24.594322 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80243e5b-e05a-45e5-b87c-f25592b52e8a None admin] RBAC: Authorization granted {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:20:24.611779 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: [pid: 80245|app: 0|req: 639/1281] 10.222.0.47 () {56 vars in 1143 bytes} [Thu Jul 21 08:20:24 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 08:20:24.617582 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22294c97-7472-493e-96ca-d2ea0a45ab0d None admin] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:20:24.706803 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG dbcounter [-] [80244] Writing DB stats keystone:SELECT=100 {{(pid=80244) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:20:24.716702 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22294c97-7472-493e-96ca-d2ea0a45ab0d None admin] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:20:24.749142 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a492f4fd-2835-4901-b747-32f9ca2f64e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:20:24.749788 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a492f4fd-2835-4901-b747-32f9ca2f64e4 None admin] REQUEST_METHOD: `GET` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:20:24.750079 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a492f4fd-2835-4901-b747-32f9ca2f64e4 None admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:20:24.750357 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a492f4fd-2835-4901-b747-32f9ca2f64e4 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:20:24.818511 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a492f4fd-2835-4901-b747-32f9ca2f64e4 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:20:24.820007 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a492f4fd-2835-4901-b747-32f9ca2f64e4 None admin] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:20:24.917583 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG dbcounter [-] [80244] Writing DB stats keystone:SELECT=100 {{(pid=80244) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:20:24.928336 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 643/1282] 10.222.0.47 () {58 vars in 1282 bytes} [Thu Jul 21 08:20:24 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 311 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 21 08:20:24.934499 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80243e5b-e05a-45e5-b87c-f25592b52e8a None admin] Authenticating user token {{(pid=80245) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:20:24.994497 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG dbcounter [-] [80245] Writing DB stats keystone:SELECT=100 {{(pid=80245) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:20:25.035396 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80243e5b-e05a-45e5-b87c-f25592b52e8a None admin] Validating token access rules against request {{(pid=80245) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:20:25.077407 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bfc3b6a-859f-4af5-8b83-5d49b97328b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80245) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:20:25.078200 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bfc3b6a-859f-4af5-8b83-5d49b97328b0 None admin] REQUEST_METHOD: `GET` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:20:25.078488 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bfc3b6a-859f-4af5-8b83-5d49b97328b0 None admin] SCRIPT_NAME: `/identity` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:20:25.078768 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bfc3b6a-859f-4af5-8b83-5d49b97328b0 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:20:25.088516 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bfc3b6a-859f-4af5-8b83-5d49b97328b0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:20:25.089956 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bfc3b6a-859f-4af5-8b83-5d49b97328b0 None admin] RBAC: Authorization granted {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:20:25.107137 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application [None req-3bfc3b6a-859f-4af5-8b83-5d49b97328b0 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 08:20:25.107137 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 08:20:25.107137 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 08:20:25.107137 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 08:20:25.107137 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 08:20:25.107137 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 08:20:25.107137 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 08:20:25.107137 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 08:20:25.107137 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 08:20:25.107137 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 08:20:25.107137 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 08:20:25.107137 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 08:20:25.107137 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 21 08:20:25.107137 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 21 08:20:25.107137 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 21 08:20:25.108738 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 21 08:20:25.108738 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 08:20:25.108738 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 08:20:25.108738 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 08:20:25.108738 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 08:20:25.108738 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 08:20:25.108738 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 08:20:25.108738 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 08:20:25.108738 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application with Lock( Jul 21 08:20:25.108738 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 08:20:25.108738 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application return self._enter() Jul 21 08:20:25.108738 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 08:20:25.108738 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 08:20:25.108738 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 08:20:25.108738 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application return self.creator() Jul 21 08:20:25.108738 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 08:20:25.108738 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application created_value = creator( Jul 21 08:20:25.109650 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 21 08:20:25.109650 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 21 08:20:25.109650 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 08:20:25.109650 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application Jul 21 08:20:25.109650 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: [pid: 80245|app: 0|req: 640/1283] 10.222.0.47 () {56 vars in 1108 bytes} [Thu Jul 21 08:20:24 2022] GET /identity/v3/domains/Default => generated 87 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 08:20:25.114623 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a492f4fd-2835-4901-b747-32f9ca2f64e4 None admin] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:20:25.225084 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a492f4fd-2835-4901-b747-32f9ca2f64e4 None admin] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:20:25.258891 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96560587-e456-46d4-85f2-7bd5d152813b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:20:25.259577 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96560587-e456-46d4-85f2-7bd5d152813b None admin] REQUEST_METHOD: `GET` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:20:25.259865 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96560587-e456-46d4-85f2-7bd5d152813b None admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:20:25.260145 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96560587-e456-46d4-85f2-7bd5d152813b None admin] PATH_INFO: `/v3/domains` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:20:25.260863 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96560587-e456-46d4-85f2-7bd5d152813b None admin] RBAC: Adding query filter params (name=Default) {{(pid=80244) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 08:20:25.261174 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96560587-e456-46d4-85f2-7bd5d152813b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:20:25.262541 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96560587-e456-46d4-85f2-7bd5d152813b None admin] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:20:25.273508 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 644/1284] 10.222.0.47 () {56 vars in 1109 bytes} [Thu Jul 21 08:20:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 08:20:25.279137 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bfc3b6a-859f-4af5-8b83-5d49b97328b0 None admin] Authenticating user token {{(pid=80245) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:20:25.375988 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bfc3b6a-859f-4af5-8b83-5d49b97328b0 None admin] Validating token access rules against request {{(pid=80245) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:20:25.389758 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG dbcounter [-] [80245] Writing DB stats keystone:SELECT=100 {{(pid=80245) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:20:25.409083 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a58c4500-9ebe-4915-bbb1-ab26df67fd35 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80245) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:20:25.409901 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a58c4500-9ebe-4915-bbb1-ab26df67fd35 None admin] REQUEST_METHOD: `GET` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:20:25.410181 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a58c4500-9ebe-4915-bbb1-ab26df67fd35 None admin] SCRIPT_NAME: `/identity` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:20:25.410455 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a58c4500-9ebe-4915-bbb1-ab26df67fd35 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:20:25.416993 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a58c4500-9ebe-4915-bbb1-ab26df67fd35 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:20:25.418426 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a58c4500-9ebe-4915-bbb1-ab26df67fd35 None admin] RBAC: Authorization granted {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:20:25.432373 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application [None req-a58c4500-9ebe-4915-bbb1-ab26df67fd35 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 21 08:20:25.432373 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 08:20:25.432373 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 08:20:25.432373 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 08:20:25.432373 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 08:20:25.432373 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 08:20:25.432373 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 08:20:25.432373 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 08:20:25.432373 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 08:20:25.432373 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 08:20:25.432373 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 08:20:25.432373 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 08:20:25.432373 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 21 08:20:25.432373 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 21 08:20:25.432373 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 21 08:20:25.433448 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 21 08:20:25.433448 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 08:20:25.433448 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 08:20:25.433448 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 08:20:25.433448 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 08:20:25.433448 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 08:20:25.433448 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 08:20:25.433448 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 08:20:25.433448 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application with Lock( Jul 21 08:20:25.433448 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 08:20:25.433448 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application return self._enter() Jul 21 08:20:25.433448 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 08:20:25.433448 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 08:20:25.433448 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 08:20:25.433448 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application return self.creator() Jul 21 08:20:25.433448 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 08:20:25.433448 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application created_value = creator( Jul 21 08:20:25.434605 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 21 08:20:25.434605 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 21 08:20:25.434605 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 21 08:20:25.434605 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 21 08:20:25.434605 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 21 08:20:25.434605 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 21 08:20:25.434605 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 21 08:20:25.434605 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: ERROR keystone.server.flask.application Jul 21 08:20:25.434605 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: [pid: 80245|app: 0|req: 641/1285] 10.222.0.47 () {56 vars in 1111 bytes} [Thu Jul 21 08:20:25 2022] GET /identity/v3/projects/service => generated 88 bytes in 155 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 08:20:25.439999 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96560587-e456-46d4-85f2-7bd5d152813b None admin] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:20:25.489786 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG dbcounter [-] [80244] Writing DB stats keystone:SELECT=100 {{(pid=80244) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:20:25.539409 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96560587-e456-46d4-85f2-7bd5d152813b None admin] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:20:25.572219 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dcc7438-e074-4922-9e1d-898bc4a1b7cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:20:25.572873 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dcc7438-e074-4922-9e1d-898bc4a1b7cb None admin] REQUEST_METHOD: `GET` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:20:25.573157 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dcc7438-e074-4922-9e1d-898bc4a1b7cb None admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:20:25.573431 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dcc7438-e074-4922-9e1d-898bc4a1b7cb None admin] PATH_INFO: `/v3/projects` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:20:25.574215 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dcc7438-e074-4922-9e1d-898bc4a1b7cb None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80244) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 08:20:25.574524 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dcc7438-e074-4922-9e1d-898bc4a1b7cb None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:20:25.575989 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dcc7438-e074-4922-9e1d-898bc4a1b7cb None admin] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:20:25.587653 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 645/1286] 10.222.0.47 () {56 vars in 1148 bytes} [Thu Jul 21 08:20:25 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 08:20:25.594252 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a58c4500-9ebe-4915-bbb1-ab26df67fd35 None admin] Authenticating user token {{(pid=80245) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:20:25.691548 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a58c4500-9ebe-4915-bbb1-ab26df67fd35 None admin] Validating token access rules against request {{(pid=80245) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:20:25.723894 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40674110-c2ed-4713-ba00-27d2903cb56f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80245) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:20:25.724549 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40674110-c2ed-4713-ba00-27d2903cb56f None admin] REQUEST_METHOD: `GET` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:20:25.724834 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40674110-c2ed-4713-ba00-27d2903cb56f None admin] SCRIPT_NAME: `/identity` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:20:25.725111 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40674110-c2ed-4713-ba00-27d2903cb56f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:20:25.725935 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40674110-c2ed-4713-ba00-27d2903cb56f None admin] RBAC: Adding query filter params (role.id=e29870f6e9e2411a9db0d5bf45f32ca5, scope.project.id=4faef13aa50e4e36b6c265fb1398fa14, user.id=3379b32543474d0a8e897484e85c7d04) {{(pid=80245) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 08:20:25.726249 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40674110-c2ed-4713-ba00-27d2903cb56f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:20:25.727716 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40674110-c2ed-4713-ba00-27d2903cb56f None admin] RBAC: Authorization granted {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:20:25.733995 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: [pid: 80245|app: 0|req: 642/1287] 10.222.0.47 () {56 vars in 1374 bytes} [Thu Jul 21 08:20:25 2022] GET /identity/v3/role_assignments?user.id=3379b32543474d0a8e897484e85c7d04&scope.project.id=4faef13aa50e4e36b6c265fb1398fa14&role.id=e29870f6e9e2411a9db0d5bf45f32ca5 => generated 608 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 08:20:27.783949 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eb92948-8409-4a4a-955c-6d42327ac8f9 None None] REQUEST_METHOD: `GET` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:20:27.784443 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eb92948-8409-4a4a-955c-6d42327ac8f9 None None] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:20:27.784443 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eb92948-8409-4a4a-955c-6d42327ac8f9 None None] PATH_INFO: `/` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:20:27.785790 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 646/1288] 10.222.0.47 () {54 vars in 926 bytes} [Thu Jul 21 08:20:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 08:20:27.790935 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65ec1811-3086-43f5-9e6c-ae92f4491d96 None None] REQUEST_METHOD: `POST` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:20:27.791372 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65ec1811-3086-43f5-9e6c-ae92f4491d96 None None] SCRIPT_NAME: `/identity` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:20:27.791521 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65ec1811-3086-43f5-9e6c-ae92f4491d96 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:20:27.807902 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.auth.core [None req-65ec1811-3086-43f5-9e6c-ae92f4491d96 None None] MFA Rules not processed for user `2dce40b8d74c41ec94cc36e6ff05d412`. Rule list: `[]` (Enabled: `True`). {{(pid=80245) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 08:20:27.809086 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.fernet_utils [None req-65ec1811-3086-43f5-9e6c-ae92f4491d96 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80245) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 08:20:27.839181 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG dbcounter [-] [80245] Writing DB stats keystone:SELECT=100 {{(pid=80245) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:20:27.879737 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: [pid: 80245|app: 0|req: 643/1289] 10.222.0.47 () {58 vars in 1023 bytes} [Thu Jul 21 08:20:27 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 21 08:20:27.902764 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9eb92948-8409-4a4a-955c-6d42327ac8f9 None None] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:20:28.002823 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9eb92948-8409-4a4a-955c-6d42327ac8f9 None None] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:20:28.014827 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG dbcounter [-] [80244] Writing DB stats keystone:SELECT=100 {{(pid=80244) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:20:28.038502 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeb336e2-5f4d-41bf-9888-b8250476a5eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:20:28.039221 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeb336e2-5f4d-41bf-9888-b8250476a5eb None admin] REQUEST_METHOD: `GET` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:20:28.039535 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeb336e2-5f4d-41bf-9888-b8250476a5eb None admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:20:28.039814 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeb336e2-5f4d-41bf-9888-b8250476a5eb None admin] PATH_INFO: `/v3/services/placement` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:20:28.040466 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aeb336e2-5f4d-41bf-9888-b8250476a5eb None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:20:28.041836 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aeb336e2-5f4d-41bf-9888-b8250476a5eb None admin] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:20:28.053716 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application [None req-aeb336e2-5f4d-41bf-9888-b8250476a5eb None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Jul 21 08:20:28.053716 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 08:20:28.053716 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 21 08:20:28.053716 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 21 08:20:28.053716 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 21 08:20:28.053716 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 21 08:20:28.053716 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 21 08:20:28.053716 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 21 08:20:28.053716 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Jul 21 08:20:28.053716 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application Jul 21 08:20:28.053716 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 21 08:20:28.053716 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application Jul 21 08:20:28.053716 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 08:20:28.053716 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 08:20:28.053716 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 08:20:28.053716 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 08:20:28.053716 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 08:20:28.054899 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 08:20:28.054899 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 08:20:28.054899 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 08:20:28.054899 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 08:20:28.054899 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 08:20:28.054899 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 08:20:28.054899 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 21 08:20:28.054899 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 21 08:20:28.054899 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 21 08:20:28.054899 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 21 08:20:28.054899 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 08:20:28.054899 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 08:20:28.054899 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 08:20:28.054899 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 08:20:28.054899 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 08:20:28.054899 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 08:20:28.055942 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 08:20:28.055942 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application with Lock( Jul 21 08:20:28.055942 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 08:20:28.055942 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self._enter() Jul 21 08:20:28.055942 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 08:20:28.055942 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 08:20:28.055942 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 08:20:28.055942 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self.creator() Jul 21 08:20:28.055942 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 08:20:28.055942 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application created_value = creator( Jul 21 08:20:28.055942 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 21 08:20:28.055942 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 21 08:20:28.055942 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Jul 21 08:20:28.055942 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application Jul 21 08:20:28.055942 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 647/1290] 10.222.0.47 () {56 vars in 1117 bytes} [Thu Jul 21 08:20:27 2022] GET /identity/v3/services/placement => generated 90 bytes in 153 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 08:20:28.061463 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65ec1811-3086-43f5-9e6c-ae92f4491d96 None None] Authenticating user token {{(pid=80245) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:20:28.157796 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65ec1811-3086-43f5-9e6c-ae92f4491d96 None None] Validating token access rules against request {{(pid=80245) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:20:28.201412 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aa0dc99-51c6-4cd7-af1a-b34353bd8927 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80245) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:20:28.202112 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aa0dc99-51c6-4cd7-af1a-b34353bd8927 None admin] REQUEST_METHOD: `GET` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:20:28.202383 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aa0dc99-51c6-4cd7-af1a-b34353bd8927 None admin] SCRIPT_NAME: `/identity` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:20:28.202655 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aa0dc99-51c6-4cd7-af1a-b34353bd8927 None admin] PATH_INFO: `/v3/services` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:20:28.203413 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5aa0dc99-51c6-4cd7-af1a-b34353bd8927 None admin] RBAC: Adding query filter params (name=placement) {{(pid=80245) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 08:20:28.203807 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5aa0dc99-51c6-4cd7-af1a-b34353bd8927 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:20:28.205377 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5aa0dc99-51c6-4cd7-af1a-b34353bd8927 None admin] RBAC: Authorization granted {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:20:28.210685 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: [pid: 80245|app: 0|req: 644/1291] 10.222.0.47 () {56 vars in 1116 bytes} [Thu Jul 21 08:20:28 2022] GET /identity/v3/services?name=placement => generated 126 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 08:20:28.218141 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeb336e2-5f4d-41bf-9888-b8250476a5eb None admin] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:20:28.320819 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeb336e2-5f4d-41bf-9888-b8250476a5eb None admin] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:20:28.352539 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81571b98-f9a4-4d0a-b3ce-881982933a01 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:20:28.353202 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81571b98-f9a4-4d0a-b3ce-881982933a01 None admin] REQUEST_METHOD: `GET` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:20:28.353487 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81571b98-f9a4-4d0a-b3ce-881982933a01 None admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:20:28.353767 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81571b98-f9a4-4d0a-b3ce-881982933a01 None admin] PATH_INFO: `/v3/services` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:20:28.354467 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81571b98-f9a4-4d0a-b3ce-881982933a01 None admin] RBAC: Adding query filter params (type=placement) {{(pid=80244) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 08:20:28.354772 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81571b98-f9a4-4d0a-b3ce-881982933a01 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:20:28.356163 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81571b98-f9a4-4d0a-b3ce-881982933a01 None admin] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:20:28.361046 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 648/1292] 10.222.0.47 () {56 vars in 1116 bytes} [Thu Jul 21 08:20:28 2022] GET /identity/v3/services?type=placement => generated 126 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 08:20:30.353609 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7592fb58-12a0-48af-b01e-378954235974 None None] REQUEST_METHOD: `GET` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:20:30.354156 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7592fb58-12a0-48af-b01e-378954235974 None None] SCRIPT_NAME: `/identity` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:20:30.354156 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7592fb58-12a0-48af-b01e-378954235974 None None] PATH_INFO: `/` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:20:30.355575 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: [pid: 80245|app: 0|req: 645/1293] 10.222.0.47 () {54 vars in 926 bytes} [Thu Jul 21 08:20:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 08:20:30.360632 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e1eee61-f624-4fbf-a163-e171b90af720 None None] REQUEST_METHOD: `POST` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:20:30.361024 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e1eee61-f624-4fbf-a163-e171b90af720 None None] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:20:30.361184 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e1eee61-f624-4fbf-a163-e171b90af720 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:20:30.377407 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.auth.core [None req-9e1eee61-f624-4fbf-a163-e171b90af720 None None] MFA Rules not processed for user `2dce40b8d74c41ec94cc36e6ff05d412`. Rule list: `[]` (Enabled: `True`). {{(pid=80244) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 08:20:30.378570 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.fernet_utils [None req-9e1eee61-f624-4fbf-a163-e171b90af720 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80244) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 08:20:30.408643 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG dbcounter [-] [80244] Writing DB stats keystone:SELECT=100 {{(pid=80244) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:20:30.444824 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 649/1294] 10.222.0.47 () {58 vars in 1023 bytes} [Thu Jul 21 08:20:30 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 21 08:20:30.468214 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7592fb58-12a0-48af-b01e-378954235974 None None] Authenticating user token {{(pid=80245) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:20:30.499049 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG dbcounter [-] [80245] Writing DB stats keystone:SELECT=100 {{(pid=80245) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:20:30.568857 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7592fb58-12a0-48af-b01e-378954235974 None None] Validating token access rules against request {{(pid=80245) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:20:30.602822 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a74f80b4-0efa-4242-854e-11a91d0653cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80245) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:20:30.603518 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a74f80b4-0efa-4242-854e-11a91d0653cb None admin] REQUEST_METHOD: `POST` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:20:30.603855 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a74f80b4-0efa-4242-854e-11a91d0653cb None admin] SCRIPT_NAME: `/identity` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:20:30.604081 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a74f80b4-0efa-4242-854e-11a91d0653cb None admin] PATH_INFO: `/v3/services` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:20:30.604803 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a74f80b4-0efa-4242-854e-11a91d0653cb None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:20:30.606195 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a74f80b4-0efa-4242-854e-11a91d0653cb None admin] RBAC: Authorization granted {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:20:30.619850 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: [pid: 80245|app: 0|req: 646/1295] 10.222.0.47 () {60 vars in 1141 bytes} [Thu Jul 21 08:20:30 2022] POST /identity/v3/services => generated 246 bytes in 153 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 21 08:20:32.684043 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a094de87-a574-41f1-aaa6-1d5929bbf920 None None] REQUEST_METHOD: `GET` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:20:32.684559 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a094de87-a574-41f1-aaa6-1d5929bbf920 None None] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:20:32.684653 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a094de87-a574-41f1-aaa6-1d5929bbf920 None None] PATH_INFO: `/` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:20:32.685970 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 650/1296] 10.222.0.47 () {54 vars in 926 bytes} [Thu Jul 21 08:20:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 08:20:32.691314 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0a5fec5-2dd2-4bd6-a2da-672237ce4304 None None] REQUEST_METHOD: `POST` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:20:32.691753 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0a5fec5-2dd2-4bd6-a2da-672237ce4304 None None] SCRIPT_NAME: `/identity` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:20:32.691898 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0a5fec5-2dd2-4bd6-a2da-672237ce4304 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:20:32.709070 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.auth.core [None req-a0a5fec5-2dd2-4bd6-a2da-672237ce4304 None None] MFA Rules not processed for user `2dce40b8d74c41ec94cc36e6ff05d412`. Rule list: `[]` (Enabled: `True`). {{(pid=80245) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 08:20:32.710251 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.fernet_utils [None req-a0a5fec5-2dd2-4bd6-a2da-672237ce4304 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80245) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 08:20:32.781553 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: [pid: 80245|app: 0|req: 647/1297] 10.222.0.47 () {58 vars in 1023 bytes} [Thu Jul 21 08:20:32 2022] POST /identity/v3/auth/tokens => generated 2211 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 21 08:20:32.803607 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a094de87-a574-41f1-aaa6-1d5929bbf920 None None] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:20:32.920252 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a094de87-a574-41f1-aaa6-1d5929bbf920 None None] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:20:32.957556 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89eff2ef-37a1-42b9-a3fe-037a22c71e0e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:20:32.958358 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89eff2ef-37a1-42b9-a3fe-037a22c71e0e None admin] REQUEST_METHOD: `GET` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:20:32.958648 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89eff2ef-37a1-42b9-a3fe-037a22c71e0e None admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:20:32.958921 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89eff2ef-37a1-42b9-a3fe-037a22c71e0e None admin] PATH_INFO: `/v3/services/placement` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:20:32.959593 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89eff2ef-37a1-42b9-a3fe-037a22c71e0e None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:20:32.960958 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89eff2ef-37a1-42b9-a3fe-037a22c71e0e None admin] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:20:32.972615 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application [None req-89eff2ef-37a1-42b9-a3fe-037a22c71e0e None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Jul 21 08:20:32.972615 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 08:20:32.972615 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 21 08:20:32.972615 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 21 08:20:32.972615 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 21 08:20:32.972615 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 21 08:20:32.972615 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 21 08:20:32.972615 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 21 08:20:32.972615 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Jul 21 08:20:32.972615 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application Jul 21 08:20:32.972615 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 21 08:20:32.972615 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application Jul 21 08:20:32.972615 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 08:20:32.972615 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 08:20:32.972615 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 08:20:32.972615 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 08:20:32.972615 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 08:20:32.973586 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 08:20:32.973586 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 08:20:32.973586 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 08:20:32.973586 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 08:20:32.973586 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 08:20:32.973586 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 08:20:32.973586 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 21 08:20:32.973586 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 21 08:20:32.973586 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 21 08:20:32.973586 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 21 08:20:32.973586 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 08:20:32.973586 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 08:20:32.973586 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 08:20:32.973586 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 08:20:32.973586 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 08:20:32.973586 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 08:20:32.974443 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 08:20:32.974443 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application with Lock( Jul 21 08:20:32.974443 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 08:20:32.974443 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self._enter() Jul 21 08:20:32.974443 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 08:20:32.974443 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 08:20:32.974443 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 08:20:32.974443 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self.creator() Jul 21 08:20:32.974443 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 08:20:32.974443 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application created_value = creator( Jul 21 08:20:32.974443 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 21 08:20:32.974443 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 21 08:20:32.974443 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Jul 21 08:20:32.974443 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application Jul 21 08:20:32.974443 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 651/1298] 10.222.0.47 () {56 vars in 1117 bytes} [Thu Jul 21 08:20:32 2022] GET /identity/v3/services/placement => generated 90 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 08:20:32.979821 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0a5fec5-2dd2-4bd6-a2da-672237ce4304 None None] Authenticating user token {{(pid=80245) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:20:32.997439 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG dbcounter [-] [80245] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80245) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:20:33.084369 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0a5fec5-2dd2-4bd6-a2da-672237ce4304 None None] Validating token access rules against request {{(pid=80245) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:20:33.120056 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc57ec80-89f9-4529-a3a6-ce866444821f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80245) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:20:33.120716 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc57ec80-89f9-4529-a3a6-ce866444821f None admin] REQUEST_METHOD: `GET` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:20:33.120999 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc57ec80-89f9-4529-a3a6-ce866444821f None admin] SCRIPT_NAME: `/identity` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:20:33.121273 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc57ec80-89f9-4529-a3a6-ce866444821f None admin] PATH_INFO: `/v3/services` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:20:33.121976 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc57ec80-89f9-4529-a3a6-ce866444821f None admin] RBAC: Adding query filter params (name=placement) {{(pid=80245) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 08:20:33.122335 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc57ec80-89f9-4529-a3a6-ce866444821f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:20:33.124203 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc57ec80-89f9-4529-a3a6-ce866444821f None admin] RBAC: Authorization granted {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:20:33.129241 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: [pid: 80245|app: 0|req: 648/1299] 10.222.0.47 () {56 vars in 1116 bytes} [Thu Jul 21 08:20:32 2022] GET /identity/v3/services?name=placement => generated 358 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 08:20:33.135560 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89eff2ef-37a1-42b9-a3fe-037a22c71e0e None admin] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:20:33.170389 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG dbcounter [-] [80244] Writing DB stats keystone:SELECT=100 {{(pid=80244) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:20:33.257278 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89eff2ef-37a1-42b9-a3fe-037a22c71e0e None admin] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:20:33.292857 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96b69207-579b-43b8-beef-1a5223c6161c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:20:33.293525 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96b69207-579b-43b8-beef-1a5223c6161c None admin] REQUEST_METHOD: `GET` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:20:33.293803 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96b69207-579b-43b8-beef-1a5223c6161c None admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:20:33.294071 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96b69207-579b-43b8-beef-1a5223c6161c None admin] PATH_INFO: `/v3/endpoints` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:20:33.294890 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96b69207-579b-43b8-beef-1a5223c6161c None admin] RBAC: Adding query filter params (interface=public, service_id=5f4fb1ec09de40bbb09d4cc289895164, region_id=RegionOne) {{(pid=80244) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 08:20:33.295201 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96b69207-579b-43b8-beef-1a5223c6161c None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:20:33.296610 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96b69207-579b-43b8-beef-1a5223c6161c None admin] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:20:33.301599 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 652/1300] 10.222.0.47 () {56 vars in 1251 bytes} [Thu Jul 21 08:20:33 2022] GET /identity/v3/endpoints?service_id=5f4fb1ec09de40bbb09d4cc289895164&interface=public®ion_id=RegionOne => generated 194 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 08:20:33.307462 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc57ec80-89f9-4529-a3a6-ce866444821f None admin] Authenticating user token {{(pid=80245) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:20:33.394659 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG dbcounter [-] [80245] Writing DB stats keystone:SELECT=100 {{(pid=80245) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:20:33.418939 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc57ec80-89f9-4529-a3a6-ce866444821f None admin] Validating token access rules against request {{(pid=80245) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:20:33.455195 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2eaa6b3-4869-4106-b712-2898ddf03870 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80245) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:20:33.455907 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2eaa6b3-4869-4106-b712-2898ddf03870 None admin] REQUEST_METHOD: `GET` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:20:33.456194 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2eaa6b3-4869-4106-b712-2898ddf03870 None admin] SCRIPT_NAME: `/identity` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:20:33.456472 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2eaa6b3-4869-4106-b712-2898ddf03870 None admin] PATH_INFO: `/v3/services` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:20:33.457162 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2eaa6b3-4869-4106-b712-2898ddf03870 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:20:33.458536 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2eaa6b3-4869-4106-b712-2898ddf03870 None admin] RBAC: Authorization granted {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:20:33.465524 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: [pid: 80245|app: 0|req: 649/1301] 10.222.0.47 () {56 vars in 1087 bytes} [Thu Jul 21 08:20:33 2022] GET /identity/v3/services => generated 2191 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 08:20:35.514943 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-023040e5-43b0-4cff-8d6e-8f5aa938b718 None None] REQUEST_METHOD: `GET` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:20:35.515475 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-023040e5-43b0-4cff-8d6e-8f5aa938b718 None None] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:20:35.515584 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-023040e5-43b0-4cff-8d6e-8f5aa938b718 None None] PATH_INFO: `/` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:20:35.516905 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 653/1302] 10.222.0.47 () {54 vars in 926 bytes} [Thu Jul 21 08:20:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 08:20:35.522909 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2e8abd7-25b6-460b-a888-fa0f0debd2c1 None None] REQUEST_METHOD: `POST` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:20:35.523441 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2e8abd7-25b6-460b-a888-fa0f0debd2c1 None None] SCRIPT_NAME: `/identity` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:20:35.523533 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2e8abd7-25b6-460b-a888-fa0f0debd2c1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:20:35.540612 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.auth.core [None req-d2e8abd7-25b6-460b-a888-fa0f0debd2c1 None None] MFA Rules not processed for user `2dce40b8d74c41ec94cc36e6ff05d412`. Rule list: `[]` (Enabled: `True`). {{(pid=80245) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 08:20:35.541803 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.fernet_utils [None req-d2e8abd7-25b6-460b-a888-fa0f0debd2c1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80245) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 08:20:35.608299 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: [pid: 80245|app: 0|req: 650/1303] 10.222.0.47 () {58 vars in 1023 bytes} [Thu Jul 21 08:20:35 2022] POST /identity/v3/auth/tokens => generated 2211 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 21 08:20:35.630599 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-023040e5-43b0-4cff-8d6e-8f5aa938b718 None None] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:20:35.724431 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG dbcounter [-] [80244] Writing DB stats keystone:SELECT=100 {{(pid=80244) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:20:35.731683 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-023040e5-43b0-4cff-8d6e-8f5aa938b718 None None] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:20:35.764902 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-334fa31b-4909-4e70-9775-d3bfa7faa8db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:20:35.765640 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-334fa31b-4909-4e70-9775-d3bfa7faa8db None admin] REQUEST_METHOD: `GET` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:20:35.765924 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-334fa31b-4909-4e70-9775-d3bfa7faa8db None admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:20:35.766199 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-334fa31b-4909-4e70-9775-d3bfa7faa8db None admin] PATH_INFO: `/v3/services/placement` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:20:35.766845 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-334fa31b-4909-4e70-9775-d3bfa7faa8db None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:20:35.768240 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-334fa31b-4909-4e70-9775-d3bfa7faa8db None admin] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:20:35.780421 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application [None req-334fa31b-4909-4e70-9775-d3bfa7faa8db None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Jul 21 08:20:35.780421 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 08:20:35.780421 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 21 08:20:35.780421 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 21 08:20:35.780421 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 21 08:20:35.780421 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 21 08:20:35.780421 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 21 08:20:35.780421 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 21 08:20:35.780421 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Jul 21 08:20:35.780421 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application Jul 21 08:20:35.780421 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 21 08:20:35.780421 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application Jul 21 08:20:35.780421 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 08:20:35.780421 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 08:20:35.780421 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 08:20:35.780421 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 08:20:35.780421 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 08:20:35.781421 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 08:20:35.781421 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 08:20:35.781421 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 08:20:35.781421 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 08:20:35.781421 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 08:20:35.781421 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 08:20:35.781421 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 21 08:20:35.781421 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 21 08:20:35.781421 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 21 08:20:35.781421 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 21 08:20:35.781421 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 08:20:35.781421 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 08:20:35.781421 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 08:20:35.781421 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 08:20:35.781421 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 08:20:35.781421 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 08:20:35.782264 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 08:20:35.782264 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application with Lock( Jul 21 08:20:35.782264 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 08:20:35.782264 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self._enter() Jul 21 08:20:35.782264 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 08:20:35.782264 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 08:20:35.782264 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 08:20:35.782264 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self.creator() Jul 21 08:20:35.782264 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 08:20:35.782264 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application created_value = creator( Jul 21 08:20:35.782264 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 21 08:20:35.782264 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 21 08:20:35.782264 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Jul 21 08:20:35.782264 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application Jul 21 08:20:35.782264 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 654/1304] 10.222.0.47 () {56 vars in 1117 bytes} [Thu Jul 21 08:20:35 2022] GET /identity/v3/services/placement => generated 90 bytes in 151 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 08:20:35.787626 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2e8abd7-25b6-460b-a888-fa0f0debd2c1 None None] Authenticating user token {{(pid=80245) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:20:35.860719 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG dbcounter [-] [80245] Writing DB stats keystone:SELECT=100 {{(pid=80245) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:20:35.883191 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2e8abd7-25b6-460b-a888-fa0f0debd2c1 None None] Validating token access rules against request {{(pid=80245) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:20:35.918128 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-935713bc-9fcc-4455-827b-65ad4a541034 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80245) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:20:35.918797 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-935713bc-9fcc-4455-827b-65ad4a541034 None admin] REQUEST_METHOD: `GET` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:20:35.919083 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-935713bc-9fcc-4455-827b-65ad4a541034 None admin] SCRIPT_NAME: `/identity` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:20:35.919383 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-935713bc-9fcc-4455-827b-65ad4a541034 None admin] PATH_INFO: `/v3/services` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:20:35.920115 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-935713bc-9fcc-4455-827b-65ad4a541034 None admin] RBAC: Adding query filter params (name=placement) {{(pid=80245) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 08:20:35.920424 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-935713bc-9fcc-4455-827b-65ad4a541034 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:20:35.921789 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-935713bc-9fcc-4455-827b-65ad4a541034 None admin] RBAC: Authorization granted {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:20:35.926891 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: [pid: 80245|app: 0|req: 651/1305] 10.222.0.47 () {56 vars in 1116 bytes} [Thu Jul 21 08:20:35 2022] GET /identity/v3/services?name=placement => generated 358 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 08:20:35.932594 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-334fa31b-4909-4e70-9775-d3bfa7faa8db None admin] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:20:36.031520 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-334fa31b-4909-4e70-9775-d3bfa7faa8db None admin] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:20:36.065035 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-081e9c6a-ab60-40af-bf79-3fff645871bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:20:36.065689 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-081e9c6a-ab60-40af-bf79-3fff645871bb None admin] REQUEST_METHOD: `POST` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:20:36.065974 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-081e9c6a-ab60-40af-bf79-3fff645871bb None admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:20:36.066328 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-081e9c6a-ab60-40af-bf79-3fff645871bb None admin] PATH_INFO: `/v3/endpoints` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:20:36.067007 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-081e9c6a-ab60-40af-bf79-3fff645871bb None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:20:36.068402 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-081e9c6a-ab60-40af-bf79-3fff645871bb None admin] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:20:36.086604 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 655/1306] 10.222.0.47 () {60 vars in 1144 bytes} [Thu Jul 21 08:20:35 2022] POST /identity/v3/endpoints => generated 331 bytes in 154 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 08:20:45.924312 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG dbcounter [-] [80245] Writing DB stats keystone:SELECT=30 {{(pid=80245) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:20:46.082772 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG dbcounter [-] [80244] Writing DB stats keystone:SELECT=91,keystone:INSERT=1 {{(pid=80244) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:21:39.631977 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bc71706-3c0c-4ec2-a23c-5917d58e5018 None None] REQUEST_METHOD: `GET` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:21:39.633916 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bc71706-3c0c-4ec2-a23c-5917d58e5018 None None] SCRIPT_NAME: `/identity` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:21:39.635526 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bc71706-3c0c-4ec2-a23c-5917d58e5018 None None] PATH_INFO: `/` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:21:39.643188 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-783f409c-0391-49aa-a4ba-053cefdc3f1d None None] REQUEST_METHOD: `POST` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:21:39.643711 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-783f409c-0391-49aa-a4ba-053cefdc3f1d None None] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:21:39.643800 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-783f409c-0391-49aa-a4ba-053cefdc3f1d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:21:39.647396 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: [pid: 80245|app: 0|req: 652/1307] 10.222.0.47 () {54 vars in 926 bytes} [Thu Jul 21 08:21:39 2022] GET /identity => generated 270 bytes in 20 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 08:21:39.756424 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.auth.core [None req-783f409c-0391-49aa-a4ba-053cefdc3f1d None None] MFA Rules not processed for user `2dce40b8d74c41ec94cc36e6ff05d412`. Rule list: `[]` (Enabled: `True`). {{(pid=80244) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 08:21:39.757844 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.fernet_utils [None req-783f409c-0391-49aa-a4ba-053cefdc3f1d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80244) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 08:21:39.908615 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 656/1308] 10.222.0.47 () {58 vars in 1023 bytes} [Thu Jul 21 08:21:39 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 258 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 21 08:21:39.949010 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99bce977-5349-4aa1-9789-73e5b892bf74 None None] REQUEST_METHOD: `GET` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:21:39.949290 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99bce977-5349-4aa1-9789-73e5b892bf74 None None] SCRIPT_NAME: `/identity` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:21:39.949770 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99bce977-5349-4aa1-9789-73e5b892bf74 None None] PATH_INFO: `/` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:21:39.951380 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: [pid: 80245|app: 0|req: 653/1309] 10.222.0.47 () {54 vars in 965 bytes} [Thu Jul 21 08:21:39 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 08:21:39.958191 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbe3b0d0-de87-473c-afab-f5887a4a9276 None None] REQUEST_METHOD: `POST` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:21:39.958498 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbe3b0d0-de87-473c-afab-f5887a4a9276 None None] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:21:39.958912 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbe3b0d0-de87-473c-afab-f5887a4a9276 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:21:40.047452 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.auth.core [None req-dbe3b0d0-de87-473c-afab-f5887a4a9276 None None] MFA Rules not processed for user `20ca3c3bd3774691b3374cfbfec6e494`. Rule list: `[]` (Enabled: `True`). {{(pid=80244) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 08:21:40.048646 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.fernet_utils [None req-dbe3b0d0-de87-473c-afab-f5887a4a9276 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80244) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 08:21:40.072100 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG dbcounter [-] [80244] Writing DB stats keystone:SELECT=100 {{(pid=80244) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:21:40.133981 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 657/1310] 10.222.0.47 () {58 vars in 1062 bytes} [Thu Jul 21 08:21:39 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 177 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 08:21:40.144742 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99bce977-5349-4aa1-9789-73e5b892bf74 None None] Authenticating user token {{(pid=80245) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:21:40.146540 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.fernet_utils [None req-99bce977-5349-4aa1-9789-73e5b892bf74 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80245) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 08:21:40.273219 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99bce977-5349-4aa1-9789-73e5b892bf74 None None] Validating token access rules against request {{(pid=80245) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:21:40.354381 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abf4c50e-c13b-4fca-8aed-2f68cc9019c5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20ca3c3bd3774691b3374cfbfec6e494', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4faef13aa50e4e36b6c265fb1398fa14', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80245) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:21:40.354994 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abf4c50e-c13b-4fca-8aed-2f68cc9019c5 service neutron] REQUEST_METHOD: `GET` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:21:40.355282 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abf4c50e-c13b-4fca-8aed-2f68cc9019c5 service neutron] SCRIPT_NAME: `/identity` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:21:40.355605 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abf4c50e-c13b-4fca-8aed-2f68cc9019c5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:21:40.357266 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.fernet_utils [None req-abf4c50e-c13b-4fca-8aed-2f68cc9019c5 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80245) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 08:21:40.414438 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG dbcounter [-] [80245] Writing DB stats keystone:SELECT=100 {{(pid=80245) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:21:40.475089 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abf4c50e-c13b-4fca-8aed-2f68cc9019c5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:21:40.476592 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abf4c50e-c13b-4fca-8aed-2f68cc9019c5 service neutron] RBAC: Authorization granted {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:21:40.501441 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: [pid: 80245|app: 0|req: 654/1311] 10.222.0.47 () {60 vars in 1365 bytes} [Thu Jul 21 08:21:40 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 357 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 21 08:21:42.875265 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10ef6183-c08c-4f6a-b375-a0ba592a618c None None] REQUEST_METHOD: `GET` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:21:42.875830 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10ef6183-c08c-4f6a-b375-a0ba592a618c None None] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:21:42.875830 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10ef6183-c08c-4f6a-b375-a0ba592a618c None None] PATH_INFO: `/` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:21:42.883150 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16808cf9-0d2d-4576-84ec-3d97cdbb9de4 None None] REQUEST_METHOD: `POST` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:21:42.883932 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16808cf9-0d2d-4576-84ec-3d97cdbb9de4 None None] SCRIPT_NAME: `/identity` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:21:42.884027 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16808cf9-0d2d-4576-84ec-3d97cdbb9de4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:21:42.887542 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 658/1312] 10.222.0.47 () {54 vars in 926 bytes} [Thu Jul 21 08:21:42 2022] GET /identity => generated 270 bytes in 13 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 08:21:42.978132 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.auth.core [None req-16808cf9-0d2d-4576-84ec-3d97cdbb9de4 None None] MFA Rules not processed for user `2dce40b8d74c41ec94cc36e6ff05d412`. Rule list: `[]` (Enabled: `True`). {{(pid=80245) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 08:21:42.980110 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.fernet_utils [None req-16808cf9-0d2d-4576-84ec-3d97cdbb9de4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80245) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 08:21:43.107086 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: [pid: 80245|app: 0|req: 655/1313] 10.222.0.47 () {58 vars in 1023 bytes} [Thu Jul 21 08:21:42 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 225 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 21 08:21:43.133043 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10ef6183-c08c-4f6a-b375-a0ba592a618c None None] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:21:43.135134 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.fernet_utils [None req-10ef6183-c08c-4f6a-b375-a0ba592a618c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80244) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 08:21:43.266354 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10ef6183-c08c-4f6a-b375-a0ba592a618c None None] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:21:43.348354 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG dbcounter [-] [80244] Writing DB stats keystone:SELECT=100 {{(pid=80244) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:21:43.351235 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc4de46e-bac5-437e-a9bb-a910308c00be service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20ca3c3bd3774691b3374cfbfec6e494', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4faef13aa50e4e36b6c265fb1398fa14', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:21:43.351967 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc4de46e-bac5-437e-a9bb-a910308c00be service neutron] REQUEST_METHOD: `GET` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:21:43.352256 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc4de46e-bac5-437e-a9bb-a910308c00be service neutron] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:21:43.352561 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc4de46e-bac5-437e-a9bb-a910308c00be service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:21:43.354792 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.fernet_utils [None req-bc4de46e-bac5-437e-a9bb-a910308c00be service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80244) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 08:21:43.471058 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc4de46e-bac5-437e-a9bb-a910308c00be service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:21:43.472567 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc4de46e-bac5-437e-a9bb-a910308c00be service neutron] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:21:43.494088 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 659/1314] 10.222.0.47 () {60 vars in 1365 bytes} [Thu Jul 21 08:21:43 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 362 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 08:21:45.862784 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40f06c57-a37b-4911-b8a2-10602f7c26c3 None None] REQUEST_METHOD: `GET` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:21:45.863371 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40f06c57-a37b-4911-b8a2-10602f7c26c3 None None] SCRIPT_NAME: `/identity` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:21:45.863371 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40f06c57-a37b-4911-b8a2-10602f7c26c3 None None] PATH_INFO: `/` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:21:45.864756 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: [pid: 80245|app: 0|req: 656/1315] 10.222.0.47 () {54 vars in 926 bytes} [Thu Jul 21 08:21:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 08:21:45.870509 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b6195a0-4170-4e59-a497-ba31025b6c7d None None] REQUEST_METHOD: `POST` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:21:45.870888 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b6195a0-4170-4e59-a497-ba31025b6c7d None None] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:21:45.871059 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b6195a0-4170-4e59-a497-ba31025b6c7d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:21:45.937471 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.auth.core [None req-8b6195a0-4170-4e59-a497-ba31025b6c7d None None] MFA Rules not processed for user `39bf6631ce574c758a674faedecf2254`. Rule list: `[]` (Enabled: `True`). {{(pid=80244) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 08:21:45.938684 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.fernet_utils [None req-8b6195a0-4170-4e59-a497-ba31025b6c7d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80244) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 08:21:45.989321 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG dbcounter [-] [80244] Writing DB stats keystone:SELECT=100 {{(pid=80244) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:21:46.034697 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 660/1316] 10.222.0.47 () {58 vars in 1023 bytes} [Thu Jul 21 08:21:45 2022] POST /identity/v3/auth/tokens => generated 3233 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 08:21:46.090505 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d7f8bc8-1e97-46e3-a860-07fabe73e72c None None] REQUEST_METHOD: `GET` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:21:46.090801 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d7f8bc8-1e97-46e3-a860-07fabe73e72c None None] SCRIPT_NAME: `/identity` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:21:46.091081 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d7f8bc8-1e97-46e3-a860-07fabe73e72c None None] PATH_INFO: `/` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:21:46.092507 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: [pid: 80245|app: 0|req: 657/1317] 10.222.0.47 () {54 vars in 965 bytes} [Thu Jul 21 08:21:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 08:21:46.098377 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2b4362a-287b-477a-9e49-84d79582e4f5 None None] REQUEST_METHOD: `POST` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:21:46.098685 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2b4362a-287b-477a-9e49-84d79582e4f5 None None] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:21:46.098962 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2b4362a-287b-477a-9e49-84d79582e4f5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:21:46.175536 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.auth.core [None req-f2b4362a-287b-477a-9e49-84d79582e4f5 None None] MFA Rules not processed for user `20ca3c3bd3774691b3374cfbfec6e494`. Rule list: `[]` (Enabled: `True`). {{(pid=80244) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 08:21:46.176758 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.fernet_utils [None req-f2b4362a-287b-477a-9e49-84d79582e4f5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80244) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 08:21:46.251856 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 661/1318] 10.222.0.47 () {58 vars in 1062 bytes} [Thu Jul 21 08:21:46 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 154 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 08:21:46.260957 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d7f8bc8-1e97-46e3-a860-07fabe73e72c None None] Authenticating user token {{(pid=80245) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:21:46.262504 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.fernet_utils [None req-5d7f8bc8-1e97-46e3-a860-07fabe73e72c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80245) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 08:21:46.272279 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG dbcounter [-] [80245] Writing DB stats keystone:SELECT=100 {{(pid=80245) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:21:46.364052 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d7f8bc8-1e97-46e3-a860-07fabe73e72c None None] Validating token access rules against request {{(pid=80245) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:21:46.437713 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ae43f60-dadf-403f-ba2b-a8f638cbfab5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20ca3c3bd3774691b3374cfbfec6e494', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4faef13aa50e4e36b6c265fb1398fa14', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80245) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:21:46.438275 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ae43f60-dadf-403f-ba2b-a8f638cbfab5 service neutron] REQUEST_METHOD: `GET` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:21:46.438557 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ae43f60-dadf-403f-ba2b-a8f638cbfab5 service neutron] SCRIPT_NAME: `/identity` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:21:46.438831 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ae43f60-dadf-403f-ba2b-a8f638cbfab5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:21:46.440484 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.fernet_utils [None req-3ae43f60-dadf-403f-ba2b-a8f638cbfab5 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80245) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 08:21:46.502817 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG dbcounter [-] [80245] Writing DB stats keystone:SELECT=100 {{(pid=80245) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:21:46.550698 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ae43f60-dadf-403f-ba2b-a8f638cbfab5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:21:46.552180 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ae43f60-dadf-403f-ba2b-a8f638cbfab5 service neutron] RBAC: Authorization granted {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:21:46.570089 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: [pid: 80245|app: 0|req: 658/1319] 10.222.0.47 () {60 vars in 1365 bytes} [Thu Jul 21 08:21:46 2022] GET /identity/v3/auth/tokens => generated 3233 bytes in 310 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 08:21:49.572178 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaee6665-2661-4efb-94ef-3b8500890056 None None] REQUEST_METHOD: `GET` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:21:49.572690 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaee6665-2661-4efb-94ef-3b8500890056 None None] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:21:49.572690 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaee6665-2661-4efb-94ef-3b8500890056 None None] PATH_INFO: `/` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:21:49.574015 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 662/1320] 10.222.0.47 () {54 vars in 926 bytes} [Thu Jul 21 08:21:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 08:21:49.579144 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1b53980-9071-4868-85fd-f4360d57422c None None] REQUEST_METHOD: `POST` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:21:49.579560 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1b53980-9071-4868-85fd-f4360d57422c None None] SCRIPT_NAME: `/identity` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:21:49.579755 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1b53980-9071-4868-85fd-f4360d57422c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:21:49.643950 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.auth.core [None req-c1b53980-9071-4868-85fd-f4360d57422c None None] MFA Rules not processed for user `2dce40b8d74c41ec94cc36e6ff05d412`. Rule list: `[]` (Enabled: `True`). {{(pid=80245) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 08:21:49.645180 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.fernet_utils [None req-c1b53980-9071-4868-85fd-f4360d57422c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80245) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 08:21:49.739575 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: [pid: 80245|app: 0|req: 659/1321] 10.222.0.47 () {58 vars in 1023 bytes} [Thu Jul 21 08:21:49 2022] POST /identity/v3/auth/tokens => generated 3228 bytes in 161 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 08:21:49.779086 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaee6665-2661-4efb-94ef-3b8500890056 None None] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:21:49.780630 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.fernet_utils [None req-aaee6665-2661-4efb-94ef-3b8500890056 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80244) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 08:21:49.831651 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG dbcounter [-] [80244] Writing DB stats keystone:SELECT=100 {{(pid=80244) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:21:49.883990 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaee6665-2661-4efb-94ef-3b8500890056 None None] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:21:49.959380 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e2c2451-20c9-473d-bea5-171dd57c1824 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20ca3c3bd3774691b3374cfbfec6e494', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4faef13aa50e4e36b6c265fb1398fa14', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:21:49.960032 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e2c2451-20c9-473d-bea5-171dd57c1824 service neutron] REQUEST_METHOD: `GET` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:21:49.960318 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e2c2451-20c9-473d-bea5-171dd57c1824 service neutron] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:21:49.960595 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e2c2451-20c9-473d-bea5-171dd57c1824 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:21:49.962239 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.fernet_utils [None req-0e2c2451-20c9-473d-bea5-171dd57c1824 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80244) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 08:21:50.056723 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG dbcounter [-] [80244] Writing DB stats keystone:SELECT=100 {{(pid=80244) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:21:50.070675 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e2c2451-20c9-473d-bea5-171dd57c1824 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:21:50.072149 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e2c2451-20c9-473d-bea5-171dd57c1824 service neutron] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:21:50.091621 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 663/1322] 10.222.0.47 () {60 vars in 1365 bytes} [Thu Jul 21 08:21:49 2022] GET /identity/v3/auth/tokens => generated 3228 bytes in 313 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 08:21:52.860168 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d48c27b5-2f28-4435-9c9b-f716208be092 None None] REQUEST_METHOD: `GET` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:21:52.860741 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d48c27b5-2f28-4435-9c9b-f716208be092 None None] SCRIPT_NAME: `/identity` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:21:52.861042 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d48c27b5-2f28-4435-9c9b-f716208be092 None None] PATH_INFO: `/` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:21:52.863185 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: [pid: 80245|app: 0|req: 660/1323] 10.222.0.47 () {54 vars in 926 bytes} [Thu Jul 21 08:21:52 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 08:21:52.868525 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3b22891-1747-43ba-8967-e614a40bb011 None None] REQUEST_METHOD: `POST` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:21:52.869308 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3b22891-1747-43ba-8967-e614a40bb011 None None] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:21:52.869637 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3b22891-1747-43ba-8967-e614a40bb011 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:21:52.935201 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.auth.core [None req-d3b22891-1747-43ba-8967-e614a40bb011 None None] MFA Rules not processed for user `2dce40b8d74c41ec94cc36e6ff05d412`. Rule list: `[]` (Enabled: `True`). {{(pid=80244) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 08:21:52.936598 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.fernet_utils [None req-d3b22891-1747-43ba-8967-e614a40bb011 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80244) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 08:21:53.040368 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 664/1324] 10.222.0.47 () {58 vars in 1023 bytes} [Thu Jul 21 08:21:52 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 08:21:53.077814 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d48c27b5-2f28-4435-9c9b-f716208be092 None None] Authenticating user token {{(pid=80245) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:21:53.079529 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.fernet_utils [None req-d48c27b5-2f28-4435-9c9b-f716208be092 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80245) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 08:21:53.099832 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG dbcounter [-] [80245] Writing DB stats keystone:SELECT=100 {{(pid=80245) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:21:53.191274 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d48c27b5-2f28-4435-9c9b-f716208be092 None None] Validating token access rules against request {{(pid=80245) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:21:53.265788 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ff871df-4272-48d2-9c1b-5eec5336c629 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20ca3c3bd3774691b3374cfbfec6e494', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4faef13aa50e4e36b6c265fb1398fa14', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80245) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:21:53.266449 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ff871df-4272-48d2-9c1b-5eec5336c629 service neutron] REQUEST_METHOD: `GET` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:21:53.266727 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ff871df-4272-48d2-9c1b-5eec5336c629 service neutron] SCRIPT_NAME: `/identity` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:21:53.267004 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ff871df-4272-48d2-9c1b-5eec5336c629 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:21:53.268692 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.fernet_utils [None req-5ff871df-4272-48d2-9c1b-5eec5336c629 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80245) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 08:21:53.340868 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG dbcounter [-] [80245] Writing DB stats keystone:SELECT=100 {{(pid=80245) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:21:53.378971 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ff871df-4272-48d2-9c1b-5eec5336c629 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:21:53.380368 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ff871df-4272-48d2-9c1b-5eec5336c629 service neutron] RBAC: Authorization granted {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:21:53.403896 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: [pid: 80245|app: 0|req: 661/1325] 10.222.0.47 () {60 vars in 1365 bytes} [Thu Jul 21 08:21:53 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 326 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 08:21:55.551632 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c26bc589-02c3-409f-be27-fff3ad9d3705 None None] REQUEST_METHOD: `GET` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:21:55.552217 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c26bc589-02c3-409f-be27-fff3ad9d3705 None None] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:21:55.552320 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c26bc589-02c3-409f-be27-fff3ad9d3705 None None] PATH_INFO: `/` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:21:55.553835 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 665/1326] 10.222.0.47 () {54 vars in 926 bytes} [Thu Jul 21 08:21:55 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 08:21:55.560246 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a558c34-dc56-4059-be44-6ca7d704bf27 None None] REQUEST_METHOD: `POST` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:21:55.560776 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a558c34-dc56-4059-be44-6ca7d704bf27 None None] SCRIPT_NAME: `/identity` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:21:55.560876 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a558c34-dc56-4059-be44-6ca7d704bf27 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:21:55.627918 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.auth.core [None req-2a558c34-dc56-4059-be44-6ca7d704bf27 None None] MFA Rules not processed for user `2dce40b8d74c41ec94cc36e6ff05d412`. Rule list: `[]` (Enabled: `True`). {{(pid=80245) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 08:21:55.629243 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.fernet_utils [None req-2a558c34-dc56-4059-be44-6ca7d704bf27 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80245) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 08:21:55.727083 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: [pid: 80245|app: 0|req: 662/1327] 10.222.0.47 () {58 vars in 1023 bytes} [Thu Jul 21 08:21:55 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 08:21:55.764823 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c26bc589-02c3-409f-be27-fff3ad9d3705 None None] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:21:55.766428 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.fernet_utils [None req-c26bc589-02c3-409f-be27-fff3ad9d3705 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80244) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 08:21:55.827073 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG dbcounter [-] [80244] Writing DB stats keystone:SELECT=100 {{(pid=80244) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:21:55.873089 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c26bc589-02c3-409f-be27-fff3ad9d3705 None None] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:21:55.954767 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f940a5cd-84cf-4c88-8d22-90ddcf4742df service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20ca3c3bd3774691b3374cfbfec6e494', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4faef13aa50e4e36b6c265fb1398fa14', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:21:55.955459 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f940a5cd-84cf-4c88-8d22-90ddcf4742df service neutron] REQUEST_METHOD: `GET` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:21:55.955744 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f940a5cd-84cf-4c88-8d22-90ddcf4742df service neutron] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:21:55.956020 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f940a5cd-84cf-4c88-8d22-90ddcf4742df service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:21:55.957655 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.fernet_utils [None req-f940a5cd-84cf-4c88-8d22-90ddcf4742df service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80244) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 08:21:56.065252 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG dbcounter [-] [80244] Writing DB stats keystone:SELECT=100 {{(pid=80244) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:21:56.071762 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f940a5cd-84cf-4c88-8d22-90ddcf4742df service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:21:56.074636 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f940a5cd-84cf-4c88-8d22-90ddcf4742df service neutron] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:21:56.096307 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 666/1328] 10.222.0.47 () {60 vars in 1365 bytes} [Thu Jul 21 08:21:55 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 332 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 08:21:58.253830 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14b0ad11-1d87-4d8a-b339-cf64dbcaba5e None None] REQUEST_METHOD: `GET` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:21:58.254377 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14b0ad11-1d87-4d8a-b339-cf64dbcaba5e None None] SCRIPT_NAME: `/identity` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:21:58.254377 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14b0ad11-1d87-4d8a-b339-cf64dbcaba5e None None] PATH_INFO: `/` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:21:58.255807 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: [pid: 80245|app: 0|req: 663/1329] 10.222.0.47 () {54 vars in 926 bytes} [Thu Jul 21 08:21:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 08:21:58.261538 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fc8ba65-82f3-478a-993b-0d66c862fa2f None None] REQUEST_METHOD: `POST` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:21:58.262044 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fc8ba65-82f3-478a-993b-0d66c862fa2f None None] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:21:58.262139 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fc8ba65-82f3-478a-993b-0d66c862fa2f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:21:58.329733 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.auth.core [None req-8fc8ba65-82f3-478a-993b-0d66c862fa2f None None] MFA Rules not processed for user `39bf6631ce574c758a674faedecf2254`. Rule list: `[]` (Enabled: `True`). {{(pid=80244) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 08:21:58.330945 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.fernet_utils [None req-8fc8ba65-82f3-478a-993b-0d66c862fa2f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80244) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 08:21:58.432597 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 667/1330] 10.222.0.47 () {58 vars in 1023 bytes} [Thu Jul 21 08:21:58 2022] POST /identity/v3/auth/tokens => generated 3233 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 08:21:58.471466 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14b0ad11-1d87-4d8a-b339-cf64dbcaba5e None None] Authenticating user token {{(pid=80245) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:21:58.473031 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.fernet_utils [None req-14b0ad11-1d87-4d8a-b339-cf64dbcaba5e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80245) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 08:21:58.508534 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG dbcounter [-] [80245] Writing DB stats keystone:SELECT=100 {{(pid=80245) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:21:58.586457 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14b0ad11-1d87-4d8a-b339-cf64dbcaba5e None None] Validating token access rules against request {{(pid=80245) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:21:58.667099 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00028ab7-9c43-45d3-bb2f-8d85940b5505 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20ca3c3bd3774691b3374cfbfec6e494', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4faef13aa50e4e36b6c265fb1398fa14', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80245) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:21:58.667709 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00028ab7-9c43-45d3-bb2f-8d85940b5505 service neutron] REQUEST_METHOD: `GET` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:21:58.667989 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00028ab7-9c43-45d3-bb2f-8d85940b5505 service neutron] SCRIPT_NAME: `/identity` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:21:58.668261 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00028ab7-9c43-45d3-bb2f-8d85940b5505 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:21:58.669899 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.fernet_utils [None req-00028ab7-9c43-45d3-bb2f-8d85940b5505 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80245) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 08:21:58.762500 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG dbcounter [-] [80245] Writing DB stats keystone:SELECT=100 {{(pid=80245) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:21:58.800810 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00028ab7-9c43-45d3-bb2f-8d85940b5505 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:21:58.802267 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00028ab7-9c43-45d3-bb2f-8d85940b5505 service neutron] RBAC: Authorization granted {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:21:58.821772 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: [pid: 80245|app: 0|req: 664/1331] 10.222.0.47 () {60 vars in 1365 bytes} [Thu Jul 21 08:21:58 2022] GET /identity/v3/auth/tokens => generated 3233 bytes in 351 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 08:22:01.137372 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61231392-bd14-460f-8312-4e72a3a17537 None None] REQUEST_METHOD: `GET` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:22:01.137936 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61231392-bd14-460f-8312-4e72a3a17537 None None] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:22:01.137936 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61231392-bd14-460f-8312-4e72a3a17537 None None] PATH_INFO: `/` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:22:01.139318 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 668/1332] 10.222.0.47 () {54 vars in 926 bytes} [Thu Jul 21 08:22:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 21 08:22:01.144584 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8859618c-de8f-4a00-a274-6837753df504 None None] REQUEST_METHOD: `POST` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:22:01.145015 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8859618c-de8f-4a00-a274-6837753df504 None None] SCRIPT_NAME: `/identity` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:22:01.145145 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8859618c-de8f-4a00-a274-6837753df504 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:22:01.210829 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.auth.core [None req-8859618c-de8f-4a00-a274-6837753df504 None None] MFA Rules not processed for user `2dce40b8d74c41ec94cc36e6ff05d412`. Rule list: `[]` (Enabled: `True`). {{(pid=80245) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 08:22:01.212108 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.fernet_utils [None req-8859618c-de8f-4a00-a274-6837753df504 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80245) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 08:22:01.314780 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: [pid: 80245|app: 0|req: 665/1333] 10.222.0.47 () {58 vars in 1023 bytes} [Thu Jul 21 08:22:01 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 08:22:01.351795 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61231392-bd14-460f-8312-4e72a3a17537 None None] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:22:01.353354 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.fernet_utils [None req-61231392-bd14-460f-8312-4e72a3a17537 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80244) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 08:22:01.433015 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG dbcounter [-] [80244] Writing DB stats keystone:SELECT=100 {{(pid=80244) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:22:01.470953 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61231392-bd14-460f-8312-4e72a3a17537 None None] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:22:01.550608 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e887dd4-9ce1-484c-ab86-4b6ad16eca25 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20ca3c3bd3774691b3374cfbfec6e494', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4faef13aa50e4e36b6c265fb1398fa14', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:22:01.551269 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e887dd4-9ce1-484c-ab86-4b6ad16eca25 service neutron] REQUEST_METHOD: `GET` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:22:01.551583 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e887dd4-9ce1-484c-ab86-4b6ad16eca25 service neutron] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:22:01.551860 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e887dd4-9ce1-484c-ab86-4b6ad16eca25 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:22:01.553658 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.fernet_utils [None req-5e887dd4-9ce1-484c-ab86-4b6ad16eca25 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80244) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 08:22:01.664822 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e887dd4-9ce1-484c-ab86-4b6ad16eca25 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:22:01.666796 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e887dd4-9ce1-484c-ab86-4b6ad16eca25 service neutron] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:22:01.678200 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG dbcounter [-] [80244] Writing DB stats keystone:SELECT=100 {{(pid=80244) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:22:01.688197 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 669/1334] 10.222.0.47 () {60 vars in 1365 bytes} [Thu Jul 21 08:22:01 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 337 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 08:22:03.799009 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8e032a1-30b7-4f6d-8520-f80336bcff65 None None] REQUEST_METHOD: `GET` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:22:03.799598 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8e032a1-30b7-4f6d-8520-f80336bcff65 None None] SCRIPT_NAME: `/identity` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:22:03.799598 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8e032a1-30b7-4f6d-8520-f80336bcff65 None None] PATH_INFO: `/` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:22:03.801055 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: [pid: 80245|app: 0|req: 666/1335] 10.222.0.47 () {54 vars in 926 bytes} [Thu Jul 21 08:22:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 08:22:03.806149 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e92411cd-7774-4b03-88c6-0a10cd6725df None None] REQUEST_METHOD: `POST` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:22:03.806529 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e92411cd-7774-4b03-88c6-0a10cd6725df None None] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:22:03.806734 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e92411cd-7774-4b03-88c6-0a10cd6725df None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:22:03.869772 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.auth.core [None req-e92411cd-7774-4b03-88c6-0a10cd6725df None None] MFA Rules not processed for user `2dce40b8d74c41ec94cc36e6ff05d412`. Rule list: `[]` (Enabled: `True`). {{(pid=80244) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 08:22:03.870965 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.fernet_utils [None req-e92411cd-7774-4b03-88c6-0a10cd6725df None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80244) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 08:22:03.968833 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 670/1336] 10.222.0.47 () {58 vars in 1023 bytes} [Thu Jul 21 08:22:03 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 08:22:04.010068 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8e032a1-30b7-4f6d-8520-f80336bcff65 None None] Authenticating user token {{(pid=80245) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:22:04.011642 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.fernet_utils [None req-f8e032a1-30b7-4f6d-8520-f80336bcff65 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80245) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 08:22:04.051578 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG dbcounter [-] [80245] Writing DB stats keystone:SELECT=100 {{(pid=80245) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:22:04.114522 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8e032a1-30b7-4f6d-8520-f80336bcff65 None None] Validating token access rules against request {{(pid=80245) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:22:04.187052 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1a82dc8-0286-4ec9-8a95-84ee583fa955 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20ca3c3bd3774691b3374cfbfec6e494', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4faef13aa50e4e36b6c265fb1398fa14', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80245) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:22:04.187732 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1a82dc8-0286-4ec9-8a95-84ee583fa955 service neutron] REQUEST_METHOD: `GET` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:22:04.188015 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1a82dc8-0286-4ec9-8a95-84ee583fa955 service neutron] SCRIPT_NAME: `/identity` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:22:04.188290 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1a82dc8-0286-4ec9-8a95-84ee583fa955 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:22:04.189913 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.fernet_utils [None req-e1a82dc8-0286-4ec9-8a95-84ee583fa955 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80245) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 08:22:04.272397 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG dbcounter [-] [80245] Writing DB stats keystone:SELECT=100 {{(pid=80245) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:22:04.298634 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1a82dc8-0286-4ec9-8a95-84ee583fa955 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:22:04.300124 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1a82dc8-0286-4ec9-8a95-84ee583fa955 service neutron] RBAC: Authorization granted {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:22:04.319510 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: [pid: 80245|app: 0|req: 667/1337] 10.222.0.47 () {60 vars in 1365 bytes} [Thu Jul 21 08:22:04 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 310 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 08:22:06.848654 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34c26f0d-ec94-4403-a32e-cbfd371371e6 None None] REQUEST_METHOD: `GET` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:22:06.849171 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34c26f0d-ec94-4403-a32e-cbfd371371e6 None None] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:22:06.849171 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34c26f0d-ec94-4403-a32e-cbfd371371e6 None None] PATH_INFO: `/` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:22:06.850502 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 671/1338] 10.222.0.47 () {54 vars in 926 bytes} [Thu Jul 21 08:22:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 08:22:06.855792 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68c52b08-f53c-44e9-901c-14384ef8bda3 None None] REQUEST_METHOD: `POST` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:22:06.856231 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68c52b08-f53c-44e9-901c-14384ef8bda3 None None] SCRIPT_NAME: `/identity` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:22:06.856334 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68c52b08-f53c-44e9-901c-14384ef8bda3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:22:06.920359 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.auth.core [None req-68c52b08-f53c-44e9-901c-14384ef8bda3 None None] MFA Rules not processed for user `2dce40b8d74c41ec94cc36e6ff05d412`. Rule list: `[]` (Enabled: `True`). {{(pid=80245) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 08:22:06.921577 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.fernet_utils [None req-68c52b08-f53c-44e9-901c-14384ef8bda3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80245) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 08:22:07.016016 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: [pid: 80245|app: 0|req: 668/1339] 10.222.0.47 () {58 vars in 1023 bytes} [Thu Jul 21 08:22:06 2022] POST /identity/v3/auth/tokens => generated 3228 bytes in 161 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 08:22:07.052460 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34c26f0d-ec94-4403-a32e-cbfd371371e6 None None] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:22:07.053977 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.fernet_utils [None req-34c26f0d-ec94-4403-a32e-cbfd371371e6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80244) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 08:22:07.130598 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG dbcounter [-] [80244] Writing DB stats keystone:SELECT=100 {{(pid=80244) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:22:07.158149 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34c26f0d-ec94-4403-a32e-cbfd371371e6 None None] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:22:07.237390 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0a2cee2-a5df-4bd9-83d1-5eca69a1acd1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20ca3c3bd3774691b3374cfbfec6e494', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4faef13aa50e4e36b6c265fb1398fa14', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:22:07.238046 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0a2cee2-a5df-4bd9-83d1-5eca69a1acd1 service neutron] REQUEST_METHOD: `GET` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:22:07.238332 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0a2cee2-a5df-4bd9-83d1-5eca69a1acd1 service neutron] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:22:07.238610 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0a2cee2-a5df-4bd9-83d1-5eca69a1acd1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:22:07.240304 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.fernet_utils [None req-b0a2cee2-a5df-4bd9-83d1-5eca69a1acd1 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80244) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 08:22:07.353084 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0a2cee2-a5df-4bd9-83d1-5eca69a1acd1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:22:07.355018 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0a2cee2-a5df-4bd9-83d1-5eca69a1acd1 service neutron] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:22:07.373585 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 672/1340] 10.222.0.47 () {60 vars in 1365 bytes} [Thu Jul 21 08:22:07 2022] GET /identity/v3/auth/tokens => generated 3228 bytes in 322 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 08:22:11.439010 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e542cb5-21d5-4e6c-a629-811de7352180 None None] REQUEST_METHOD: `GET` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:22:11.439564 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e542cb5-21d5-4e6c-a629-811de7352180 None None] SCRIPT_NAME: `/identity` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:22:11.439564 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e542cb5-21d5-4e6c-a629-811de7352180 None None] PATH_INFO: `/` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:22:11.440949 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: [pid: 80245|app: 0|req: 669/1341] 10.222.0.47 () {54 vars in 926 bytes} [Thu Jul 21 08:22:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 08:22:11.446495 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d81e521-909a-47a6-bed2-8e43488e6f69 None None] REQUEST_METHOD: `POST` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:22:11.446865 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d81e521-909a-47a6-bed2-8e43488e6f69 None None] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:22:11.447699 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d81e521-909a-47a6-bed2-8e43488e6f69 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:22:11.452140 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG dbcounter [-] [80244] Writing DB stats keystone:SELECT=100 {{(pid=80244) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:22:11.521535 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.auth.core [None req-0d81e521-909a-47a6-bed2-8e43488e6f69 None None] MFA Rules not processed for user `2dce40b8d74c41ec94cc36e6ff05d412`. Rule list: `[]` (Enabled: `True`). {{(pid=80244) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 08:22:11.522739 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.fernet_utils [None req-0d81e521-909a-47a6-bed2-8e43488e6f69 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80244) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 08:22:11.620577 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 673/1342] 10.222.0.47 () {58 vars in 1023 bytes} [Thu Jul 21 08:22:11 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 08:22:11.658645 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e542cb5-21d5-4e6c-a629-811de7352180 None None] Authenticating user token {{(pid=80245) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:22:11.660241 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.fernet_utils [None req-3e542cb5-21d5-4e6c-a629-811de7352180 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80245) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 08:22:11.717517 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG dbcounter [-] [80245] Writing DB stats keystone:SELECT=100 {{(pid=80245) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:22:11.777962 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e542cb5-21d5-4e6c-a629-811de7352180 None None] Validating token access rules against request {{(pid=80245) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:22:11.850647 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e67fa633-2015-4c5d-af58-7b8b899549a6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20ca3c3bd3774691b3374cfbfec6e494', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4faef13aa50e4e36b6c265fb1398fa14', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80245) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:22:11.851306 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e67fa633-2015-4c5d-af58-7b8b899549a6 service neutron] REQUEST_METHOD: `GET` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:22:11.851616 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e67fa633-2015-4c5d-af58-7b8b899549a6 service neutron] SCRIPT_NAME: `/identity` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:22:11.851893 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e67fa633-2015-4c5d-af58-7b8b899549a6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:22:11.854034 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.fernet_utils [None req-e67fa633-2015-4c5d-af58-7b8b899549a6 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80245) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 08:22:11.951871 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG dbcounter [-] [80245] Writing DB stats keystone:SELECT=100 {{(pid=80245) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:22:11.969559 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e67fa633-2015-4c5d-af58-7b8b899549a6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:22:11.971017 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e67fa633-2015-4c5d-af58-7b8b899549a6 service neutron] RBAC: Authorization granted {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:22:11.990754 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: [pid: 80245|app: 0|req: 670/1343] 10.222.0.47 () {60 vars in 1365 bytes} [Thu Jul 21 08:22:11 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 333 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 08:22:14.751246 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fef8e1cb-dd74-45b4-b6fb-7fbce861b62c None None] REQUEST_METHOD: `GET` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:22:14.751814 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fef8e1cb-dd74-45b4-b6fb-7fbce861b62c None None] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:22:14.751814 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fef8e1cb-dd74-45b4-b6fb-7fbce861b62c None None] PATH_INFO: `/` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:22:14.753142 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 674/1344] 10.222.0.47 () {54 vars in 926 bytes} [Thu Jul 21 08:22:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 08:22:14.758819 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f901f56-996e-4cd4-82b4-a938ef26498b None None] REQUEST_METHOD: `POST` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:22:14.759206 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f901f56-996e-4cd4-82b4-a938ef26498b None None] SCRIPT_NAME: `/identity` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:22:14.759450 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f901f56-996e-4cd4-82b4-a938ef26498b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:22:14.828232 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.auth.core [None req-6f901f56-996e-4cd4-82b4-a938ef26498b None None] MFA Rules not processed for user `2dce40b8d74c41ec94cc36e6ff05d412`. Rule list: `[]` (Enabled: `True`). {{(pid=80245) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 08:22:14.829448 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.fernet_utils [None req-6f901f56-996e-4cd4-82b4-a938ef26498b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80245) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 08:22:14.929871 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: [pid: 80245|app: 0|req: 671/1345] 10.222.0.47 () {58 vars in 1023 bytes} [Thu Jul 21 08:22:14 2022] POST /identity/v3/auth/tokens => generated 3228 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 21 08:22:14.968346 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fef8e1cb-dd74-45b4-b6fb-7fbce861b62c None None] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:22:14.969980 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.fernet_utils [None req-fef8e1cb-dd74-45b4-b6fb-7fbce861b62c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80244) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 08:22:15.062128 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG dbcounter [-] [80244] Writing DB stats keystone:SELECT=100 {{(pid=80244) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:22:15.079934 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fef8e1cb-dd74-45b4-b6fb-7fbce861b62c None None] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:22:15.152736 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd8e87b6-2d6f-42b4-ba89-97020032a50f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20ca3c3bd3774691b3374cfbfec6e494', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4faef13aa50e4e36b6c265fb1398fa14', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:22:15.153383 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd8e87b6-2d6f-42b4-ba89-97020032a50f service neutron] REQUEST_METHOD: `GET` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:22:15.153665 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd8e87b6-2d6f-42b4-ba89-97020032a50f service neutron] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:22:15.153937 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd8e87b6-2d6f-42b4-ba89-97020032a50f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:22:15.155602 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.fernet_utils [None req-bd8e87b6-2d6f-42b4-ba89-97020032a50f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80244) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 08:22:15.276327 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd8e87b6-2d6f-42b4-ba89-97020032a50f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:22:15.277776 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd8e87b6-2d6f-42b4-ba89-97020032a50f service neutron] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:22:15.300875 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 675/1346] 10.222.0.47 () {60 vars in 1365 bytes} [Thu Jul 21 08:22:14 2022] GET /identity/v3/auth/tokens => generated 3228 bytes in 333 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 08:22:19.530776 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deacae6f-62ae-4501-95f9-8d3912ad4c5d None None] REQUEST_METHOD: `GET` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:22:19.531241 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deacae6f-62ae-4501-95f9-8d3912ad4c5d None None] SCRIPT_NAME: `/identity` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:22:19.531388 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deacae6f-62ae-4501-95f9-8d3912ad4c5d None None] PATH_INFO: `/` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:22:19.532841 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: [pid: 80245|app: 0|req: 672/1347] 10.222.0.47 () {54 vars in 926 bytes} [Thu Jul 21 08:22:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 08:22:19.538641 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b8ae406-e5bf-4832-b65e-85ebb2ead731 None None] REQUEST_METHOD: `POST` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:22:19.539139 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b8ae406-e5bf-4832-b65e-85ebb2ead731 None None] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:22:19.539319 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b8ae406-e5bf-4832-b65e-85ebb2ead731 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:22:19.554074 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG dbcounter [-] [80244] Writing DB stats keystone:SELECT=100 {{(pid=80244) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:22:19.614717 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.auth.core [None req-3b8ae406-e5bf-4832-b65e-85ebb2ead731 None None] MFA Rules not processed for user `2dce40b8d74c41ec94cc36e6ff05d412`. Rule list: `[]` (Enabled: `True`). {{(pid=80244) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 08:22:19.616005 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.fernet_utils [None req-3b8ae406-e5bf-4832-b65e-85ebb2ead731 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80244) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 08:22:19.725845 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 676/1348] 10.222.0.47 () {58 vars in 1023 bytes} [Thu Jul 21 08:22:19 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 188 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 08:22:19.765369 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deacae6f-62ae-4501-95f9-8d3912ad4c5d None None] Authenticating user token {{(pid=80245) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:22:19.766936 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.fernet_utils [None req-deacae6f-62ae-4501-95f9-8d3912ad4c5d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80245) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 08:22:19.835293 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG dbcounter [-] [80245] Writing DB stats keystone:SELECT=100 {{(pid=80245) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:22:19.890075 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deacae6f-62ae-4501-95f9-8d3912ad4c5d None None] Validating token access rules against request {{(pid=80245) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:22:19.961564 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1522f382-97af-462e-b9e8-c8db0998c4cb service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20ca3c3bd3774691b3374cfbfec6e494', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4faef13aa50e4e36b6c265fb1398fa14', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80245) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:22:19.962220 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1522f382-97af-462e-b9e8-c8db0998c4cb service neutron] REQUEST_METHOD: `GET` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:22:19.962524 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1522f382-97af-462e-b9e8-c8db0998c4cb service neutron] SCRIPT_NAME: `/identity` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:22:19.962797 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1522f382-97af-462e-b9e8-c8db0998c4cb service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:22:19.964465 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.fernet_utils [None req-1522f382-97af-462e-b9e8-c8db0998c4cb service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80245) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 08:22:20.076460 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG dbcounter [-] [80245] Writing DB stats keystone:SELECT=100 {{(pid=80245) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:22:20.081067 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1522f382-97af-462e-b9e8-c8db0998c4cb service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:22:20.082567 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1522f382-97af-462e-b9e8-c8db0998c4cb service neutron] RBAC: Authorization granted {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:22:20.101889 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: [pid: 80245|app: 0|req: 673/1349] 10.222.0.47 () {60 vars in 1365 bytes} [Thu Jul 21 08:22:19 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 337 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 08:22:25.129636 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e9087ae-629f-4feb-bcf1-12e9064305f4 None None] REQUEST_METHOD: `GET` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:22:25.130154 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e9087ae-629f-4feb-bcf1-12e9064305f4 None None] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:22:25.130154 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e9087ae-629f-4feb-bcf1-12e9064305f4 None None] PATH_INFO: `/` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:22:25.132230 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 677/1350] 10.222.0.47 () {54 vars in 921 bytes} [Thu Jul 21 08:22:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 08:22:25.140954 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a174ca6-86f0-43de-8a2e-01cd2d854ecb None None] REQUEST_METHOD: `POST` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:22:25.141407 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a174ca6-86f0-43de-8a2e-01cd2d854ecb None None] SCRIPT_NAME: `/identity` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:22:25.141512 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a174ca6-86f0-43de-8a2e-01cd2d854ecb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:22:25.244952 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.auth.core [None req-2a174ca6-86f0-43de-8a2e-01cd2d854ecb None None] MFA Rules not processed for user `3379b32543474d0a8e897484e85c7d04`. Rule list: `[]` (Enabled: `True`). {{(pid=80245) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 08:22:25.246069 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.fernet_utils [None req-2a174ca6-86f0-43de-8a2e-01cd2d854ecb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80245) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 08:22:25.369500 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: [pid: 80245|app: 0|req: 674/1351] 10.222.0.47 () {58 vars in 1018 bytes} [Thu Jul 21 08:22:25 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 229 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 08:22:29.541480 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-115d1056-6123-4f42-aa1c-05fdfdabee85 None None] REQUEST_METHOD: `GET` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:22:29.541982 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-115d1056-6123-4f42-aa1c-05fdfdabee85 None None] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:22:29.542097 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-115d1056-6123-4f42-aa1c-05fdfdabee85 None None] PATH_INFO: `/` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:22:29.543607 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 678/1352] 10.222.0.47 () {54 vars in 921 bytes} [Thu Jul 21 08:22:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 21 08:22:29.548687 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-322978f2-20c3-4bad-ba38-a8859d7e6c2b None None] REQUEST_METHOD: `POST` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:22:29.549114 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-322978f2-20c3-4bad-ba38-a8859d7e6c2b None None] SCRIPT_NAME: `/identity` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:22:29.549282 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-322978f2-20c3-4bad-ba38-a8859d7e6c2b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:22:29.607963 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG dbcounter [-] [80245] Writing DB stats keystone:SELECT=100 {{(pid=80245) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:22:29.640879 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.auth.core [None req-322978f2-20c3-4bad-ba38-a8859d7e6c2b None None] MFA Rules not processed for user `3379b32543474d0a8e897484e85c7d04`. Rule list: `[]` (Enabled: `True`). {{(pid=80245) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 08:22:29.642073 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.fernet_utils [None req-322978f2-20c3-4bad-ba38-a8859d7e6c2b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80245) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 08:22:29.723725 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG dbcounter [-] [80244] Writing DB stats keystone:SELECT=59 {{(pid=80244) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:22:29.753072 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: [pid: 80245|app: 0|req: 675/1353] 10.222.0.47 () {58 vars in 1018 bytes} [Thu Jul 21 08:22:29 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 205 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 08:22:31.155704 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54b67d80-48dc-458f-9bf5-6d83f29c9265 None None] REQUEST_METHOD: `GET` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:22:31.156246 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54b67d80-48dc-458f-9bf5-6d83f29c9265 None None] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:22:31.156246 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54b67d80-48dc-458f-9bf5-6d83f29c9265 None None] PATH_INFO: `/` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:22:31.157870 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 679/1354] 10.222.0.47 () {54 vars in 921 bytes} [Thu Jul 21 08:22:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 08:22:31.162773 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa576659-fb8e-4b97-9e0e-3805f5533f60 None None] REQUEST_METHOD: `POST` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:22:31.163207 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa576659-fb8e-4b97-9e0e-3805f5533f60 None None] SCRIPT_NAME: `/identity` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:22:31.163481 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa576659-fb8e-4b97-9e0e-3805f5533f60 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:22:31.240595 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.auth.core [None req-aa576659-fb8e-4b97-9e0e-3805f5533f60 None None] MFA Rules not processed for user `3379b32543474d0a8e897484e85c7d04`. Rule list: `[]` (Enabled: `True`). {{(pid=80245) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 08:22:31.241800 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.fernet_utils [None req-aa576659-fb8e-4b97-9e0e-3805f5533f60 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80245) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 08:22:31.285515 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG dbcounter [-] [80245] Writing DB stats keystone:SELECT=100 {{(pid=80245) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:22:31.358243 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: [pid: 80245|app: 0|req: 676/1355] 10.222.0.47 () {58 vars in 1018 bytes} [Thu Jul 21 08:22:31 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 196 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 08:22:34.841551 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3db8b87a-9d1a-40d3-89cd-e077a876d96b None None] REQUEST_METHOD: `GET` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:22:34.842534 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3db8b87a-9d1a-40d3-89cd-e077a876d96b None None] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:22:34.843047 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3db8b87a-9d1a-40d3-89cd-e077a876d96b None None] PATH_INFO: `/` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:22:34.849231 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 680/1356] 10.222.0.47 () {54 vars in 926 bytes} [Thu Jul 21 08:22:34 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 08:22:34.851185 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c954c712-45ae-4451-9f77-ce257f28ae53 None None] REQUEST_METHOD: `POST` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:22:34.851185 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c954c712-45ae-4451-9f77-ce257f28ae53 None None] SCRIPT_NAME: `/identity` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:22:34.855899 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c954c712-45ae-4451-9f77-ce257f28ae53 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:22:34.940122 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.auth.core [None req-c954c712-45ae-4451-9f77-ce257f28ae53 None None] MFA Rules not processed for user `2dce40b8d74c41ec94cc36e6ff05d412`. Rule list: `[]` (Enabled: `True`). {{(pid=80245) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 08:22:34.941733 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.fernet_utils [None req-c954c712-45ae-4451-9f77-ce257f28ae53 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80245) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 08:22:35.067490 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: [pid: 80245|app: 0|req: 677/1357] 10.222.0.47 () {58 vars in 1023 bytes} [Thu Jul 21 08:22:34 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 213 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 21 08:22:35.117210 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47b93b9c-08ba-400d-b218-d351249fbbf9 None None] REQUEST_METHOD: `GET` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:22:35.117505 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47b93b9c-08ba-400d-b218-d351249fbbf9 None None] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:22:35.118541 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47b93b9c-08ba-400d-b218-d351249fbbf9 None None] PATH_INFO: `/` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:22:35.119740 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 681/1358] 10.222.0.47 () {54 vars in 962 bytes} [Thu Jul 21 08:22:35 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 21 08:22:35.128204 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4774c04-d21d-430b-80f1-274b23c29858 None None] REQUEST_METHOD: `POST` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:22:35.128793 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4774c04-d21d-430b-80f1-274b23c29858 None None] SCRIPT_NAME: `/identity` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:22:35.129249 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4774c04-d21d-430b-80f1-274b23c29858 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:22:35.148435 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG dbcounter [-] [80245] Writing DB stats keystone:SELECT=100 {{(pid=80245) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:22:35.226880 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.auth.core [None req-b4774c04-d21d-430b-80f1-274b23c29858 None None] MFA Rules not processed for user `ac2b5b99a36442148fe743269054e991`. Rule list: `[]` (Enabled: `True`). {{(pid=80245) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 08:22:35.228869 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.fernet_utils [None req-b4774c04-d21d-430b-80f1-274b23c29858 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80245) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 08:22:35.375810 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: [pid: 80245|app: 0|req: 678/1359] 10.222.0.47 () {58 vars in 1059 bytes} [Thu Jul 21 08:22:35 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 249 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 21 08:22:35.384347 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47b93b9c-08ba-400d-b218-d351249fbbf9 None None] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:22:35.386091 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.fernet_utils [None req-47b93b9c-08ba-400d-b218-d351249fbbf9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80244) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 08:22:35.537969 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47b93b9c-08ba-400d-b218-d351249fbbf9 None None] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:22:35.628691 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG dbcounter [-] [80244] Writing DB stats keystone:SELECT=100 {{(pid=80244) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:22:35.673392 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-426c2b35-2808-4899-b5f4-3da574f871f4 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2b5b99a36442148fe743269054e991', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4faef13aa50e4e36b6c265fb1398fa14', 'project_domain_id': 'default', 'roles': ['member', 'service', 'ResellerAdmin', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:22:35.674153 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-426c2b35-2808-4899-b5f4-3da574f871f4 service nova] REQUEST_METHOD: `GET` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:22:35.674529 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-426c2b35-2808-4899-b5f4-3da574f871f4 service nova] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:22:35.674927 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-426c2b35-2808-4899-b5f4-3da574f871f4 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:22:35.676712 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.fernet_utils [None req-426c2b35-2808-4899-b5f4-3da574f871f4 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80244) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 08:22:35.773441 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-426c2b35-2808-4899-b5f4-3da574f871f4 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:22:35.775615 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-426c2b35-2808-4899-b5f4-3da574f871f4 service nova] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:22:35.794691 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 682/1360] 10.222.0.47 () {60 vars in 1365 bytes} [Thu Jul 21 08:22:35 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 411 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 08:22:38.099554 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eda98ff-9bc2-4c17-af7a-61cc3ee6dfc8 None None] REQUEST_METHOD: `GET` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:22:38.100111 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eda98ff-9bc2-4c17-af7a-61cc3ee6dfc8 None None] SCRIPT_NAME: `/identity` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:22:38.100111 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eda98ff-9bc2-4c17-af7a-61cc3ee6dfc8 None None] PATH_INFO: `/` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:22:38.102308 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: [pid: 80245|app: 0|req: 679/1361] 10.222.0.47 () {54 vars in 926 bytes} [Thu Jul 21 08:22:38 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 08:22:38.107712 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb975996-2585-4535-b426-b1178b404f14 None None] REQUEST_METHOD: `POST` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:22:38.108177 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb975996-2585-4535-b426-b1178b404f14 None None] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:22:38.108259 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb975996-2585-4535-b426-b1178b404f14 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:22:38.178904 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.auth.core [None req-cb975996-2585-4535-b426-b1178b404f14 None None] MFA Rules not processed for user `2dce40b8d74c41ec94cc36e6ff05d412`. Rule list: `[]` (Enabled: `True`). {{(pid=80244) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 08:22:38.181171 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.fernet_utils [None req-cb975996-2585-4535-b426-b1178b404f14 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80244) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 08:22:38.222707 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG dbcounter [-] [80244] Writing DB stats keystone:SELECT=100 {{(pid=80244) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:22:38.303476 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 683/1362] 10.222.0.47 () {58 vars in 1023 bytes} [Thu Jul 21 08:22:38 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 197 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 08:22:38.347839 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eda98ff-9bc2-4c17-af7a-61cc3ee6dfc8 None None] Authenticating user token {{(pid=80245) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:22:38.349619 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.fernet_utils [None req-4eda98ff-9bc2-4c17-af7a-61cc3ee6dfc8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80245) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 08:22:38.438702 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG dbcounter [-] [80245] Writing DB stats keystone:SELECT=100 {{(pid=80245) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:22:38.549485 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eda98ff-9bc2-4c17-af7a-61cc3ee6dfc8 None None] Validating token access rules against request {{(pid=80245) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:22:38.681493 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f780ebb-ee6d-4f72-9053-8ebe4adf5b29 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2b5b99a36442148fe743269054e991', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4faef13aa50e4e36b6c265fb1398fa14', 'project_domain_id': 'default', 'roles': ['member', 'service', 'ResellerAdmin', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80245) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:22:38.682156 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f780ebb-ee6d-4f72-9053-8ebe4adf5b29 service nova] REQUEST_METHOD: `GET` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:22:38.682440 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f780ebb-ee6d-4f72-9053-8ebe4adf5b29 service nova] SCRIPT_NAME: `/identity` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:22:38.682715 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f780ebb-ee6d-4f72-9053-8ebe4adf5b29 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:22:38.684417 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.fernet_utils [None req-6f780ebb-ee6d-4f72-9053-8ebe4adf5b29 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80245) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 08:22:38.708762 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG dbcounter [-] [80245] Writing DB stats keystone:SELECT=100 {{(pid=80245) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:22:38.785155 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f780ebb-ee6d-4f72-9053-8ebe4adf5b29 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:22:38.786761 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f780ebb-ee6d-4f72-9053-8ebe4adf5b29 service nova] RBAC: Authorization granted {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:22:38.808942 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: [pid: 80245|app: 0|req: 680/1363] 10.222.0.47 () {60 vars in 1365 bytes} [Thu Jul 21 08:22:38 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 462 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 08:22:41.206208 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43739fff-52af-413f-84c0-7a23f7ee82ec None None] REQUEST_METHOD: `GET` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:22:41.206823 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43739fff-52af-413f-84c0-7a23f7ee82ec None None] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:22:41.206823 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43739fff-52af-413f-84c0-7a23f7ee82ec None None] PATH_INFO: `/` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:22:41.208492 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 684/1364] 10.222.0.47 () {54 vars in 926 bytes} [Thu Jul 21 08:22:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 08:22:41.218358 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8410c1e1-fef9-4e93-82ec-134e4f3cf379 None None] REQUEST_METHOD: `POST` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:22:41.218358 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8410c1e1-fef9-4e93-82ec-134e4f3cf379 None None] SCRIPT_NAME: `/identity` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:22:41.218358 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8410c1e1-fef9-4e93-82ec-134e4f3cf379 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:22:41.294131 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.auth.core [None req-8410c1e1-fef9-4e93-82ec-134e4f3cf379 None None] MFA Rules not processed for user `2dce40b8d74c41ec94cc36e6ff05d412`. Rule list: `[]` (Enabled: `True`). {{(pid=80245) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 08:22:41.295360 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.fernet_utils [None req-8410c1e1-fef9-4e93-82ec-134e4f3cf379 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80245) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 08:22:41.402512 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG dbcounter [-] [80245] Writing DB stats keystone:SELECT=100 {{(pid=80245) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:22:41.405598 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: [pid: 80245|app: 0|req: 681/1365] 10.222.0.47 () {58 vars in 1023 bytes} [Thu Jul 21 08:22:41 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 192 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 08:22:41.453147 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e14ab73-6893-4c51-95d1-adf4fa3c8e41 None None] REQUEST_METHOD: `GET` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:22:41.453442 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e14ab73-6893-4c51-95d1-adf4fa3c8e41 None None] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:22:41.453849 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e14ab73-6893-4c51-95d1-adf4fa3c8e41 None None] PATH_INFO: `/` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:22:41.455654 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 685/1366] 10.222.0.47 () {54 vars in 962 bytes} [Thu Jul 21 08:22:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 08:22:41.461964 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62aaf9fb-0117-4906-b16d-52f027a8df65 None None] REQUEST_METHOD: `POST` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:22:41.462253 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62aaf9fb-0117-4906-b16d-52f027a8df65 None None] SCRIPT_NAME: `/identity` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:22:41.462531 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62aaf9fb-0117-4906-b16d-52f027a8df65 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:22:41.546372 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.auth.core [None req-62aaf9fb-0117-4906-b16d-52f027a8df65 None None] MFA Rules not processed for user `ac2b5b99a36442148fe743269054e991`. Rule list: `[]` (Enabled: `True`). {{(pid=80245) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 08:22:41.548390 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.fernet_utils [None req-62aaf9fb-0117-4906-b16d-52f027a8df65 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80245) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 08:22:41.698133 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: [pid: 80245|app: 0|req: 682/1367] 10.222.0.47 () {58 vars in 1059 bytes} [Thu Jul 21 08:22:41 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 237 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 08:22:41.706551 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e14ab73-6893-4c51-95d1-adf4fa3c8e41 None None] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:22:41.708038 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.fernet_utils [None req-1e14ab73-6893-4c51-95d1-adf4fa3c8e41 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80244) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 08:22:41.871308 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e14ab73-6893-4c51-95d1-adf4fa3c8e41 None None] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:22:41.900688 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG dbcounter [-] [80244] Writing DB stats keystone:SELECT=100 {{(pid=80244) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:22:42.003491 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-060333ae-7cc1-4472-837e-5a71e664906a service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2b5b99a36442148fe743269054e991', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4faef13aa50e4e36b6c265fb1398fa14', 'project_domain_id': 'default', 'roles': ['member', 'service', 'ResellerAdmin', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:22:42.004066 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-060333ae-7cc1-4472-837e-5a71e664906a service nova] REQUEST_METHOD: `GET` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:22:42.004353 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-060333ae-7cc1-4472-837e-5a71e664906a service nova] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:22:42.004636 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-060333ae-7cc1-4472-837e-5a71e664906a service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:22:42.006293 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.fernet_utils [None req-060333ae-7cc1-4472-837e-5a71e664906a service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80244) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 08:22:42.102631 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-060333ae-7cc1-4472-837e-5a71e664906a service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:22:42.104278 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-060333ae-7cc1-4472-837e-5a71e664906a service nova] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:22:42.123365 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 686/1368] 10.222.0.47 () {60 vars in 1365 bytes} [Thu Jul 21 08:22:41 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 417 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 08:22:43.168810 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82be87b9-d0e6-4355-98a1-8f41f6e1bd4f None None] REQUEST_METHOD: `GET` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:22:43.169377 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82be87b9-d0e6-4355-98a1-8f41f6e1bd4f None None] SCRIPT_NAME: `/identity` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:22:43.169377 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82be87b9-d0e6-4355-98a1-8f41f6e1bd4f None None] PATH_INFO: `/` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:22:43.170955 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: [pid: 80245|app: 0|req: 683/1369] 10.222.0.47 () {54 vars in 926 bytes} [Thu Jul 21 08:22:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 08:22:43.176743 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9356ac7-a612-4380-9e87-48644acea0f0 None None] REQUEST_METHOD: `POST` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:22:43.177663 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9356ac7-a612-4380-9e87-48644acea0f0 None None] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:22:43.177997 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9356ac7-a612-4380-9e87-48644acea0f0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:22:43.210941 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG dbcounter [-] [80244] Writing DB stats keystone:SELECT=100 {{(pid=80244) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:22:43.251385 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.auth.core [None req-e9356ac7-a612-4380-9e87-48644acea0f0 None None] MFA Rules not processed for user `2dce40b8d74c41ec94cc36e6ff05d412`. Rule list: `[]` (Enabled: `True`). {{(pid=80244) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 08:22:43.253465 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.fernet_utils [None req-e9356ac7-a612-4380-9e87-48644acea0f0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80244) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 08:22:43.390123 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 687/1370] 10.222.0.47 () {58 vars in 1023 bytes} [Thu Jul 21 08:22:43 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 214 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 08:22:43.723832 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7caf255a-e486-4e37-8380-aa11d47c1753 None None] REQUEST_METHOD: `GET` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:22:43.724155 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7caf255a-e486-4e37-8380-aa11d47c1753 None None] SCRIPT_NAME: `/identity` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:22:43.724407 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7caf255a-e486-4e37-8380-aa11d47c1753 None None] PATH_INFO: `/` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:22:43.726574 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: [pid: 80245|app: 0|req: 684/1371] 10.222.0.47 () {54 vars in 964 bytes} [Thu Jul 21 08:22:43 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 08:22:43.732559 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-940b3962-b53d-4b41-a25b-0822fa2671a9 None None] REQUEST_METHOD: `POST` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:22:43.733716 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-940b3962-b53d-4b41-a25b-0822fa2671a9 None None] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:22:43.733716 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-940b3962-b53d-4b41-a25b-0822fa2671a9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:22:43.816058 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.auth.core [None req-940b3962-b53d-4b41-a25b-0822fa2671a9 None None] MFA Rules not processed for user `b4f40410b7674687ab5362ac982248ef`. Rule list: `[]` (Enabled: `True`). {{(pid=80244) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 08:22:43.817328 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.fernet_utils [None req-940b3962-b53d-4b41-a25b-0822fa2671a9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80244) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 08:22:43.868201 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG dbcounter [-] [80244] Writing DB stats keystone:SELECT=100 {{(pid=80244) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:22:43.904404 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 688/1372] 10.222.0.47 () {58 vars in 1061 bytes} [Thu Jul 21 08:22:43 2022] POST /identity/v3/auth/tokens => generated 3110 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 08:22:43.912681 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7caf255a-e486-4e37-8380-aa11d47c1753 None None] Authenticating user token {{(pid=80245) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:22:43.914444 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.fernet_utils [None req-7caf255a-e486-4e37-8380-aa11d47c1753 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80245) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 08:22:43.968789 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG dbcounter [-] [80245] Writing DB stats keystone:SELECT=100 {{(pid=80245) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:22:44.034991 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7caf255a-e486-4e37-8380-aa11d47c1753 None None] Validating token access rules against request {{(pid=80245) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:22:44.116386 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9977bc41-c925-4c65-ae07-046bd7190e42 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4f40410b7674687ab5362ac982248ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4faef13aa50e4e36b6c265fb1398fa14', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80245) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:22:44.116972 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9977bc41-c925-4c65-ae07-046bd7190e42 service cinder] REQUEST_METHOD: `GET` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:22:44.117290 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9977bc41-c925-4c65-ae07-046bd7190e42 service cinder] SCRIPT_NAME: `/identity` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:22:44.117572 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9977bc41-c925-4c65-ae07-046bd7190e42 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:22:44.119660 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.fernet_utils [None req-9977bc41-c925-4c65-ae07-046bd7190e42 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80245) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 08:22:44.223649 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG dbcounter [-] [80245] Writing DB stats keystone:SELECT=100 {{(pid=80245) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:22:44.237624 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9977bc41-c925-4c65-ae07-046bd7190e42 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:22:44.239074 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9977bc41-c925-4c65-ae07-046bd7190e42 service cinder] RBAC: Authorization granted {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:22:44.263538 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: [pid: 80245|app: 0|req: 685/1373] 10.222.0.47 () {60 vars in 1365 bytes} [Thu Jul 21 08:22:43 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 351 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 08:22:44.305243 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70df1255-463a-4c1b-a2d7-9ac34ed2ef3e None None] REQUEST_METHOD: `GET` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:22:44.305698 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70df1255-463a-4c1b-a2d7-9ac34ed2ef3e None None] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:22:44.305926 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70df1255-463a-4c1b-a2d7-9ac34ed2ef3e None None] PATH_INFO: `/` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:22:44.311425 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 689/1374] 10.222.0.47 () {54 vars in 926 bytes} [Thu Jul 21 08:22:44 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 08:22:44.312610 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04acd2cb-dd20-4a18-bc67-90dbf01d5f5e None None] REQUEST_METHOD: `POST` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:22:44.312921 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04acd2cb-dd20-4a18-bc67-90dbf01d5f5e None None] SCRIPT_NAME: `/identity` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:22:44.313193 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04acd2cb-dd20-4a18-bc67-90dbf01d5f5e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:22:44.393269 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.auth.core [None req-04acd2cb-dd20-4a18-bc67-90dbf01d5f5e None None] MFA Rules not processed for user `2dce40b8d74c41ec94cc36e6ff05d412`. Rule list: `[]` (Enabled: `True`). {{(pid=80245) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 08:22:44.395091 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.fernet_utils [None req-04acd2cb-dd20-4a18-bc67-90dbf01d5f5e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80245) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 08:22:44.508785 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: [pid: 80245|app: 0|req: 686/1375] 10.222.0.47 () {58 vars in 1023 bytes} [Thu Jul 21 08:22:44 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 196 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 08:22:44.552634 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70df1255-463a-4c1b-a2d7-9ac34ed2ef3e None None] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:22:44.555345 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.fernet_utils [None req-70df1255-463a-4c1b-a2d7-9ac34ed2ef3e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80244) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 08:22:44.725675 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70df1255-463a-4c1b-a2d7-9ac34ed2ef3e None None] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:22:44.777973 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG dbcounter [-] [80244] Writing DB stats keystone:SELECT=100 {{(pid=80244) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:22:44.839853 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c79c13ba-6a14-4c37-ac36-0136915cbc9d service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2b5b99a36442148fe743269054e991', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4faef13aa50e4e36b6c265fb1398fa14', 'project_domain_id': 'default', 'roles': ['member', 'service', 'ResellerAdmin', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:22:44.840507 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c79c13ba-6a14-4c37-ac36-0136915cbc9d service nova] REQUEST_METHOD: `GET` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:22:44.840790 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c79c13ba-6a14-4c37-ac36-0136915cbc9d service nova] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:22:44.841063 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c79c13ba-6a14-4c37-ac36-0136915cbc9d service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:22:44.842719 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.fernet_utils [None req-c79c13ba-6a14-4c37-ac36-0136915cbc9d service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80244) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 08:22:44.937497 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c79c13ba-6a14-4c37-ac36-0136915cbc9d service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:22:44.939099 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c79c13ba-6a14-4c37-ac36-0136915cbc9d service nova] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:22:44.957347 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 690/1376] 10.222.0.47 () {60 vars in 1365 bytes} [Thu Jul 21 08:22:44 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 406 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 08:22:47.037763 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e0e48ac-b664-4c2a-8b4c-9fff4dddf0f6 None None] REQUEST_METHOD: `GET` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:22:47.038306 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e0e48ac-b664-4c2a-8b4c-9fff4dddf0f6 None None] SCRIPT_NAME: `/identity` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:22:47.038306 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e0e48ac-b664-4c2a-8b4c-9fff4dddf0f6 None None] PATH_INFO: `/` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:22:47.039683 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: [pid: 80245|app: 0|req: 687/1377] 10.222.0.47 () {54 vars in 926 bytes} [Thu Jul 21 08:22:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 08:22:47.045024 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21bd6664-6dea-433f-bba2-ea3301ec6081 None None] REQUEST_METHOD: `POST` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:22:47.045501 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21bd6664-6dea-433f-bba2-ea3301ec6081 None None] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:22:47.045598 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21bd6664-6dea-433f-bba2-ea3301ec6081 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:22:47.111004 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.auth.core [None req-21bd6664-6dea-433f-bba2-ea3301ec6081 None None] MFA Rules not processed for user `2dce40b8d74c41ec94cc36e6ff05d412`. Rule list: `[]` (Enabled: `True`). {{(pid=80244) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 08:22:47.112228 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.fernet_utils [None req-21bd6664-6dea-433f-bba2-ea3301ec6081 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80244) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 08:22:47.124144 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG dbcounter [-] [80244] Writing DB stats keystone:SELECT=100 {{(pid=80244) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:22:47.223065 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 691/1378] 10.222.0.47 () {58 vars in 1023 bytes} [Thu Jul 21 08:22:47 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 179 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 21 08:22:47.272807 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e0e48ac-b664-4c2a-8b4c-9fff4dddf0f6 None None] Authenticating user token {{(pid=80245) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:22:47.274426 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.fernet_utils [None req-6e0e48ac-b664-4c2a-8b4c-9fff4dddf0f6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80245) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 08:22:47.347657 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG dbcounter [-] [80245] Writing DB stats keystone:SELECT=100 {{(pid=80245) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:22:47.444421 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e0e48ac-b664-4c2a-8b4c-9fff4dddf0f6 None None] Validating token access rules against request {{(pid=80245) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:22:47.569012 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f09547f8-6b3b-49af-8c90-0b2cda5d35db service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2b5b99a36442148fe743269054e991', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4faef13aa50e4e36b6c265fb1398fa14', 'project_domain_id': 'default', 'roles': ['member', 'service', 'ResellerAdmin', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80245) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:22:47.569668 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f09547f8-6b3b-49af-8c90-0b2cda5d35db service nova] REQUEST_METHOD: `GET` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:22:47.569950 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f09547f8-6b3b-49af-8c90-0b2cda5d35db service nova] SCRIPT_NAME: `/identity` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:22:47.570962 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f09547f8-6b3b-49af-8c90-0b2cda5d35db service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:22:47.572626 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.fernet_utils [None req-f09547f8-6b3b-49af-8c90-0b2cda5d35db service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80245) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 08:22:47.590822 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG dbcounter [-] [80245] Writing DB stats keystone:SELECT=100 {{(pid=80245) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:22:47.663508 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f09547f8-6b3b-49af-8c90-0b2cda5d35db service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:22:47.665847 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f09547f8-6b3b-49af-8c90-0b2cda5d35db service nova] RBAC: Authorization granted {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:22:47.694366 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: [pid: 80245|app: 0|req: 688/1379] 10.222.0.47 () {60 vars in 1365 bytes} [Thu Jul 21 08:22:47 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 426 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 08:22:49.810999 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-661cb675-d40b-4dc7-9671-4e647986e64d None None] REQUEST_METHOD: `GET` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:22:49.811561 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-661cb675-d40b-4dc7-9671-4e647986e64d None None] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:22:49.811561 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-661cb675-d40b-4dc7-9671-4e647986e64d None None] PATH_INFO: `/` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:22:49.813109 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 692/1380] 10.222.0.47 () {54 vars in 926 bytes} [Thu Jul 21 08:22:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 08:22:49.818286 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb9a584a-9558-4325-9a20-d96fe4f89e81 None None] REQUEST_METHOD: `POST` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:22:49.818774 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb9a584a-9558-4325-9a20-d96fe4f89e81 None None] SCRIPT_NAME: `/identity` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:22:49.819022 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb9a584a-9558-4325-9a20-d96fe4f89e81 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:22:49.900080 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.auth.core [None req-cb9a584a-9558-4325-9a20-d96fe4f89e81 None None] MFA Rules not processed for user `2dce40b8d74c41ec94cc36e6ff05d412`. Rule list: `[]` (Enabled: `True`). {{(pid=80245) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 08:22:49.901245 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.fernet_utils [None req-cb9a584a-9558-4325-9a20-d96fe4f89e81 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80245) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 08:22:49.997881 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG dbcounter [-] [80245] Writing DB stats keystone:SELECT=100 {{(pid=80245) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:22:50.008148 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: [pid: 80245|app: 0|req: 689/1381] 10.222.0.47 () {58 vars in 1023 bytes} [Thu Jul 21 08:22:49 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 190 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 08:22:50.049899 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-661cb675-d40b-4dc7-9671-4e647986e64d None None] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:22:50.051867 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.fernet_utils [None req-661cb675-d40b-4dc7-9671-4e647986e64d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80244) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 08:22:50.216079 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG dbcounter [-] [80244] Writing DB stats keystone:SELECT=100 {{(pid=80244) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:22:50.228180 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-661cb675-d40b-4dc7-9671-4e647986e64d None None] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:22:50.349975 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98e8d95b-1b02-4fa7-b2cb-964f74695611 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2b5b99a36442148fe743269054e991', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4faef13aa50e4e36b6c265fb1398fa14', 'project_domain_id': 'default', 'roles': ['member', 'service', 'ResellerAdmin', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:22:50.350632 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98e8d95b-1b02-4fa7-b2cb-964f74695611 service nova] REQUEST_METHOD: `GET` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:22:50.350918 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98e8d95b-1b02-4fa7-b2cb-964f74695611 service nova] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:22:50.351197 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98e8d95b-1b02-4fa7-b2cb-964f74695611 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:22:50.352902 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.fernet_utils [None req-98e8d95b-1b02-4fa7-b2cb-964f74695611 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80244) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 08:22:50.451405 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98e8d95b-1b02-4fa7-b2cb-964f74695611 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:22:50.452954 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98e8d95b-1b02-4fa7-b2cb-964f74695611 service nova] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:22:50.463985 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG dbcounter [-] [80244] Writing DB stats keystone:SELECT=100 {{(pid=80244) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:22:50.478380 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 693/1382] 10.222.0.47 () {60 vars in 1365 bytes} [Thu Jul 21 08:22:50 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 429 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 08:22:52.681110 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e88122fb-e350-446f-829c-fae53ff19b20 None None] REQUEST_METHOD: `GET` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:22:52.681779 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e88122fb-e350-446f-829c-fae53ff19b20 None None] SCRIPT_NAME: `/identity` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:22:52.681779 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e88122fb-e350-446f-829c-fae53ff19b20 None None] PATH_INFO: `/` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:22:52.683286 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: [pid: 80245|app: 0|req: 690/1383] 10.222.0.47 () {54 vars in 926 bytes} [Thu Jul 21 08:22:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 08:22:52.688910 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c34da65c-747a-42ae-92fe-0a60d0cdf3a7 None None] REQUEST_METHOD: `POST` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:22:52.689326 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c34da65c-747a-42ae-92fe-0a60d0cdf3a7 None None] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:22:52.689541 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c34da65c-747a-42ae-92fe-0a60d0cdf3a7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:22:52.762411 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.auth.core [None req-c34da65c-747a-42ae-92fe-0a60d0cdf3a7 None None] MFA Rules not processed for user `2dce40b8d74c41ec94cc36e6ff05d412`. Rule list: `[]` (Enabled: `True`). {{(pid=80244) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 08:22:52.763676 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.fernet_utils [None req-c34da65c-747a-42ae-92fe-0a60d0cdf3a7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80244) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 08:22:52.873859 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 694/1384] 10.222.0.47 () {58 vars in 1023 bytes} [Thu Jul 21 08:22:52 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 186 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 08:22:52.920351 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e88122fb-e350-446f-829c-fae53ff19b20 None None] Authenticating user token {{(pid=80245) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:22:52.922268 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.fernet_utils [None req-e88122fb-e350-446f-829c-fae53ff19b20 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80245) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 08:22:53.082694 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e88122fb-e350-446f-829c-fae53ff19b20 None None] Validating token access rules against request {{(pid=80245) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:22:53.150681 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG dbcounter [-] [80245] Writing DB stats keystone:SELECT=100 {{(pid=80245) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:22:53.198491 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1ecd5d2-7949-4d51-8744-72bb2a0943b2 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2b5b99a36442148fe743269054e991', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4faef13aa50e4e36b6c265fb1398fa14', 'project_domain_id': 'default', 'roles': ['member', 'service', 'ResellerAdmin', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80245) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:22:53.199141 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1ecd5d2-7949-4d51-8744-72bb2a0943b2 service nova] REQUEST_METHOD: `GET` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:22:53.199457 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1ecd5d2-7949-4d51-8744-72bb2a0943b2 service nova] SCRIPT_NAME: `/identity` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:22:53.199736 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1ecd5d2-7949-4d51-8744-72bb2a0943b2 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:22:53.201371 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.fernet_utils [None req-d1ecd5d2-7949-4d51-8744-72bb2a0943b2 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80245) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 08:22:53.292428 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1ecd5d2-7949-4d51-8744-72bb2a0943b2 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:22:53.294237 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1ecd5d2-7949-4d51-8744-72bb2a0943b2 service nova] RBAC: Authorization granted {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:22:53.316405 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: [pid: 80245|app: 0|req: 691/1385] 10.222.0.47 () {60 vars in 1365 bytes} [Thu Jul 21 08:22:52 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 397 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 08:22:55.601956 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1497b4c2-73b9-4d37-871b-8cf02cef1462 None None] REQUEST_METHOD: `GET` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:22:55.602937 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1497b4c2-73b9-4d37-871b-8cf02cef1462 None None] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:22:55.602937 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1497b4c2-73b9-4d37-871b-8cf02cef1462 None None] PATH_INFO: `/` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:22:55.603953 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 695/1386] 10.222.0.47 () {54 vars in 926 bytes} [Thu Jul 21 08:22:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 08:22:55.609433 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5a4a2cd-5dac-4673-8bfd-6343dc654a77 None None] REQUEST_METHOD: `POST` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:22:55.609902 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5a4a2cd-5dac-4673-8bfd-6343dc654a77 None None] SCRIPT_NAME: `/identity` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:22:55.610010 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5a4a2cd-5dac-4673-8bfd-6343dc654a77 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:22:55.682950 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.auth.core [None req-e5a4a2cd-5dac-4673-8bfd-6343dc654a77 None None] MFA Rules not processed for user `2dce40b8d74c41ec94cc36e6ff05d412`. Rule list: `[]` (Enabled: `True`). {{(pid=80245) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 08:22:55.684481 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.fernet_utils [None req-e5a4a2cd-5dac-4673-8bfd-6343dc654a77 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80245) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 08:22:55.722182 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG dbcounter [-] [80245] Writing DB stats keystone:SELECT=100 {{(pid=80245) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:22:55.807448 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: [pid: 80245|app: 0|req: 692/1387] 10.222.0.47 () {58 vars in 1023 bytes} [Thu Jul 21 08:22:55 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 199 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 08:22:55.867814 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1497b4c2-73b9-4d37-871b-8cf02cef1462 None None] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:22:55.869381 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.fernet_utils [None req-1497b4c2-73b9-4d37-871b-8cf02cef1462 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80244) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 08:22:55.952348 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG dbcounter [-] [80244] Writing DB stats keystone:SELECT=100 {{(pid=80244) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:22:56.023133 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1497b4c2-73b9-4d37-871b-8cf02cef1462 None None] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:22:56.142375 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-418230e1-13ef-43ef-a281-e8df07257032 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2b5b99a36442148fe743269054e991', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4faef13aa50e4e36b6c265fb1398fa14', 'project_domain_id': 'default', 'roles': ['member', 'service', 'ResellerAdmin', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:22:56.143101 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-418230e1-13ef-43ef-a281-e8df07257032 service nova] REQUEST_METHOD: `GET` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:22:56.143490 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-418230e1-13ef-43ef-a281-e8df07257032 service nova] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:22:56.143818 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-418230e1-13ef-43ef-a281-e8df07257032 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:22:56.146419 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.fernet_utils [None req-418230e1-13ef-43ef-a281-e8df07257032 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80244) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 08:22:56.188838 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG dbcounter [-] [80244] Writing DB stats keystone:SELECT=100 {{(pid=80244) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:22:56.253660 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-418230e1-13ef-43ef-a281-e8df07257032 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:22:56.255882 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-418230e1-13ef-43ef-a281-e8df07257032 service nova] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:22:56.279587 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 696/1388] 10.222.0.47 () {60 vars in 1365 bytes} [Thu Jul 21 08:22:55 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 413 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 08:22:58.377975 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-769fbd9f-fbcf-4f24-94b0-52a61ffee436 None None] REQUEST_METHOD: `GET` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:22:58.378532 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-769fbd9f-fbcf-4f24-94b0-52a61ffee436 None None] SCRIPT_NAME: `/identity` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:22:58.378532 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-769fbd9f-fbcf-4f24-94b0-52a61ffee436 None None] PATH_INFO: `/` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:22:58.379942 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: [pid: 80245|app: 0|req: 693/1389] 10.222.0.47 () {54 vars in 926 bytes} [Thu Jul 21 08:22:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 08:22:58.387793 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f56b44e-729f-417e-86bf-a2bbb69e5e42 None None] REQUEST_METHOD: `POST` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:22:58.388761 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f56b44e-729f-417e-86bf-a2bbb69e5e42 None None] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:22:58.389104 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f56b44e-729f-417e-86bf-a2bbb69e5e42 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:22:58.467178 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.auth.core [None req-5f56b44e-729f-417e-86bf-a2bbb69e5e42 None None] MFA Rules not processed for user `2dce40b8d74c41ec94cc36e6ff05d412`. Rule list: `[]` (Enabled: `True`). {{(pid=80244) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 08:22:58.468426 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.fernet_utils [None req-5f56b44e-729f-417e-86bf-a2bbb69e5e42 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80244) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 08:22:58.572707 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 697/1390] 10.222.0.47 () {58 vars in 1023 bytes} [Thu Jul 21 08:22:58 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 187 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 08:22:58.619283 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-769fbd9f-fbcf-4f24-94b0-52a61ffee436 None None] Authenticating user token {{(pid=80245) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:22:58.620914 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.fernet_utils [None req-769fbd9f-fbcf-4f24-94b0-52a61ffee436 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80245) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 08:22:58.777437 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-769fbd9f-fbcf-4f24-94b0-52a61ffee436 None None] Validating token access rules against request {{(pid=80245) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:22:58.794959 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG dbcounter [-] [80245] Writing DB stats keystone:SELECT=100 {{(pid=80245) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:22:58.902212 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d915c455-d865-4e41-b353-2b05e51202b2 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2b5b99a36442148fe743269054e991', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4faef13aa50e4e36b6c265fb1398fa14', 'project_domain_id': 'default', 'roles': ['member', 'service', 'ResellerAdmin', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80245) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:22:58.902875 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d915c455-d865-4e41-b353-2b05e51202b2 service nova] REQUEST_METHOD: `GET` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:22:58.903205 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d915c455-d865-4e41-b353-2b05e51202b2 service nova] SCRIPT_NAME: `/identity` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:22:58.903584 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d915c455-d865-4e41-b353-2b05e51202b2 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:22:58.905677 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.fernet_utils [None req-d915c455-d865-4e41-b353-2b05e51202b2 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80245) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 08:22:59.003897 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d915c455-d865-4e41-b353-2b05e51202b2 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:22:59.005715 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d915c455-d865-4e41-b353-2b05e51202b2 service nova] RBAC: Authorization granted {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:22:59.027290 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: [pid: 80245|app: 0|req: 694/1391] 10.222.0.47 () {60 vars in 1365 bytes} [Thu Jul 21 08:22:58 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 408 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 08:23:01.216454 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff7f71f1-d80f-4d97-bbb0-f34196e62fd9 None None] REQUEST_METHOD: `GET` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:23:01.217296 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff7f71f1-d80f-4d97-bbb0-f34196e62fd9 None None] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:23:01.217296 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff7f71f1-d80f-4d97-bbb0-f34196e62fd9 None None] PATH_INFO: `/` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:23:01.218286 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 698/1392] 10.222.0.47 () {54 vars in 926 bytes} [Thu Jul 21 08:23:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 08:23:01.224722 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58027e09-307d-4048-9cba-f2fffd1bd7e5 None None] REQUEST_METHOD: `POST` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:23:01.225266 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58027e09-307d-4048-9cba-f2fffd1bd7e5 None None] SCRIPT_NAME: `/identity` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:23:01.225638 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58027e09-307d-4048-9cba-f2fffd1bd7e5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:23:01.264546 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG dbcounter [-] [80245] Writing DB stats keystone:SELECT=100 {{(pid=80245) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:23:01.311406 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.auth.core [None req-58027e09-307d-4048-9cba-f2fffd1bd7e5 None None] MFA Rules not processed for user `2dce40b8d74c41ec94cc36e6ff05d412`. Rule list: `[]` (Enabled: `True`). {{(pid=80245) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 08:23:01.312643 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.fernet_utils [None req-58027e09-307d-4048-9cba-f2fffd1bd7e5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80245) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 08:23:01.428159 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: [pid: 80245|app: 0|req: 695/1393] 10.222.0.47 () {58 vars in 1023 bytes} [Thu Jul 21 08:23:01 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 205 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 08:23:01.476606 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff7f71f1-d80f-4d97-bbb0-f34196e62fd9 None None] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:23:01.478064 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.fernet_utils [None req-ff7f71f1-d80f-4d97-bbb0-f34196e62fd9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80244) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 08:23:01.495401 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG dbcounter [-] [80244] Writing DB stats keystone:SELECT=100 {{(pid=80244) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:23:01.675589 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff7f71f1-d80f-4d97-bbb0-f34196e62fd9 None None] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:23:01.779699 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG dbcounter [-] [80244] Writing DB stats keystone:SELECT=100 {{(pid=80244) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:23:01.813881 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aa8f310-0568-4dd7-a144-cadacfd78f18 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2b5b99a36442148fe743269054e991', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4faef13aa50e4e36b6c265fb1398fa14', 'project_domain_id': 'default', 'roles': ['member', 'service', 'ResellerAdmin', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:23:01.814584 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aa8f310-0568-4dd7-a144-cadacfd78f18 service nova] REQUEST_METHOD: `GET` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:23:01.814892 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aa8f310-0568-4dd7-a144-cadacfd78f18 service nova] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:23:01.815197 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aa8f310-0568-4dd7-a144-cadacfd78f18 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:23:01.817125 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.fernet_utils [None req-4aa8f310-0568-4dd7-a144-cadacfd78f18 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80244) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 08:23:01.938222 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4aa8f310-0568-4dd7-a144-cadacfd78f18 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:23:01.940411 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4aa8f310-0568-4dd7-a144-cadacfd78f18 service nova] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:23:01.966180 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 699/1394] 10.222.0.47 () {60 vars in 1365 bytes} [Thu Jul 21 08:23:01 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 491 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 08:23:04.140872 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67d8c2fd-3da6-4d58-a10a-6af8d35cc7f4 None None] REQUEST_METHOD: `GET` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:23:04.141427 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67d8c2fd-3da6-4d58-a10a-6af8d35cc7f4 None None] SCRIPT_NAME: `/identity` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:23:04.141427 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67d8c2fd-3da6-4d58-a10a-6af8d35cc7f4 None None] PATH_INFO: `/` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:23:04.142858 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: [pid: 80245|app: 0|req: 696/1395] 10.222.0.47 () {54 vars in 926 bytes} [Thu Jul 21 08:23:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 08:23:04.148163 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ed8854b-a6de-4f0f-9204-52bb592a1088 None None] REQUEST_METHOD: `POST` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:23:04.148615 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ed8854b-a6de-4f0f-9204-52bb592a1088 None None] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:23:04.148835 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ed8854b-a6de-4f0f-9204-52bb592a1088 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:23:04.214765 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.auth.core [None req-8ed8854b-a6de-4f0f-9204-52bb592a1088 None None] MFA Rules not processed for user `2dce40b8d74c41ec94cc36e6ff05d412`. Rule list: `[]` (Enabled: `True`). {{(pid=80244) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 08:23:04.215994 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.fernet_utils [None req-8ed8854b-a6de-4f0f-9204-52bb592a1088 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80244) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 08:23:04.266882 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG dbcounter [-] [80244] Writing DB stats keystone:SELECT=100 {{(pid=80244) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:23:04.314884 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 700/1396] 10.222.0.47 () {58 vars in 1023 bytes} [Thu Jul 21 08:23:04 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 08:23:04.358595 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67d8c2fd-3da6-4d58-a10a-6af8d35cc7f4 None None] Authenticating user token {{(pid=80245) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:23:04.360249 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.fernet_utils [None req-67d8c2fd-3da6-4d58-a10a-6af8d35cc7f4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80245) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 08:23:04.460610 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG dbcounter [-] [80245] Writing DB stats keystone:SELECT=100 {{(pid=80245) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:23:04.524543 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67d8c2fd-3da6-4d58-a10a-6af8d35cc7f4 None None] Validating token access rules against request {{(pid=80245) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:23:04.603337 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0a380f0-d874-4893-bdbe-45c00e6fdad1 None None] REQUEST_METHOD: `GET` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:23:04.603754 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0a380f0-d874-4893-bdbe-45c00e6fdad1 None None] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:23:04.603965 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0a380f0-d874-4893-bdbe-45c00e6fdad1 None None] PATH_INFO: `/` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:23:04.605515 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 701/1397] 10.222.0.47 () {54 vars in 926 bytes} [Thu Jul 21 08:23:04 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 08:23:04.610597 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5490310-a79d-41ca-8afe-c116fc9de2c7 None None] REQUEST_METHOD: `POST` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:23:04.610998 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5490310-a79d-41ca-8afe-c116fc9de2c7 None None] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:23:04.611277 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5490310-a79d-41ca-8afe-c116fc9de2c7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:23:04.655430 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39daa5aa-916c-4c05-9360-1d6b290491b4 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2b5b99a36442148fe743269054e991', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4faef13aa50e4e36b6c265fb1398fa14', 'project_domain_id': 'default', 'roles': ['member', 'service', 'ResellerAdmin', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80245) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:23:04.656125 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39daa5aa-916c-4c05-9360-1d6b290491b4 service nova] REQUEST_METHOD: `GET` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:23:04.656652 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39daa5aa-916c-4c05-9360-1d6b290491b4 service nova] SCRIPT_NAME: `/identity` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:23:04.656652 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39daa5aa-916c-4c05-9360-1d6b290491b4 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:23:04.658434 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.fernet_utils [None req-39daa5aa-916c-4c05-9360-1d6b290491b4 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80245) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 08:23:04.685980 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.auth.core [None req-b5490310-a79d-41ca-8afe-c116fc9de2c7 None None] MFA Rules not processed for user `2dce40b8d74c41ec94cc36e6ff05d412`. Rule list: `[]` (Enabled: `True`). {{(pid=80244) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 08:23:04.687194 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.fernet_utils [None req-b5490310-a79d-41ca-8afe-c116fc9de2c7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80244) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 08:23:04.729110 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG dbcounter [-] [80245] Writing DB stats keystone:SELECT=100 {{(pid=80245) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:23:04.757780 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39daa5aa-916c-4c05-9360-1d6b290491b4 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:23:04.759417 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39daa5aa-916c-4c05-9360-1d6b290491b4 service nova] RBAC: Authorization granted {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:23:04.777976 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: [pid: 80245|app: 0|req: 697/1398] 10.222.0.47 () {60 vars in 1365 bytes} [Thu Jul 21 08:23:04 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 420 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 08:23:04.792251 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 702/1399] 10.222.0.47 () {58 vars in 1023 bytes} [Thu Jul 21 08:23:04 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 182 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 21 08:23:04.850116 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b27f627-067a-42b5-baca-0825a033f26d None None] REQUEST_METHOD: `GET` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:23:04.850405 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b27f627-067a-42b5-baca-0825a033f26d None None] SCRIPT_NAME: `/identity` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:23:04.850888 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b27f627-067a-42b5-baca-0825a033f26d None None] PATH_INFO: `/` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:23:04.852490 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: [pid: 80245|app: 0|req: 698/1400] 10.222.0.47 () {54 vars in 966 bytes} [Thu Jul 21 08:23:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 08:23:04.858546 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ba3d1a6-70fb-45d5-8bc1-1543e272bdaf None None] REQUEST_METHOD: `POST` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:23:04.858972 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ba3d1a6-70fb-45d5-8bc1-1543e272bdaf None None] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:23:04.859352 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ba3d1a6-70fb-45d5-8bc1-1543e272bdaf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:23:04.904115 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG dbcounter [-] [80244] Writing DB stats keystone:SELECT=100 {{(pid=80244) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:23:04.950597 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.auth.core [None req-8ba3d1a6-70fb-45d5-8bc1-1543e272bdaf None None] MFA Rules not processed for user `9b0524aed8c74abd8949b860d6b93936`. Rule list: `[]` (Enabled: `True`). {{(pid=80244) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 08:23:04.951890 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.fernet_utils [None req-8ba3d1a6-70fb-45d5-8bc1-1543e272bdaf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80244) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 08:23:05.034520 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 703/1401] 10.222.0.47 () {58 vars in 1063 bytes} [Thu Jul 21 08:23:04 2022] POST /identity/v3/auth/tokens => generated 3110 bytes in 177 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 08:23:05.044146 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b27f627-067a-42b5-baca-0825a033f26d None None] Authenticating user token {{(pid=80245) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:23:05.045912 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.fernet_utils [None req-9b27f627-067a-42b5-baca-0825a033f26d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80245) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 08:23:05.158175 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b27f627-067a-42b5-baca-0825a033f26d None None] Validating token access rules against request {{(pid=80245) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:23:05.231837 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce29fa44-af56-44dd-b0c0-942f8f1bbd1e service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9b0524aed8c74abd8949b860d6b93936', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4faef13aa50e4e36b6c265fb1398fa14', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80245) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:23:05.232495 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce29fa44-af56-44dd-b0c0-942f8f1bbd1e service glance] REQUEST_METHOD: `GET` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:23:05.232779 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce29fa44-af56-44dd-b0c0-942f8f1bbd1e service glance] SCRIPT_NAME: `/identity` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:23:05.233051 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce29fa44-af56-44dd-b0c0-942f8f1bbd1e service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:23:05.234690 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.fernet_utils [None req-ce29fa44-af56-44dd-b0c0-942f8f1bbd1e service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80245) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 08:23:05.248120 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG dbcounter [-] [80245] Writing DB stats keystone:SELECT=100 {{(pid=80245) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:23:05.359917 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce29fa44-af56-44dd-b0c0-942f8f1bbd1e service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:23:05.361340 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce29fa44-af56-44dd-b0c0-942f8f1bbd1e service glance] RBAC: Authorization granted {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:23:05.380533 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: [pid: 80245|app: 0|req: 699/1402] 10.222.0.47 () {60 vars in 1365 bytes} [Thu Jul 21 08:23:05 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 337 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 08:23:06.103061 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c776d69-8743-485b-8314-7269be64e231 None None] REQUEST_METHOD: `POST` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:23:06.103587 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c776d69-8743-485b-8314-7269be64e231 None None] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:23:06.103799 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c776d69-8743-485b-8314-7269be64e231 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:23:06.173093 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.auth.core [None req-7c776d69-8743-485b-8314-7269be64e231 None None] MFA Rules not processed for user `2ef7f1e91e554a1f88eb6a63e60a475f`. Rule list: `[]` (Enabled: `True`). {{(pid=80244) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 08:23:06.174328 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.fernet_utils [None req-7c776d69-8743-485b-8314-7269be64e231 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80244) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 08:23:06.251127 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG dbcounter [-] [80244] Writing DB stats keystone:SELECT=100 {{(pid=80244) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:23:06.264327 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 704/1403] 10.222.0.47 () {58 vars in 1009 bytes} [Thu Jul 21 08:23:06 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 21 08:23:06.306803 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ee97881-5ac3-4942-bc4a-ed811b1c5117 None None] REQUEST_METHOD: `GET` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:23:06.307289 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ee97881-5ac3-4942-bc4a-ed811b1c5117 None None] SCRIPT_NAME: `/identity` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:23:06.307414 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ee97881-5ac3-4942-bc4a-ed811b1c5117 None None] PATH_INFO: `/` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:23:06.308753 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: [pid: 80245|app: 0|req: 700/1404] 10.222.0.47 () {54 vars in 943 bytes} [Thu Jul 21 08:23:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 08:23:06.315115 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59945276-e292-4da3-8a72-cbbce74c55e5 None None] REQUEST_METHOD: `POST` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:23:06.315441 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59945276-e292-4da3-8a72-cbbce74c55e5 None None] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:23:06.315726 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59945276-e292-4da3-8a72-cbbce74c55e5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:23:06.393036 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.auth.core [None req-59945276-e292-4da3-8a72-cbbce74c55e5 None None] MFA Rules not processed for user `302351411fc84248a6b166b25d21609f`. Rule list: `[]` (Enabled: `True`). {{(pid=80244) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 08:23:06.394206 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.fernet_utils [None req-59945276-e292-4da3-8a72-cbbce74c55e5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80244) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 08:23:06.524263 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 705/1405] 10.222.0.47 () {58 vars in 1040 bytes} [Thu Jul 21 08:23:06 2022] POST /identity/v3/auth/tokens => generated 3294 bytes in 210 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 08:23:06.536111 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ee97881-5ac3-4942-bc4a-ed811b1c5117 None None] Authenticating user token {{(pid=80245) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:23:06.537717 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.fernet_utils [None req-8ee97881-5ac3-4942-bc4a-ed811b1c5117 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80245) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 08:23:06.652400 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG dbcounter [-] [80245] Writing DB stats keystone:SELECT=100 {{(pid=80245) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:23:06.677361 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ee97881-5ac3-4942-bc4a-ed811b1c5117 None None] Validating token access rules against request {{(pid=80245) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:23:06.790533 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e971670-65ec-4767-8b05-a700d324b29d service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '302351411fc84248a6b166b25d21609f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4faef13aa50e4e36b6c265fb1398fa14', 'project_domain_id': 'default', 'roles': ['service', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80245) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:23:06.791194 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e971670-65ec-4767-8b05-a700d324b29d service swift] REQUEST_METHOD: `GET` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:23:06.791509 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e971670-65ec-4767-8b05-a700d324b29d service swift] SCRIPT_NAME: `/identity` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:23:06.791789 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e971670-65ec-4767-8b05-a700d324b29d service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:23:06.793459 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.fernet_utils [None req-7e971670-65ec-4767-8b05-a700d324b29d service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80245) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 08:23:06.880099 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e971670-65ec-4767-8b05-a700d324b29d service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:23:06.881703 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e971670-65ec-4767-8b05-a700d324b29d service swift] RBAC: Authorization granted {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:23:06.891224 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: [pid: 80245|app: 0|req: 701/1406] 10.222.0.47 () {60 vars in 1384 bytes} [Thu Jul 21 08:23:06 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 356 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 21 08:23:06.905103 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac2ea108-00c5-4d6b-8262-555154cd11ee None None] REQUEST_METHOD: `GET` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:23:06.905374 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac2ea108-00c5-4d6b-8262-555154cd11ee None None] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:23:06.905553 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac2ea108-00c5-4d6b-8262-555154cd11ee None None] PATH_INFO: `/` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:23:06.906947 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 706/1407] 10.222.0.47 () {54 vars in 926 bytes} [Thu Jul 21 08:23:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 08:23:06.912360 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-439ce6dd-40e2-4cbe-a7bd-396ce10f5348 None None] REQUEST_METHOD: `POST` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:23:06.912863 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-439ce6dd-40e2-4cbe-a7bd-396ce10f5348 None None] SCRIPT_NAME: `/identity` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:23:06.913202 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-439ce6dd-40e2-4cbe-a7bd-396ce10f5348 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:23:06.929435 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG dbcounter [-] [80245] Writing DB stats keystone:SELECT=100 {{(pid=80245) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:23:06.989801 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.auth.core [None req-439ce6dd-40e2-4cbe-a7bd-396ce10f5348 None None] MFA Rules not processed for user `2dce40b8d74c41ec94cc36e6ff05d412`. Rule list: `[]` (Enabled: `True`). {{(pid=80245) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 08:23:06.991889 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.fernet_utils [None req-439ce6dd-40e2-4cbe-a7bd-396ce10f5348 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80245) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 08:23:07.111289 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: [pid: 80245|app: 0|req: 702/1408] 10.222.0.47 () {58 vars in 1023 bytes} [Thu Jul 21 08:23:06 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 200 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 08:23:07.157800 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac2ea108-00c5-4d6b-8262-555154cd11ee None None] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:23:07.159255 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.fernet_utils [None req-ac2ea108-00c5-4d6b-8262-555154cd11ee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80244) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 08:23:07.216566 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG dbcounter [-] [80244] Writing DB stats keystone:SELECT=100 {{(pid=80244) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:23:07.340301 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac2ea108-00c5-4d6b-8262-555154cd11ee None None] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:23:07.480804 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93e929c6-3a63-46be-829d-a25c120bce3c service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2b5b99a36442148fe743269054e991', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4faef13aa50e4e36b6c265fb1398fa14', 'project_domain_id': 'default', 'roles': ['member', 'service', 'ResellerAdmin', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:23:07.481652 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93e929c6-3a63-46be-829d-a25c120bce3c service nova] REQUEST_METHOD: `GET` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:23:07.482161 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93e929c6-3a63-46be-829d-a25c120bce3c service nova] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:23:07.482655 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93e929c6-3a63-46be-829d-a25c120bce3c service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:23:07.484903 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.fernet_utils [None req-93e929c6-3a63-46be-829d-a25c120bce3c service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80244) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 08:23:07.495729 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG dbcounter [-] [80244] Writing DB stats keystone:SELECT=100 {{(pid=80244) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:23:07.593064 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93e929c6-3a63-46be-829d-a25c120bce3c service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:23:07.594759 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93e929c6-3a63-46be-829d-a25c120bce3c service nova] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:23:07.619782 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 707/1409] 10.222.0.47 () {60 vars in 1365 bytes} [Thu Jul 21 08:23:07 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 463 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 08:23:17.109230 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG dbcounter [-] [80245] Writing DB stats keystone:SELECT=59 {{(pid=80245) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:23:17.617320 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG dbcounter [-] [80244] Writing DB stats keystone:SELECT=41 {{(pid=80244) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:23:29.200698 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b486b92-02e9-4889-a39d-bf8b5c08f944 None None] REQUEST_METHOD: `GET` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:23:29.201321 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b486b92-02e9-4889-a39d-bf8b5c08f944 None None] SCRIPT_NAME: `/identity` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:23:29.201321 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b486b92-02e9-4889-a39d-bf8b5c08f944 None None] PATH_INFO: `/` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:23:29.202665 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: [pid: 80245|app: 0|req: 703/1410] 10.222.0.47 () {54 vars in 918 bytes} [Thu Jul 21 08:23:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 08:23:29.207758 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0d1f7af-9eda-4e28-9e9a-4d56f20a0012 None None] REQUEST_METHOD: `POST` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:23:29.208162 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0d1f7af-9eda-4e28-9e9a-4d56f20a0012 None None] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:23:29.208324 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0d1f7af-9eda-4e28-9e9a-4d56f20a0012 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:23:29.289488 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.auth.core [None req-d0d1f7af-9eda-4e28-9e9a-4d56f20a0012 None None] MFA Rules not processed for user `3379b32543474d0a8e897484e85c7d04`. Rule list: `[]` (Enabled: `True`). {{(pid=80244) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 08:23:29.290703 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.fernet_utils [None req-d0d1f7af-9eda-4e28-9e9a-4d56f20a0012 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80244) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 08:23:29.394793 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 708/1411] 10.222.0.47 () {58 vars in 1015 bytes} [Thu Jul 21 08:23:29 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 188 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 08:23:29.409582 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdbdf0b9-9b85-4031-863c-2c04cf7c5619 None None] REQUEST_METHOD: `GET` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:23:29.409872 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdbdf0b9-9b85-4031-863c-2c04cf7c5619 None None] SCRIPT_NAME: `/identity` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:23:29.410153 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdbdf0b9-9b85-4031-863c-2c04cf7c5619 None None] PATH_INFO: `/` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:23:29.411553 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: [pid: 80245|app: 0|req: 704/1412] 10.222.0.47 () {54 vars in 918 bytes} [Thu Jul 21 08:23:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 08:23:29.416687 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac7fa536-c9c2-41c3-830d-8f5580d8c9b0 None None] REQUEST_METHOD: `POST` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:23:29.416973 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac7fa536-c9c2-41c3-830d-8f5580d8c9b0 None None] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:23:29.417247 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac7fa536-c9c2-41c3-830d-8f5580d8c9b0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:23:29.488430 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG dbcounter [-] [80244] Writing DB stats keystone:SELECT=100 {{(pid=80244) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:23:29.491670 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.auth.core [None req-ac7fa536-c9c2-41c3-830d-8f5580d8c9b0 None None] MFA Rules not processed for user `ac2b5b99a36442148fe743269054e991`. Rule list: `[]` (Enabled: `True`). {{(pid=80244) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 08:23:29.493450 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.fernet_utils [None req-ac7fa536-c9c2-41c3-830d-8f5580d8c9b0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80244) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 08:23:29.620736 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 709/1413] 10.222.0.47 () {58 vars in 1015 bytes} [Thu Jul 21 08:23:29 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 205 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 08:23:32.180732 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1419be50-4343-42d3-b7ec-3ab148ad4823 None None] REQUEST_METHOD: `GET` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:23:32.181366 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1419be50-4343-42d3-b7ec-3ab148ad4823 None None] SCRIPT_NAME: `/identity` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:23:32.181532 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1419be50-4343-42d3-b7ec-3ab148ad4823 None None] PATH_INFO: `/` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:23:32.182927 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: [pid: 80245|app: 0|req: 705/1414] 127.0.0.1 () {54 vars in 916 bytes} [Thu Jul 21 08:23:32 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 08:23:32.188363 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4b12eec-4f23-402b-9f4e-9b83af19b24d None None] REQUEST_METHOD: `POST` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:23:32.188822 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4b12eec-4f23-402b-9f4e-9b83af19b24d None None] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:23:32.188918 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4b12eec-4f23-402b-9f4e-9b83af19b24d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:23:32.253389 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.auth.core [None req-b4b12eec-4f23-402b-9f4e-9b83af19b24d None None] MFA Rules not processed for user `2dce40b8d74c41ec94cc36e6ff05d412`. Rule list: `[]` (Enabled: `True`). {{(pid=80244) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 08:23:32.254594 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.fernet_utils [None req-b4b12eec-4f23-402b-9f4e-9b83af19b24d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80244) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 08:23:32.310372 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG dbcounter [-] [80244] Writing DB stats keystone:SELECT=100 {{(pid=80244) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:23:32.352587 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 710/1415] 127.0.0.1 () {58 vars in 1013 bytes} [Thu Jul 21 08:23:32 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 08:23:32.395461 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1419be50-4343-42d3-b7ec-3ab148ad4823 None None] Authenticating user token {{(pid=80245) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:23:32.397571 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.fernet_utils [None req-1419be50-4343-42d3-b7ec-3ab148ad4823 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80245) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 08:23:32.542648 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1419be50-4343-42d3-b7ec-3ab148ad4823 None None] Validating token access rules against request {{(pid=80245) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:23:32.622957 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG dbcounter [-] [80245] Writing DB stats keystone:SELECT=100 {{(pid=80245) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:23:32.664640 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-002acf4e-4bf8-4f66-9159-1f836944faa1 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2b5b99a36442148fe743269054e991', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4faef13aa50e4e36b6c265fb1398fa14', 'project_domain_id': 'default', 'roles': ['member', 'service', 'ResellerAdmin', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80245) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:23:32.665300 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-002acf4e-4bf8-4f66-9159-1f836944faa1 service nova] REQUEST_METHOD: `GET` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:23:32.665580 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-002acf4e-4bf8-4f66-9159-1f836944faa1 service nova] SCRIPT_NAME: `/identity` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:23:32.665852 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-002acf4e-4bf8-4f66-9159-1f836944faa1 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:23:32.667516 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.fernet_utils [None req-002acf4e-4bf8-4f66-9159-1f836944faa1 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80245) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 08:23:32.754785 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-002acf4e-4bf8-4f66-9159-1f836944faa1 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:23:32.756870 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-002acf4e-4bf8-4f66-9159-1f836944faa1 service nova] RBAC: Authorization granted {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:23:32.775306 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: [pid: 80245|app: 0|req: 706/1416] 10.222.0.47 () {60 vars in 1365 bytes} [Thu Jul 21 08:23:32 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 380 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 08:23:33.681667 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d9a477a-5361-4259-a970-aea27bea3baa None None] REQUEST_METHOD: `GET` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:23:33.682209 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d9a477a-5361-4259-a970-aea27bea3baa None None] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:23:33.682209 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d9a477a-5361-4259-a970-aea27bea3baa None None] PATH_INFO: `/` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:23:33.683567 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 711/1417] 127.0.0.1 () {54 vars in 901 bytes} [Thu Jul 21 08:23:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 08:23:33.688473 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a57ba08-2415-447d-976d-c31a099bb012 None None] REQUEST_METHOD: `POST` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:23:33.688919 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a57ba08-2415-447d-976d-c31a099bb012 None None] SCRIPT_NAME: `/identity` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:23:33.689135 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a57ba08-2415-447d-976d-c31a099bb012 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:23:33.762620 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.auth.core [None req-9a57ba08-2415-447d-976d-c31a099bb012 None None] MFA Rules not processed for user `2dce40b8d74c41ec94cc36e6ff05d412`. Rule list: `[]` (Enabled: `True`). {{(pid=80245) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 08:23:33.763873 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.fernet_utils [None req-9a57ba08-2415-447d-976d-c31a099bb012 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80245) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 08:23:33.790905 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG dbcounter [-] [80245] Writing DB stats keystone:SELECT=100 {{(pid=80245) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:23:33.858716 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: [pid: 80245|app: 0|req: 707/1418] 127.0.0.1 () {58 vars in 998 bytes} [Thu Jul 21 08:23:33 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 08:23:34.009250 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d9a477a-5361-4259-a970-aea27bea3baa None None] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:23:34.010682 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.fernet_utils [None req-3d9a477a-5361-4259-a970-aea27bea3baa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80244) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 08:23:34.153942 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d9a477a-5361-4259-a970-aea27bea3baa None None] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:23:34.192690 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG dbcounter [-] [80244] Writing DB stats keystone:SELECT=100 {{(pid=80244) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:23:34.266974 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54391746-2640-4af5-98e8-3a23541e4db6 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2b5b99a36442148fe743269054e991', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4faef13aa50e4e36b6c265fb1398fa14', 'project_domain_id': 'default', 'roles': ['member', 'service', 'ResellerAdmin', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:23:34.267579 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54391746-2640-4af5-98e8-3a23541e4db6 service nova] REQUEST_METHOD: `GET` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:23:34.267867 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54391746-2640-4af5-98e8-3a23541e4db6 service nova] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:23:34.268147 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54391746-2640-4af5-98e8-3a23541e4db6 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:23:34.269792 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.fernet_utils [None req-54391746-2640-4af5-98e8-3a23541e4db6 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80244) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 08:23:34.357956 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54391746-2640-4af5-98e8-3a23541e4db6 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:23:34.359602 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54391746-2640-4af5-98e8-3a23541e4db6 service nova] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:23:34.377167 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 712/1419] 10.222.0.47 () {60 vars in 1365 bytes} [Thu Jul 21 08:23:34 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 368 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 08:23:35.269939 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f298710-b6c5-4b80-8342-7584ce0aea8a None None] REQUEST_METHOD: `GET` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:23:35.270509 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f298710-b6c5-4b80-8342-7584ce0aea8a None None] SCRIPT_NAME: `/identity` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:23:35.270509 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f298710-b6c5-4b80-8342-7584ce0aea8a None None] PATH_INFO: `/` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:23:35.272411 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: [pid: 80245|app: 0|req: 708/1420] 127.0.0.1 () {54 vars in 901 bytes} [Thu Jul 21 08:23:35 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 08:23:35.277290 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b628c7d8-6e59-47d5-93dc-83e930a32e99 None None] REQUEST_METHOD: `POST` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:23:35.277711 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b628c7d8-6e59-47d5-93dc-83e930a32e99 None None] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:23:35.277960 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b628c7d8-6e59-47d5-93dc-83e930a32e99 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:23:35.332822 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG dbcounter [-] [80244] Writing DB stats keystone:SELECT=100 {{(pid=80244) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:23:35.357369 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.auth.core [None req-b628c7d8-6e59-47d5-93dc-83e930a32e99 None None] MFA Rules not processed for user `2dce40b8d74c41ec94cc36e6ff05d412`. Rule list: `[]` (Enabled: `True`). {{(pid=80244) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 08:23:35.358629 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.fernet_utils [None req-b628c7d8-6e59-47d5-93dc-83e930a32e99 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80244) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 08:23:35.455847 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 713/1421] 127.0.0.1 () {58 vars in 998 bytes} [Thu Jul 21 08:23:35 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 180 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 08:23:35.612401 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f298710-b6c5-4b80-8342-7584ce0aea8a None None] Authenticating user token {{(pid=80245) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:23:35.614146 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.fernet_utils [None req-3f298710-b6c5-4b80-8342-7584ce0aea8a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80245) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 08:23:35.764303 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f298710-b6c5-4b80-8342-7584ce0aea8a None None] Validating token access rules against request {{(pid=80245) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:23:35.777514 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG dbcounter [-] [80245] Writing DB stats keystone:SELECT=100 {{(pid=80245) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:23:35.878182 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fec2483-aa51-466a-bc35-3ca80469cfe4 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2b5b99a36442148fe743269054e991', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4faef13aa50e4e36b6c265fb1398fa14', 'project_domain_id': 'default', 'roles': ['member', 'service', 'ResellerAdmin', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80245) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:23:35.878742 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fec2483-aa51-466a-bc35-3ca80469cfe4 service nova] REQUEST_METHOD: `GET` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:23:35.879024 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fec2483-aa51-466a-bc35-3ca80469cfe4 service nova] SCRIPT_NAME: `/identity` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:23:35.879299 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fec2483-aa51-466a-bc35-3ca80469cfe4 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:23:35.881453 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.fernet_utils [None req-0fec2483-aa51-466a-bc35-3ca80469cfe4 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80245) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 08:23:35.971215 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fec2483-aa51-466a-bc35-3ca80469cfe4 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:23:35.972840 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fec2483-aa51-466a-bc35-3ca80469cfe4 service nova] RBAC: Authorization granted {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:23:35.991084 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: [pid: 80245|app: 0|req: 709/1422] 10.222.0.47 () {60 vars in 1365 bytes} [Thu Jul 21 08:23:35 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 379 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 08:23:36.851488 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77cbe533-c2ec-4f83-b4e8-755b4ecd01db None None] REQUEST_METHOD: `GET` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:23:36.852018 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77cbe533-c2ec-4f83-b4e8-755b4ecd01db None None] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:23:36.852018 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77cbe533-c2ec-4f83-b4e8-755b4ecd01db None None] PATH_INFO: `/` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:23:36.853311 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 714/1423] 127.0.0.1 () {54 vars in 901 bytes} [Thu Jul 21 08:23:36 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 08:23:36.858432 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfa5329a-6c12-49bb-a947-6c8f37f22aea None None] REQUEST_METHOD: `POST` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:23:36.858874 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfa5329a-6c12-49bb-a947-6c8f37f22aea None None] SCRIPT_NAME: `/identity` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:23:36.858997 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfa5329a-6c12-49bb-a947-6c8f37f22aea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:23:36.877233 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG dbcounter [-] [80245] Writing DB stats keystone:SELECT=100 {{(pid=80245) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:23:36.933861 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.auth.core [None req-cfa5329a-6c12-49bb-a947-6c8f37f22aea None None] MFA Rules not processed for user `2dce40b8d74c41ec94cc36e6ff05d412`. Rule list: `[]` (Enabled: `True`). {{(pid=80245) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 08:23:36.935064 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.fernet_utils [None req-cfa5329a-6c12-49bb-a947-6c8f37f22aea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80245) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 08:23:37.040609 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: [pid: 80245|app: 0|req: 710/1424] 127.0.0.1 () {58 vars in 998 bytes} [Thu Jul 21 08:23:36 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 183 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 08:23:37.190943 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77cbe533-c2ec-4f83-b4e8-755b4ecd01db None None] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:23:37.192527 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.fernet_utils [None req-77cbe533-c2ec-4f83-b4e8-755b4ecd01db None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80244) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 08:23:37.302566 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG dbcounter [-] [80244] Writing DB stats keystone:SELECT=100 {{(pid=80244) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:23:37.344323 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77cbe533-c2ec-4f83-b4e8-755b4ecd01db None None] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:23:37.456798 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4c641a5-46c5-459b-b3b2-8cc162a468f2 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2b5b99a36442148fe743269054e991', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4faef13aa50e4e36b6c265fb1398fa14', 'project_domain_id': 'default', 'roles': ['member', 'service', 'ResellerAdmin', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:23:37.457387 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4c641a5-46c5-459b-b3b2-8cc162a468f2 service nova] REQUEST_METHOD: `GET` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:23:37.457668 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4c641a5-46c5-459b-b3b2-8cc162a468f2 service nova] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:23:37.457944 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4c641a5-46c5-459b-b3b2-8cc162a468f2 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:23:37.459661 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.fernet_utils [None req-f4c641a5-46c5-459b-b3b2-8cc162a468f2 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80244) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 08:23:37.534991 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG dbcounter [-] [80244] Writing DB stats keystone:SELECT=100 {{(pid=80244) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:23:37.549166 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4c641a5-46c5-459b-b3b2-8cc162a468f2 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:23:37.550705 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4c641a5-46c5-459b-b3b2-8cc162a468f2 service nova] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:23:37.569629 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 715/1425] 10.222.0.47 () {60 vars in 1365 bytes} [Thu Jul 21 08:23:37 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 379 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 08:23:38.450822 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d8a5cd7-1123-4bc8-8609-9f44f86a1ff6 None None] REQUEST_METHOD: `GET` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:23:38.451372 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d8a5cd7-1123-4bc8-8609-9f44f86a1ff6 None None] SCRIPT_NAME: `/identity` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:23:38.451499 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d8a5cd7-1123-4bc8-8609-9f44f86a1ff6 None None] PATH_INFO: `/` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:23:38.452794 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: [pid: 80245|app: 0|req: 711/1426] 127.0.0.1 () {54 vars in 901 bytes} [Thu Jul 21 08:23:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 08:23:38.457804 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b307f565-9873-4364-a6ec-1f11dbf83f47 None None] REQUEST_METHOD: `POST` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:23:38.458245 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b307f565-9873-4364-a6ec-1f11dbf83f47 None None] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:23:38.458849 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b307f565-9873-4364-a6ec-1f11dbf83f47 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:23:38.533497 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.auth.core [None req-b307f565-9873-4364-a6ec-1f11dbf83f47 None None] MFA Rules not processed for user `2dce40b8d74c41ec94cc36e6ff05d412`. Rule list: `[]` (Enabled: `True`). {{(pid=80244) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 08:23:38.534714 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.fernet_utils [None req-b307f565-9873-4364-a6ec-1f11dbf83f47 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80244) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 08:23:38.633766 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 716/1427] 127.0.0.1 () {58 vars in 998 bytes} [Thu Jul 21 08:23:38 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 177 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 08:23:38.784851 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d8a5cd7-1123-4bc8-8609-9f44f86a1ff6 None None] Authenticating user token {{(pid=80245) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:23:38.786404 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.fernet_utils [None req-0d8a5cd7-1123-4bc8-8609-9f44f86a1ff6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80245) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 08:23:38.875657 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG dbcounter [-] [80245] Writing DB stats keystone:SELECT=100 {{(pid=80245) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:23:38.948925 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d8a5cd7-1123-4bc8-8609-9f44f86a1ff6 None None] Validating token access rules against request {{(pid=80245) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:23:39.086691 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3c7895e-1ba1-4e4b-b1d1-e185bdcb6604 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2b5b99a36442148fe743269054e991', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4faef13aa50e4e36b6c265fb1398fa14', 'project_domain_id': 'default', 'roles': ['member', 'service', 'ResellerAdmin', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80245) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:23:39.087382 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3c7895e-1ba1-4e4b-b1d1-e185bdcb6604 service nova] REQUEST_METHOD: `GET` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:23:39.087668 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3c7895e-1ba1-4e4b-b1d1-e185bdcb6604 service nova] SCRIPT_NAME: `/identity` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:23:39.087946 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3c7895e-1ba1-4e4b-b1d1-e185bdcb6604 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:23:39.089619 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.fernet_utils [None req-a3c7895e-1ba1-4e4b-b1d1-e185bdcb6604 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80245) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 08:23:39.138286 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG dbcounter [-] [80245] Writing DB stats keystone:SELECT=100 {{(pid=80245) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:23:39.180087 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3c7895e-1ba1-4e4b-b1d1-e185bdcb6604 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:23:39.182173 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3c7895e-1ba1-4e4b-b1d1-e185bdcb6604 service nova] RBAC: Authorization granted {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:23:39.201133 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: [pid: 80245|app: 0|req: 712/1428] 10.222.0.47 () {60 vars in 1365 bytes} [Thu Jul 21 08:23:38 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 417 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 08:23:40.027164 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dad3188-184d-47a6-aead-70710e244ac5 None None] REQUEST_METHOD: `GET` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:23:40.027878 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dad3188-184d-47a6-aead-70710e244ac5 None None] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:23:40.027878 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dad3188-184d-47a6-aead-70710e244ac5 None None] PATH_INFO: `/` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:23:40.029182 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 717/1429] 127.0.0.1 () {54 vars in 901 bytes} [Thu Jul 21 08:23:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 08:23:40.034208 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4316b539-b48a-4d5c-b024-afc6bccd5534 None None] REQUEST_METHOD: `POST` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:23:40.034629 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4316b539-b48a-4d5c-b024-afc6bccd5534 None None] SCRIPT_NAME: `/identity` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:23:40.034757 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4316b539-b48a-4d5c-b024-afc6bccd5534 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:23:40.107343 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.auth.core [None req-4316b539-b48a-4d5c-b024-afc6bccd5534 None None] MFA Rules not processed for user `2dce40b8d74c41ec94cc36e6ff05d412`. Rule list: `[]` (Enabled: `True`). {{(pid=80245) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 08:23:40.108544 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.fernet_utils [None req-4316b539-b48a-4d5c-b024-afc6bccd5534 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80245) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 08:23:40.203135 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: [pid: 80245|app: 0|req: 713/1430] 127.0.0.1 () {58 vars in 998 bytes} [Thu Jul 21 08:23:40 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 08:23:40.358385 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dad3188-184d-47a6-aead-70710e244ac5 None None] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:23:40.359950 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.fernet_utils [None req-5dad3188-184d-47a6-aead-70710e244ac5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80244) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 08:23:40.409629 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG dbcounter [-] [80244] Writing DB stats keystone:SELECT=100 {{(pid=80244) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:23:40.508539 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dad3188-184d-47a6-aead-70710e244ac5 None None] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:23:40.624169 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dca6fc0-89fa-4593-8798-a5abc3d607b1 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2b5b99a36442148fe743269054e991', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4faef13aa50e4e36b6c265fb1398fa14', 'project_domain_id': 'default', 'roles': ['member', 'service', 'ResellerAdmin', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:23:40.624819 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dca6fc0-89fa-4593-8798-a5abc3d607b1 service nova] REQUEST_METHOD: `GET` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:23:40.625104 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dca6fc0-89fa-4593-8798-a5abc3d607b1 service nova] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:23:40.625373 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dca6fc0-89fa-4593-8798-a5abc3d607b1 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:23:40.627032 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.fernet_utils [None req-4dca6fc0-89fa-4593-8798-a5abc3d607b1 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80244) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 08:23:40.634251 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG dbcounter [-] [80244] Writing DB stats keystone:SELECT=100 {{(pid=80244) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:23:40.716444 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dca6fc0-89fa-4593-8798-a5abc3d607b1 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:23:40.718064 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dca6fc0-89fa-4593-8798-a5abc3d607b1 service nova] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:23:40.737867 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 718/1431] 10.222.0.47 () {60 vars in 1365 bytes} [Thu Jul 21 08:23:40 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 380 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 08:23:41.538137 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-badb9cdd-b296-4206-ba16-f72712f8791c None None] REQUEST_METHOD: `GET` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:23:41.538678 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-badb9cdd-b296-4206-ba16-f72712f8791c None None] SCRIPT_NAME: `/identity` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:23:41.538788 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-badb9cdd-b296-4206-ba16-f72712f8791c None None] PATH_INFO: `/` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:23:41.540144 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: [pid: 80245|app: 0|req: 714/1432] 127.0.0.1 () {54 vars in 901 bytes} [Thu Jul 21 08:23:41 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 08:23:41.545094 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-171d60ca-3eda-4eec-bc89-fcf4c2fd1956 None None] REQUEST_METHOD: `POST` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:23:41.545508 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-171d60ca-3eda-4eec-bc89-fcf4c2fd1956 None None] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:23:41.545607 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-171d60ca-3eda-4eec-bc89-fcf4c2fd1956 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:23:41.621093 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.auth.core [None req-171d60ca-3eda-4eec-bc89-fcf4c2fd1956 None None] MFA Rules not processed for user `2dce40b8d74c41ec94cc36e6ff05d412`. Rule list: `[]` (Enabled: `True`). {{(pid=80244) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 08:23:41.622655 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.fernet_utils [None req-171d60ca-3eda-4eec-bc89-fcf4c2fd1956 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80244) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 08:23:41.692290 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG dbcounter [-] [80244] Writing DB stats keystone:SELECT=100 {{(pid=80244) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:23:41.723579 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 719/1433] 127.0.0.1 () {58 vars in 998 bytes} [Thu Jul 21 08:23:41 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 179 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 08:23:41.877398 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-badb9cdd-b296-4206-ba16-f72712f8791c None None] Authenticating user token {{(pid=80245) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:23:41.878940 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.fernet_utils [None req-badb9cdd-b296-4206-ba16-f72712f8791c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80245) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 08:23:41.897888 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG dbcounter [-] [80245] Writing DB stats keystone:SELECT=100 {{(pid=80245) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:23:42.024271 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-badb9cdd-b296-4206-ba16-f72712f8791c None None] Validating token access rules against request {{(pid=80245) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:23:42.111724 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG dbcounter [-] [80245] Writing DB stats keystone:SELECT=100 {{(pid=80245) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:23:42.138800 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25d49d7c-fae5-4ff1-91a3-44b3945e8cb6 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2b5b99a36442148fe743269054e991', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4faef13aa50e4e36b6c265fb1398fa14', 'project_domain_id': 'default', 'roles': ['member', 'service', 'ResellerAdmin', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80245) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:23:42.139483 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25d49d7c-fae5-4ff1-91a3-44b3945e8cb6 service nova] REQUEST_METHOD: `GET` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:23:42.139775 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25d49d7c-fae5-4ff1-91a3-44b3945e8cb6 service nova] SCRIPT_NAME: `/identity` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:23:42.140048 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25d49d7c-fae5-4ff1-91a3-44b3945e8cb6 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:23:42.141694 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.fernet_utils [None req-25d49d7c-fae5-4ff1-91a3-44b3945e8cb6 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80245) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 08:23:42.228514 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25d49d7c-fae5-4ff1-91a3-44b3945e8cb6 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:23:42.230136 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25d49d7c-fae5-4ff1-91a3-44b3945e8cb6 service nova] RBAC: Authorization granted {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:23:42.248480 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: [pid: 80245|app: 0|req: 715/1434] 10.222.0.47 () {60 vars in 1365 bytes} [Thu Jul 21 08:23:41 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 371 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 08:23:43.065609 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f98e6ab5-5ec4-42ea-bb86-79d8d9042281 None None] REQUEST_METHOD: `GET` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:23:43.066181 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f98e6ab5-5ec4-42ea-bb86-79d8d9042281 None None] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:23:43.066181 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f98e6ab5-5ec4-42ea-bb86-79d8d9042281 None None] PATH_INFO: `/` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:23:43.068344 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 720/1435] 127.0.0.1 () {54 vars in 901 bytes} [Thu Jul 21 08:23:43 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 21 08:23:43.073474 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e407d10-71cf-4b56-b478-7635eb32fec9 None None] REQUEST_METHOD: `POST` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:23:43.073907 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e407d10-71cf-4b56-b478-7635eb32fec9 None None] SCRIPT_NAME: `/identity` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:23:43.074033 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e407d10-71cf-4b56-b478-7635eb32fec9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:23:43.147068 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.auth.core [None req-4e407d10-71cf-4b56-b478-7635eb32fec9 None None] MFA Rules not processed for user `2dce40b8d74c41ec94cc36e6ff05d412`. Rule list: `[]` (Enabled: `True`). {{(pid=80245) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 08:23:43.148298 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.fernet_utils [None req-4e407d10-71cf-4b56-b478-7635eb32fec9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80245) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 08:23:43.190018 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG dbcounter [-] [80245] Writing DB stats keystone:SELECT=100 {{(pid=80245) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:23:43.246012 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: [pid: 80245|app: 0|req: 716/1436] 127.0.0.1 () {58 vars in 998 bytes} [Thu Jul 21 08:23:43 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 21 08:23:43.401252 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f98e6ab5-5ec4-42ea-bb86-79d8d9042281 None None] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:23:43.402870 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.fernet_utils [None req-f98e6ab5-5ec4-42ea-bb86-79d8d9042281 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80244) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 08:23:43.550495 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f98e6ab5-5ec4-42ea-bb86-79d8d9042281 None None] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:23:43.606828 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG dbcounter [-] [80244] Writing DB stats keystone:SELECT=100 {{(pid=80244) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:23:43.670484 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca506052-9a55-471a-b503-873a280585d4 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2b5b99a36442148fe743269054e991', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4faef13aa50e4e36b6c265fb1398fa14', 'project_domain_id': 'default', 'roles': ['member', 'service', 'ResellerAdmin', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:23:43.671061 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca506052-9a55-471a-b503-873a280585d4 service nova] REQUEST_METHOD: `GET` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:23:43.671362 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca506052-9a55-471a-b503-873a280585d4 service nova] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:23:43.671646 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca506052-9a55-471a-b503-873a280585d4 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:23:43.673298 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.fernet_utils [None req-ca506052-9a55-471a-b503-873a280585d4 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80244) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 08:23:43.761094 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca506052-9a55-471a-b503-873a280585d4 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:23:43.762693 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca506052-9a55-471a-b503-873a280585d4 service nova] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:23:43.781017 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 721/1437] 10.222.0.47 () {60 vars in 1365 bytes} [Thu Jul 21 08:23:43 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 380 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 08:23:44.586112 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0423b22a-40bd-46fc-92f4-abaa35ed79b3 None None] REQUEST_METHOD: `GET` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:23:44.586724 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0423b22a-40bd-46fc-92f4-abaa35ed79b3 None None] SCRIPT_NAME: `/identity` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:23:44.586724 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0423b22a-40bd-46fc-92f4-abaa35ed79b3 None None] PATH_INFO: `/` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:23:44.588103 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: [pid: 80245|app: 0|req: 717/1438] 127.0.0.1 () {54 vars in 901 bytes} [Thu Jul 21 08:23:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 08:23:44.594211 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14e70a22-226f-4d5c-93d8-b6326a4430e3 None None] REQUEST_METHOD: `POST` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:23:44.594776 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14e70a22-226f-4d5c-93d8-b6326a4430e3 None None] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:23:44.595274 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14e70a22-226f-4d5c-93d8-b6326a4430e3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:23:44.673711 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.auth.core [None req-14e70a22-226f-4d5c-93d8-b6326a4430e3 None None] MFA Rules not processed for user `2dce40b8d74c41ec94cc36e6ff05d412`. Rule list: `[]` (Enabled: `True`). {{(pid=80244) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 08:23:44.674927 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.fernet_utils [None req-14e70a22-226f-4d5c-93d8-b6326a4430e3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80244) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 08:23:44.677576 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG dbcounter [-] [80244] Writing DB stats keystone:SELECT=100 {{(pid=80244) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:23:44.773051 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 722/1439] 127.0.0.1 () {58 vars in 998 bytes} [Thu Jul 21 08:23:44 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 181 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 08:23:44.922241 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0423b22a-40bd-46fc-92f4-abaa35ed79b3 None None] Authenticating user token {{(pid=80245) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:23:44.923791 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.fernet_utils [None req-0423b22a-40bd-46fc-92f4-abaa35ed79b3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80245) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 08:23:45.091193 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0423b22a-40bd-46fc-92f4-abaa35ed79b3 None None] Validating token access rules against request {{(pid=80245) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:23:45.117796 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG dbcounter [-] [80245] Writing DB stats keystone:SELECT=100 {{(pid=80245) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:23:45.206970 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2818ed45-ac44-4e78-8cc7-a682660c0a52 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2b5b99a36442148fe743269054e991', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4faef13aa50e4e36b6c265fb1398fa14', 'project_domain_id': 'default', 'roles': ['member', 'service', 'ResellerAdmin', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80245) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:23:45.207651 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2818ed45-ac44-4e78-8cc7-a682660c0a52 service nova] REQUEST_METHOD: `GET` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:23:45.207942 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2818ed45-ac44-4e78-8cc7-a682660c0a52 service nova] SCRIPT_NAME: `/identity` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:23:45.208701 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2818ed45-ac44-4e78-8cc7-a682660c0a52 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:23:45.210350 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.fernet_utils [None req-2818ed45-ac44-4e78-8cc7-a682660c0a52 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80245) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 08:23:45.306654 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2818ed45-ac44-4e78-8cc7-a682660c0a52 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:23:45.308285 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2818ed45-ac44-4e78-8cc7-a682660c0a52 service nova] RBAC: Authorization granted {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:23:45.330452 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: [pid: 80245|app: 0|req: 718/1440] 10.222.0.47 () {60 vars in 1365 bytes} [Thu Jul 21 08:23:44 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 409 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 08:23:47.272830 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b408af71-a47f-459c-85e2-99c6df7e3eb9 None None] REQUEST_METHOD: `GET` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:23:47.273494 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b408af71-a47f-459c-85e2-99c6df7e3eb9 None None] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:23:47.273494 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b408af71-a47f-459c-85e2-99c6df7e3eb9 None None] PATH_INFO: `/` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:23:47.274833 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 723/1441] 127.0.0.1 () {54 vars in 916 bytes} [Thu Jul 21 08:23:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 08:23:47.280043 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dc92ef0-7807-4c18-b475-8f24d054ba96 None None] REQUEST_METHOD: `POST` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:23:47.280566 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dc92ef0-7807-4c18-b475-8f24d054ba96 None None] SCRIPT_NAME: `/identity` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:23:47.280667 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dc92ef0-7807-4c18-b475-8f24d054ba96 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:23:47.319801 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG dbcounter [-] [80245] Writing DB stats keystone:SELECT=100 {{(pid=80245) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:23:47.346428 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.auth.core [None req-2dc92ef0-7807-4c18-b475-8f24d054ba96 None None] MFA Rules not processed for user `2dce40b8d74c41ec94cc36e6ff05d412`. Rule list: `[]` (Enabled: `True`). {{(pid=80245) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 08:23:47.347672 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.fernet_utils [None req-2dc92ef0-7807-4c18-b475-8f24d054ba96 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80245) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 08:23:47.444556 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: [pid: 80245|app: 0|req: 719/1442] 127.0.0.1 () {58 vars in 1013 bytes} [Thu Jul 21 08:23:47 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 08:23:47.483478 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b408af71-a47f-459c-85e2-99c6df7e3eb9 None None] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:23:47.485020 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.fernet_utils [None req-b408af71-a47f-459c-85e2-99c6df7e3eb9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80244) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 08:23:47.589125 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b408af71-a47f-459c-85e2-99c6df7e3eb9 None None] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:23:47.621377 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG dbcounter [-] [80244] Writing DB stats keystone:SELECT=100 {{(pid=80244) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:23:47.663748 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-527dc5b4-1449-4e2c-aa93-83951dcfd5e4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20ca3c3bd3774691b3374cfbfec6e494', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4faef13aa50e4e36b6c265fb1398fa14', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:23:47.664392 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-527dc5b4-1449-4e2c-aa93-83951dcfd5e4 service neutron] REQUEST_METHOD: `GET` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:23:47.664672 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-527dc5b4-1449-4e2c-aa93-83951dcfd5e4 service neutron] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:23:47.664944 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-527dc5b4-1449-4e2c-aa93-83951dcfd5e4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:23:47.666571 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.fernet_utils [None req-527dc5b4-1449-4e2c-aa93-83951dcfd5e4 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80244) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 08:23:47.772757 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-527dc5b4-1449-4e2c-aa93-83951dcfd5e4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:23:47.774255 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-527dc5b4-1449-4e2c-aa93-83951dcfd5e4 service neutron] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:23:47.793566 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 724/1443] 10.222.0.47 () {60 vars in 1365 bytes} [Thu Jul 21 08:23:47 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 311 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 08:23:49.976103 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-399936c1-8a58-4c8f-b399-7506d78a380d None None] REQUEST_METHOD: `GET` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:23:49.976673 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-399936c1-8a58-4c8f-b399-7506d78a380d None None] SCRIPT_NAME: `/identity` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:23:49.976673 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-399936c1-8a58-4c8f-b399-7506d78a380d None None] PATH_INFO: `/` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:23:49.978030 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: [pid: 80245|app: 0|req: 720/1444] 127.0.0.1 () {54 vars in 916 bytes} [Thu Jul 21 08:23:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 21 08:23:49.983308 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7217d853-d9c6-4607-ad68-5a39cd5deb1b None None] REQUEST_METHOD: `POST` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:23:49.983745 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7217d853-d9c6-4607-ad68-5a39cd5deb1b None None] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:23:49.983876 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7217d853-d9c6-4607-ad68-5a39cd5deb1b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:23:50.048126 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.auth.core [None req-7217d853-d9c6-4607-ad68-5a39cd5deb1b None None] MFA Rules not processed for user `2dce40b8d74c41ec94cc36e6ff05d412`. Rule list: `[]` (Enabled: `True`). {{(pid=80244) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 08:23:50.049347 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.fernet_utils [None req-7217d853-d9c6-4607-ad68-5a39cd5deb1b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80244) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 08:23:50.062649 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG dbcounter [-] [80244] Writing DB stats keystone:SELECT=100 {{(pid=80244) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:23:50.147791 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 725/1445] 127.0.0.1 () {58 vars in 1013 bytes} [Thu Jul 21 08:23:49 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 08:23:50.188250 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-399936c1-8a58-4c8f-b399-7506d78a380d None None] Authenticating user token {{(pid=80245) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:23:50.189785 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.fernet_utils [None req-399936c1-8a58-4c8f-b399-7506d78a380d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80245) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 08:23:50.292552 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-399936c1-8a58-4c8f-b399-7506d78a380d None None] Validating token access rules against request {{(pid=80245) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:23:50.304545 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG dbcounter [-] [80245] Writing DB stats keystone:SELECT=100 {{(pid=80245) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:23:50.368831 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05959818-85a8-43a3-af4e-8850bb637f4b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20ca3c3bd3774691b3374cfbfec6e494', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4faef13aa50e4e36b6c265fb1398fa14', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80245) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:23:50.369490 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05959818-85a8-43a3-af4e-8850bb637f4b service neutron] REQUEST_METHOD: `GET` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:23:50.369770 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05959818-85a8-43a3-af4e-8850bb637f4b service neutron] SCRIPT_NAME: `/identity` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:23:50.370043 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05959818-85a8-43a3-af4e-8850bb637f4b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:23:50.371709 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.fernet_utils [None req-05959818-85a8-43a3-af4e-8850bb637f4b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80245) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 08:23:50.482318 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05959818-85a8-43a3-af4e-8850bb637f4b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:23:50.483819 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05959818-85a8-43a3-af4e-8850bb637f4b service neutron] RBAC: Authorization granted {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:23:50.502863 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: [pid: 80245|app: 0|req: 721/1446] 10.222.0.47 () {60 vars in 1365 bytes} [Thu Jul 21 08:23:50 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 315 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 08:23:52.885703 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c627a80-1294-446c-8aea-fa24b87a7a90 None None] REQUEST_METHOD: `GET` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:23:52.886232 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c627a80-1294-446c-8aea-fa24b87a7a90 None None] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:23:52.886232 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c627a80-1294-446c-8aea-fa24b87a7a90 None None] PATH_INFO: `/` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:23:52.887599 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 726/1447] 127.0.0.1 () {54 vars in 916 bytes} [Thu Jul 21 08:23:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 08:23:52.893106 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96d078cc-7048-4f68-98b7-f71c6d97330e None None] REQUEST_METHOD: `POST` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:23:52.893541 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96d078cc-7048-4f68-98b7-f71c6d97330e None None] SCRIPT_NAME: `/identity` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:23:52.893680 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96d078cc-7048-4f68-98b7-f71c6d97330e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:23:52.940265 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG dbcounter [-] [80245] Writing DB stats keystone:SELECT=100 {{(pid=80245) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:23:52.958805 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.auth.core [None req-96d078cc-7048-4f68-98b7-f71c6d97330e None None] MFA Rules not processed for user `2dce40b8d74c41ec94cc36e6ff05d412`. Rule list: `[]` (Enabled: `True`). {{(pid=80245) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 08:23:52.960045 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.fernet_utils [None req-96d078cc-7048-4f68-98b7-f71c6d97330e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80245) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 08:23:53.056304 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: [pid: 80245|app: 0|req: 722/1448] 127.0.0.1 () {58 vars in 1013 bytes} [Thu Jul 21 08:23:52 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 08:23:53.095202 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c627a80-1294-446c-8aea-fa24b87a7a90 None None] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:23:53.096786 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.fernet_utils [None req-3c627a80-1294-446c-8aea-fa24b87a7a90 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80244) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 08:23:53.322007 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c627a80-1294-446c-8aea-fa24b87a7a90 None None] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:23:53.364721 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG dbcounter [-] [80244] Writing DB stats keystone:SELECT=100 {{(pid=80244) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:23:53.400540 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac9ce067-790c-406b-aba9-c890d0c6ecdd service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20ca3c3bd3774691b3374cfbfec6e494', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4faef13aa50e4e36b6c265fb1398fa14', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:23:53.400746 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac9ce067-790c-406b-aba9-c890d0c6ecdd service neutron] REQUEST_METHOD: `GET` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:23:53.401028 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac9ce067-790c-406b-aba9-c890d0c6ecdd service neutron] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:23:53.401301 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac9ce067-790c-406b-aba9-c890d0c6ecdd service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:23:53.402954 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.fernet_utils [None req-ac9ce067-790c-406b-aba9-c890d0c6ecdd service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80244) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 08:23:53.521585 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac9ce067-790c-406b-aba9-c890d0c6ecdd service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:23:53.523048 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac9ce067-790c-406b-aba9-c890d0c6ecdd service neutron] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:23:53.541725 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 727/1449] 10.222.0.47 () {60 vars in 1365 bytes} [Thu Jul 21 08:23:53 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 447 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 21 08:23:55.714277 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a3945eb-dab4-407b-83ce-1f6e8d3e9504 None None] REQUEST_METHOD: `GET` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:23:55.714783 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a3945eb-dab4-407b-83ce-1f6e8d3e9504 None None] SCRIPT_NAME: `/identity` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:23:55.714783 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a3945eb-dab4-407b-83ce-1f6e8d3e9504 None None] PATH_INFO: `/` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:23:55.716147 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: [pid: 80245|app: 0|req: 723/1450] 127.0.0.1 () {54 vars in 916 bytes} [Thu Jul 21 08:23:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 08:23:55.721275 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-026f410e-4d8e-4665-8581-920bb65a5f33 None None] REQUEST_METHOD: `POST` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:23:55.721682 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-026f410e-4d8e-4665-8581-920bb65a5f33 None None] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:23:55.721828 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-026f410e-4d8e-4665-8581-920bb65a5f33 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:23:55.789635 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.auth.core [None req-026f410e-4d8e-4665-8581-920bb65a5f33 None None] MFA Rules not processed for user `2dce40b8d74c41ec94cc36e6ff05d412`. Rule list: `[]` (Enabled: `True`). {{(pid=80244) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 08:23:55.791283 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.fernet_utils [None req-026f410e-4d8e-4665-8581-920bb65a5f33 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80244) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 08:23:55.817798 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG dbcounter [-] [80244] Writing DB stats keystone:SELECT=100 {{(pid=80244) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:23:55.894017 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 728/1451] 127.0.0.1 () {58 vars in 1013 bytes} [Thu Jul 21 08:23:55 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 08:23:55.933095 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a3945eb-dab4-407b-83ce-1f6e8d3e9504 None None] Authenticating user token {{(pid=80245) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:23:55.934640 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.fernet_utils [None req-5a3945eb-dab4-407b-83ce-1f6e8d3e9504 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80245) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 08:23:56.036338 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a3945eb-dab4-407b-83ce-1f6e8d3e9504 None None] Validating token access rules against request {{(pid=80245) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:23:56.056341 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG dbcounter [-] [80245] Writing DB stats keystone:SELECT=100 {{(pid=80245) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:23:56.119155 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4def1382-72b5-4778-96aa-d2f65c9eecb5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20ca3c3bd3774691b3374cfbfec6e494', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4faef13aa50e4e36b6c265fb1398fa14', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80245) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:23:56.119828 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4def1382-72b5-4778-96aa-d2f65c9eecb5 service neutron] REQUEST_METHOD: `GET` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:23:56.120108 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4def1382-72b5-4778-96aa-d2f65c9eecb5 service neutron] SCRIPT_NAME: `/identity` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:23:56.120380 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4def1382-72b5-4778-96aa-d2f65c9eecb5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:23:56.122006 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.fernet_utils [None req-4def1382-72b5-4778-96aa-d2f65c9eecb5 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80245) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 08:23:56.229332 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4def1382-72b5-4778-96aa-d2f65c9eecb5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:23:56.230749 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4def1382-72b5-4778-96aa-d2f65c9eecb5 service neutron] RBAC: Authorization granted {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:23:56.248609 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: [pid: 80245|app: 0|req: 724/1452] 10.222.0.47 () {60 vars in 1365 bytes} [Thu Jul 21 08:23:55 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 316 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 08:23:58.334133 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99536151-ebbc-4464-a9ec-3fc2e2e179d4 None None] REQUEST_METHOD: `GET` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:23:58.334635 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99536151-ebbc-4464-a9ec-3fc2e2e179d4 None None] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:23:58.334635 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99536151-ebbc-4464-a9ec-3fc2e2e179d4 None None] PATH_INFO: `/` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:23:58.336031 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 729/1453] 127.0.0.1 () {54 vars in 916 bytes} [Thu Jul 21 08:23:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 08:23:58.341417 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4f9c34c-8a8d-4433-ac92-6889f25fb667 None None] REQUEST_METHOD: `POST` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:23:58.341833 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4f9c34c-8a8d-4433-ac92-6889f25fb667 None None] SCRIPT_NAME: `/identity` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:23:58.342019 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4f9c34c-8a8d-4433-ac92-6889f25fb667 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:23:58.409953 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG dbcounter [-] [80245] Writing DB stats keystone:SELECT=100 {{(pid=80245) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:23:58.411954 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.auth.core [None req-f4f9c34c-8a8d-4433-ac92-6889f25fb667 None None] MFA Rules not processed for user `2dce40b8d74c41ec94cc36e6ff05d412`. Rule list: `[]` (Enabled: `True`). {{(pid=80245) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 08:23:58.413784 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.fernet_utils [None req-f4f9c34c-8a8d-4433-ac92-6889f25fb667 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80245) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 08:23:58.513125 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: [pid: 80245|app: 0|req: 725/1454] 127.0.0.1 () {58 vars in 1013 bytes} [Thu Jul 21 08:23:58 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 08:23:58.552583 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99536151-ebbc-4464-a9ec-3fc2e2e179d4 None None] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:23:58.554145 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.fernet_utils [None req-99536151-ebbc-4464-a9ec-3fc2e2e179d4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80244) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 08:23:58.656758 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99536151-ebbc-4464-a9ec-3fc2e2e179d4 None None] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:23:58.704707 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG dbcounter [-] [80244] Writing DB stats keystone:SELECT=100 {{(pid=80244) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:23:58.729890 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a484405d-c8de-42c1-ba91-4a6f6ac01f43 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20ca3c3bd3774691b3374cfbfec6e494', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4faef13aa50e4e36b6c265fb1398fa14', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:23:58.730570 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a484405d-c8de-42c1-ba91-4a6f6ac01f43 service neutron] REQUEST_METHOD: `GET` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:23:58.730850 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a484405d-c8de-42c1-ba91-4a6f6ac01f43 service neutron] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:23:58.731121 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a484405d-c8de-42c1-ba91-4a6f6ac01f43 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:23:58.732802 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.fernet_utils [None req-a484405d-c8de-42c1-ba91-4a6f6ac01f43 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80244) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 08:23:58.848381 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a484405d-c8de-42c1-ba91-4a6f6ac01f43 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:23:58.849820 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a484405d-c8de-42c1-ba91-4a6f6ac01f43 service neutron] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:23:58.868017 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 730/1455] 10.222.0.47 () {60 vars in 1365 bytes} [Thu Jul 21 08:23:58 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 316 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 08:24:01.036858 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32a6f13a-be62-4487-a15a-d2a76bb773e1 None None] REQUEST_METHOD: `GET` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:24:01.037449 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32a6f13a-be62-4487-a15a-d2a76bb773e1 None None] SCRIPT_NAME: `/identity` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:24:01.037449 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32a6f13a-be62-4487-a15a-d2a76bb773e1 None None] PATH_INFO: `/` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:24:01.038765 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: [pid: 80245|app: 0|req: 726/1456] 127.0.0.1 () {54 vars in 916 bytes} [Thu Jul 21 08:24:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 08:24:01.044149 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09925d28-1507-4b43-83d8-57e6018066ed None None] REQUEST_METHOD: `POST` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:24:01.044595 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09925d28-1507-4b43-83d8-57e6018066ed None None] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:24:01.044715 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09925d28-1507-4b43-83d8-57e6018066ed None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:24:01.110729 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.auth.core [None req-09925d28-1507-4b43-83d8-57e6018066ed None None] MFA Rules not processed for user `2dce40b8d74c41ec94cc36e6ff05d412`. Rule list: `[]` (Enabled: `True`). {{(pid=80244) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 08:24:01.112258 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.fernet_utils [None req-09925d28-1507-4b43-83d8-57e6018066ed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80244) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 08:24:01.153494 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG dbcounter [-] [80244] Writing DB stats keystone:SELECT=100 {{(pid=80244) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:24:01.227897 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 731/1457] 127.0.0.1 () {58 vars in 1013 bytes} [Thu Jul 21 08:24:01 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 185 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 08:24:01.266997 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32a6f13a-be62-4487-a15a-d2a76bb773e1 None None] Authenticating user token {{(pid=80245) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:24:01.268640 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.fernet_utils [None req-32a6f13a-be62-4487-a15a-d2a76bb773e1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80245) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 08:24:01.375957 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32a6f13a-be62-4487-a15a-d2a76bb773e1 None None] Validating token access rules against request {{(pid=80245) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:24:01.404972 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG dbcounter [-] [80245] Writing DB stats keystone:SELECT=100 {{(pid=80245) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:24:01.462550 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8a651f9-6c12-4b76-904f-20b8ddc138a1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20ca3c3bd3774691b3374cfbfec6e494', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4faef13aa50e4e36b6c265fb1398fa14', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80245) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:24:01.463209 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8a651f9-6c12-4b76-904f-20b8ddc138a1 service neutron] REQUEST_METHOD: `GET` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:24:01.463524 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8a651f9-6c12-4b76-904f-20b8ddc138a1 service neutron] SCRIPT_NAME: `/identity` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:24:01.463823 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8a651f9-6c12-4b76-904f-20b8ddc138a1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:24:01.465492 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.fernet_utils [None req-b8a651f9-6c12-4b76-904f-20b8ddc138a1 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80245) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 08:24:01.578602 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8a651f9-6c12-4b76-904f-20b8ddc138a1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:24:01.580083 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8a651f9-6c12-4b76-904f-20b8ddc138a1 service neutron] RBAC: Authorization granted {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:24:01.600233 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: [pid: 80245|app: 0|req: 727/1458] 10.222.0.47 () {60 vars in 1365 bytes} [Thu Jul 21 08:24:01 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 334 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 08:24:03.726863 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4230b2a8-da33-4a9f-963a-9691820d1d8c None None] REQUEST_METHOD: `GET` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:24:03.727393 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4230b2a8-da33-4a9f-963a-9691820d1d8c None None] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:24:03.727393 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4230b2a8-da33-4a9f-963a-9691820d1d8c None None] PATH_INFO: `/` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:24:03.728748 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 732/1459] 127.0.0.1 () {54 vars in 916 bytes} [Thu Jul 21 08:24:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 08:24:03.733948 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-307c7266-0a44-40bd-b995-8e6ea9cc4f84 None None] REQUEST_METHOD: `POST` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:24:03.734379 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-307c7266-0a44-40bd-b995-8e6ea9cc4f84 None None] SCRIPT_NAME: `/identity` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:24:03.734496 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-307c7266-0a44-40bd-b995-8e6ea9cc4f84 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:24:03.798903 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.auth.core [None req-307c7266-0a44-40bd-b995-8e6ea9cc4f84 None None] MFA Rules not processed for user `2dce40b8d74c41ec94cc36e6ff05d412`. Rule list: `[]` (Enabled: `True`). {{(pid=80245) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 08:24:03.800153 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.fernet_utils [None req-307c7266-0a44-40bd-b995-8e6ea9cc4f84 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80245) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 08:24:03.808991 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG dbcounter [-] [80245] Writing DB stats keystone:SELECT=100 {{(pid=80245) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:24:03.899734 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: [pid: 80245|app: 0|req: 728/1460] 127.0.0.1 () {58 vars in 1013 bytes} [Thu Jul 21 08:24:03 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 08:24:03.937280 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4230b2a8-da33-4a9f-963a-9691820d1d8c None None] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:24:03.938809 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.fernet_utils [None req-4230b2a8-da33-4a9f-963a-9691820d1d8c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80244) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 08:24:04.040535 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4230b2a8-da33-4a9f-963a-9691820d1d8c None None] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:24:04.094133 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG dbcounter [-] [80244] Writing DB stats keystone:SELECT=100 {{(pid=80244) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:24:04.112332 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f85bb902-6895-49b1-806a-6f45377350a0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20ca3c3bd3774691b3374cfbfec6e494', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4faef13aa50e4e36b6c265fb1398fa14', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:24:04.112970 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f85bb902-6895-49b1-806a-6f45377350a0 service neutron] REQUEST_METHOD: `GET` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:24:04.113249 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f85bb902-6895-49b1-806a-6f45377350a0 service neutron] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:24:04.114060 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f85bb902-6895-49b1-806a-6f45377350a0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:24:04.115716 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.fernet_utils [None req-f85bb902-6895-49b1-806a-6f45377350a0 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80244) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 08:24:04.221009 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f85bb902-6895-49b1-806a-6f45377350a0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:24:04.222440 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f85bb902-6895-49b1-806a-6f45377350a0 service neutron] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:24:04.241397 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 733/1461] 10.222.0.47 () {60 vars in 1365 bytes} [Thu Jul 21 08:24:03 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 305 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 08:24:06.347527 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-694dcff4-17f3-40ad-ba6c-fa73f78b733a None None] REQUEST_METHOD: `GET` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:24:06.348059 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-694dcff4-17f3-40ad-ba6c-fa73f78b733a None None] SCRIPT_NAME: `/identity` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:24:06.348197 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-694dcff4-17f3-40ad-ba6c-fa73f78b733a None None] PATH_INFO: `/` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:24:06.349514 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: [pid: 80245|app: 0|req: 729/1462] 127.0.0.1 () {54 vars in 916 bytes} [Thu Jul 21 08:24:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 08:24:06.354870 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfb31445-1807-4eea-811a-6b383ea71db6 None None] REQUEST_METHOD: `POST` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:24:06.355284 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfb31445-1807-4eea-811a-6b383ea71db6 None None] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:24:06.355821 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfb31445-1807-4eea-811a-6b383ea71db6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:24:06.419688 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.auth.core [None req-bfb31445-1807-4eea-811a-6b383ea71db6 None None] MFA Rules not processed for user `2dce40b8d74c41ec94cc36e6ff05d412`. Rule list: `[]` (Enabled: `True`). {{(pid=80244) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 08:24:06.420903 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.fernet_utils [None req-bfb31445-1807-4eea-811a-6b383ea71db6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80244) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 08:24:06.458968 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG dbcounter [-] [80244] Writing DB stats keystone:SELECT=100 {{(pid=80244) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:24:06.527952 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 734/1463] 127.0.0.1 () {58 vars in 1013 bytes} [Thu Jul 21 08:24:06 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 08:24:06.564845 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-694dcff4-17f3-40ad-ba6c-fa73f78b733a None None] Authenticating user token {{(pid=80245) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:24:06.566864 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.fernet_utils [None req-694dcff4-17f3-40ad-ba6c-fa73f78b733a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80245) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 08:24:06.669244 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-694dcff4-17f3-40ad-ba6c-fa73f78b733a None None] Validating token access rules against request {{(pid=80245) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:24:06.708320 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG dbcounter [-] [80245] Writing DB stats keystone:SELECT=100 {{(pid=80245) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:24:06.746949 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdf7379d-ba2e-4dd2-b166-1dc7c462b8fd service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20ca3c3bd3774691b3374cfbfec6e494', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4faef13aa50e4e36b6c265fb1398fa14', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80245) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:24:06.747641 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdf7379d-ba2e-4dd2-b166-1dc7c462b8fd service neutron] REQUEST_METHOD: `GET` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:24:06.748010 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdf7379d-ba2e-4dd2-b166-1dc7c462b8fd service neutron] SCRIPT_NAME: `/identity` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:24:06.748306 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdf7379d-ba2e-4dd2-b166-1dc7c462b8fd service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:24:06.749886 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.fernet_utils [None req-bdf7379d-ba2e-4dd2-b166-1dc7c462b8fd service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80245) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 08:24:06.858271 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdf7379d-ba2e-4dd2-b166-1dc7c462b8fd service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:24:06.859761 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdf7379d-ba2e-4dd2-b166-1dc7c462b8fd service neutron] RBAC: Authorization granted {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:24:06.878317 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: [pid: 80245|app: 0|req: 730/1464] 10.222.0.47 () {60 vars in 1365 bytes} [Thu Jul 21 08:24:06 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 314 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 08:24:08.985900 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc3b625e-5f40-49c2-a82f-a74f4cf3d1e8 None None] REQUEST_METHOD: `GET` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:24:08.986457 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc3b625e-5f40-49c2-a82f-a74f4cf3d1e8 None None] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:24:08.986457 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc3b625e-5f40-49c2-a82f-a74f4cf3d1e8 None None] PATH_INFO: `/` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:24:08.987865 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 735/1465] 127.0.0.1 () {54 vars in 916 bytes} [Thu Jul 21 08:24:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 08:24:08.993310 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe014125-f4fc-4124-9436-53073df6196d None None] REQUEST_METHOD: `POST` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:24:08.993706 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe014125-f4fc-4124-9436-53073df6196d None None] SCRIPT_NAME: `/identity` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:24:08.993873 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe014125-f4fc-4124-9436-53073df6196d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:24:09.057658 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.auth.core [None req-fe014125-f4fc-4124-9436-53073df6196d None None] MFA Rules not processed for user `2dce40b8d74c41ec94cc36e6ff05d412`. Rule list: `[]` (Enabled: `True`). {{(pid=80245) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 08:24:09.058871 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.fernet_utils [None req-fe014125-f4fc-4124-9436-53073df6196d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80245) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 08:24:09.076665 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG dbcounter [-] [80245] Writing DB stats keystone:SELECT=100 {{(pid=80245) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:24:09.155597 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: [pid: 80245|app: 0|req: 731/1466] 127.0.0.1 () {58 vars in 1013 bytes} [Thu Jul 21 08:24:08 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 08:24:09.195064 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc3b625e-5f40-49c2-a82f-a74f4cf3d1e8 None None] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:24:09.197478 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.fernet_utils [None req-fc3b625e-5f40-49c2-a82f-a74f4cf3d1e8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80244) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 08:24:09.300890 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc3b625e-5f40-49c2-a82f-a74f4cf3d1e8 None None] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:24:09.366988 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG dbcounter [-] [80244] Writing DB stats keystone:SELECT=100 {{(pid=80244) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:24:09.374010 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67a823cd-dd7c-49aa-9aad-55521aee19d0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20ca3c3bd3774691b3374cfbfec6e494', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4faef13aa50e4e36b6c265fb1398fa14', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:24:09.374660 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67a823cd-dd7c-49aa-9aad-55521aee19d0 service neutron] REQUEST_METHOD: `GET` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:24:09.374945 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67a823cd-dd7c-49aa-9aad-55521aee19d0 service neutron] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:24:09.375217 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67a823cd-dd7c-49aa-9aad-55521aee19d0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:24:09.376890 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.fernet_utils [None req-67a823cd-dd7c-49aa-9aad-55521aee19d0 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80244) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 08:24:09.485453 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67a823cd-dd7c-49aa-9aad-55521aee19d0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:24:09.486815 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67a823cd-dd7c-49aa-9aad-55521aee19d0 service neutron] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:24:09.505528 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 736/1467] 10.222.0.47 () {60 vars in 1365 bytes} [Thu Jul 21 08:24:09 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 311 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 08:24:11.582272 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b638f27-c85e-4300-b08a-a46d9b57fb79 None None] REQUEST_METHOD: `GET` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:24:11.582885 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b638f27-c85e-4300-b08a-a46d9b57fb79 None None] SCRIPT_NAME: `/identity` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:24:11.582885 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b638f27-c85e-4300-b08a-a46d9b57fb79 None None] PATH_INFO: `/` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:24:11.584251 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: [pid: 80245|app: 0|req: 732/1468] 127.0.0.1 () {54 vars in 916 bytes} [Thu Jul 21 08:24:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 08:24:11.589665 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0790e5bc-d184-4765-b6c1-cb685ea8b00d None None] REQUEST_METHOD: `POST` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:24:11.590093 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0790e5bc-d184-4765-b6c1-cb685ea8b00d None None] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:24:11.590222 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0790e5bc-d184-4765-b6c1-cb685ea8b00d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:24:11.654725 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.auth.core [None req-0790e5bc-d184-4765-b6c1-cb685ea8b00d None None] MFA Rules not processed for user `2dce40b8d74c41ec94cc36e6ff05d412`. Rule list: `[]` (Enabled: `True`). {{(pid=80244) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 08:24:11.655981 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.fernet_utils [None req-0790e5bc-d184-4765-b6c1-cb685ea8b00d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80244) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 08:24:11.707365 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG dbcounter [-] [80244] Writing DB stats keystone:SELECT=100 {{(pid=80244) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:24:11.762473 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 737/1469] 127.0.0.1 () {58 vars in 1013 bytes} [Thu Jul 21 08:24:11 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 08:24:11.800139 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b638f27-c85e-4300-b08a-a46d9b57fb79 None None] Authenticating user token {{(pid=80245) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:24:11.802371 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.fernet_utils [None req-7b638f27-c85e-4300-b08a-a46d9b57fb79 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80245) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 08:24:11.906776 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b638f27-c85e-4300-b08a-a46d9b57fb79 None None] Validating token access rules against request {{(pid=80245) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:24:11.954267 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG dbcounter [-] [80245] Writing DB stats keystone:SELECT=100 {{(pid=80245) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:24:11.981444 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bf9de3a-704f-4d30-a81c-1d2f36d3679a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20ca3c3bd3774691b3374cfbfec6e494', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4faef13aa50e4e36b6c265fb1398fa14', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80245) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:24:11.982102 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bf9de3a-704f-4d30-a81c-1d2f36d3679a service neutron] REQUEST_METHOD: `GET` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:24:11.982387 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bf9de3a-704f-4d30-a81c-1d2f36d3679a service neutron] SCRIPT_NAME: `/identity` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:24:11.982665 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bf9de3a-704f-4d30-a81c-1d2f36d3679a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:24:11.984345 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.fernet_utils [None req-4bf9de3a-704f-4d30-a81c-1d2f36d3679a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80245) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 08:24:12.091908 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bf9de3a-704f-4d30-a81c-1d2f36d3679a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:24:12.093346 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bf9de3a-704f-4d30-a81c-1d2f36d3679a service neutron] RBAC: Authorization granted {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:24:12.111639 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: [pid: 80245|app: 0|req: 733/1470] 10.222.0.47 () {60 vars in 1365 bytes} [Thu Jul 21 08:24:11 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 312 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 08:24:21.759917 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG dbcounter [-] [80244] Writing DB stats keystone:SELECT=22 {{(pid=80244) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:24:22.109287 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG dbcounter [-] [80245] Writing DB stats keystone:SELECT=66 {{(pid=80245) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:24:32.000441 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bce75fab-9c01-4e61-aaeb-5b10d659567a None None] REQUEST_METHOD: `GET` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:24:32.000944 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bce75fab-9c01-4e61-aaeb-5b10d659567a None None] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:24:32.000944 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bce75fab-9c01-4e61-aaeb-5b10d659567a None None] PATH_INFO: `/` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:24:32.002344 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 738/1471] 10.222.0.47 () {54 vars in 926 bytes} [Thu Jul 21 08:24:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 08:24:32.008317 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95f5cc76-e2d8-418c-84c7-6970fad71378 None None] REQUEST_METHOD: `POST` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:24:32.008724 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95f5cc76-e2d8-418c-84c7-6970fad71378 None None] SCRIPT_NAME: `/identity` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:24:32.008881 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95f5cc76-e2d8-418c-84c7-6970fad71378 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:24:32.073110 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.auth.core [None req-95f5cc76-e2d8-418c-84c7-6970fad71378 None None] MFA Rules not processed for user `2dce40b8d74c41ec94cc36e6ff05d412`. Rule list: `[]` (Enabled: `True`). {{(pid=80245) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 08:24:32.074317 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.fernet_utils [None req-95f5cc76-e2d8-418c-84c7-6970fad71378 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80245) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 08:24:32.181032 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: [pid: 80245|app: 0|req: 734/1472] 10.222.0.47 () {58 vars in 1023 bytes} [Thu Jul 21 08:24:32 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 08:24:32.266494 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54f8e0d6-2a17-47e9-80dc-148a9700b1de None None] REQUEST_METHOD: `GET` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:24:32.266778 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54f8e0d6-2a17-47e9-80dc-148a9700b1de None None] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:24:32.267047 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54f8e0d6-2a17-47e9-80dc-148a9700b1de None None] PATH_INFO: `/` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:24:32.268427 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 739/1473] 10.222.0.47 () {54 vars in 966 bytes} [Thu Jul 21 08:24:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 08:24:32.273507 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c1a8bbe-3b53-44d3-9bf2-49f511a612b9 None None] REQUEST_METHOD: `POST` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:24:32.273803 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c1a8bbe-3b53-44d3-9bf2-49f511a612b9 None None] SCRIPT_NAME: `/identity` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:24:32.274083 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c1a8bbe-3b53-44d3-9bf2-49f511a612b9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:24:32.349838 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.auth.core [None req-2c1a8bbe-3b53-44d3-9bf2-49f511a612b9 None None] MFA Rules not processed for user `9b0524aed8c74abd8949b860d6b93936`. Rule list: `[]` (Enabled: `True`). {{(pid=80245) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 08:24:32.351048 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.fernet_utils [None req-2c1a8bbe-3b53-44d3-9bf2-49f511a612b9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80245) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 08:24:32.372050 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG dbcounter [-] [80245] Writing DB stats keystone:SELECT=100 {{(pid=80245) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:24:32.429285 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: [pid: 80245|app: 0|req: 735/1474] 10.222.0.47 () {58 vars in 1063 bytes} [Thu Jul 21 08:24:32 2022] POST /identity/v3/auth/tokens => generated 3110 bytes in 157 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 08:24:32.437441 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54f8e0d6-2a17-47e9-80dc-148a9700b1de None None] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:24:32.438986 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.fernet_utils [None req-54f8e0d6-2a17-47e9-80dc-148a9700b1de None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80244) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 08:24:32.543652 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54f8e0d6-2a17-47e9-80dc-148a9700b1de None None] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:24:32.615779 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd12f4dc-b206-4a2a-b1b2-96d8ce14af6a service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9b0524aed8c74abd8949b860d6b93936', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4faef13aa50e4e36b6c265fb1398fa14', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:24:32.616425 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd12f4dc-b206-4a2a-b1b2-96d8ce14af6a service glance] REQUEST_METHOD: `GET` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:24:32.616708 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd12f4dc-b206-4a2a-b1b2-96d8ce14af6a service glance] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:24:32.616982 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd12f4dc-b206-4a2a-b1b2-96d8ce14af6a service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:24:32.618627 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.fernet_utils [None req-dd12f4dc-b206-4a2a-b1b2-96d8ce14af6a service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80244) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 08:24:32.673245 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG dbcounter [-] [80244] Writing DB stats keystone:SELECT=100 {{(pid=80244) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:24:32.727419 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd12f4dc-b206-4a2a-b1b2-96d8ce14af6a service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:24:32.728859 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd12f4dc-b206-4a2a-b1b2-96d8ce14af6a service glance] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:24:32.747011 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 740/1475] 10.222.0.47 () {60 vars in 1365 bytes} [Thu Jul 21 08:24:32 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 310 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 08:24:34.888798 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce3935fa-e7dc-49eb-8e89-c91d64e59c09 None None] REQUEST_METHOD: `GET` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:24:34.889321 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce3935fa-e7dc-49eb-8e89-c91d64e59c09 None None] SCRIPT_NAME: `/identity` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:24:34.889321 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce3935fa-e7dc-49eb-8e89-c91d64e59c09 None None] PATH_INFO: `/` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:24:34.890665 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: [pid: 80245|app: 0|req: 736/1476] 10.222.0.47 () {54 vars in 926 bytes} [Thu Jul 21 08:24:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 08:24:34.896470 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf3675bb-9fac-4e13-974a-14dae7bd1fe2 None None] REQUEST_METHOD: `POST` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:24:34.896934 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf3675bb-9fac-4e13-974a-14dae7bd1fe2 None None] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:24:34.897039 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf3675bb-9fac-4e13-974a-14dae7bd1fe2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:24:34.961477 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.auth.core [None req-bf3675bb-9fac-4e13-974a-14dae7bd1fe2 None None] MFA Rules not processed for user `2dce40b8d74c41ec94cc36e6ff05d412`. Rule list: `[]` (Enabled: `True`). {{(pid=80244) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 08:24:34.962695 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.fernet_utils [None req-bf3675bb-9fac-4e13-974a-14dae7bd1fe2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80244) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 08:24:35.059530 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 741/1477] 10.222.0.47 () {58 vars in 1023 bytes} [Thu Jul 21 08:24:34 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 08:24:35.082861 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce3935fa-e7dc-49eb-8e89-c91d64e59c09 None None] Authenticating user token {{(pid=80245) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:24:35.084435 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.fernet_utils [None req-ce3935fa-e7dc-49eb-8e89-c91d64e59c09 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80245) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 08:24:35.212321 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce3935fa-e7dc-49eb-8e89-c91d64e59c09 None None] Validating token access rules against request {{(pid=80245) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:24:35.261067 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG dbcounter [-] [80245] Writing DB stats keystone:SELECT=100 {{(pid=80245) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:24:35.307032 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1d32a8a-05e6-46fa-80b0-8b81230134e5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '42d15fc0515945acacbbffcf2356c7a5', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80245) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:24:35.307758 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1d32a8a-05e6-46fa-80b0-8b81230134e5 admin admin] REQUEST_METHOD: `GET` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:24:35.308046 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1d32a8a-05e6-46fa-80b0-8b81230134e5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:24:35.308324 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1d32a8a-05e6-46fa-80b0-8b81230134e5 admin admin] PATH_INFO: `/v3/projects` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:24:35.309065 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1d32a8a-05e6-46fa-80b0-8b81230134e5 admin admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:24:35.310850 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: WARNING py.warnings [None req-b1d32a8a-05e6-46fa-80b0-8b81230134e5 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_projects": "(role:reader and system_scope:all) or (role:reader and domain_id:%(target.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jul 21 08:24:35.310850 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: warnings.warn(msg) Jul 21 08:24:35.310850 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: Jul 21 08:24:35.312117 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1d32a8a-05e6-46fa-80b0-8b81230134e5 admin admin] RBAC: Authorization granted {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:24:35.332789 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: [pid: 80245|app: 0|req: 737/1478] 10.222.0.47 () {56 vars in 1108 bytes} [Thu Jul 21 08:24:35 2022] GET /identity/v3/projects => generated 2553 bytes in 250 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 08:24:37.410490 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9676ca4-14de-4419-a79b-fdeb492c754e None None] REQUEST_METHOD: `GET` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:24:37.411061 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9676ca4-14de-4419-a79b-fdeb492c754e None None] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:24:37.411061 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9676ca4-14de-4419-a79b-fdeb492c754e None None] PATH_INFO: `/` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:24:37.412433 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 742/1479] 10.222.0.47 () {54 vars in 926 bytes} [Thu Jul 21 08:24:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 08:24:37.417716 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e45ddfb-39d1-4c68-ae8d-640595e64c2f None None] REQUEST_METHOD: `POST` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:24:37.418124 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e45ddfb-39d1-4c68-ae8d-640595e64c2f None None] SCRIPT_NAME: `/identity` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:24:37.418322 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e45ddfb-39d1-4c68-ae8d-640595e64c2f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:24:37.482237 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.auth.core [None req-1e45ddfb-39d1-4c68-ae8d-640595e64c2f None None] MFA Rules not processed for user `2dce40b8d74c41ec94cc36e6ff05d412`. Rule list: `[]` (Enabled: `True`). {{(pid=80245) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 08:24:37.483476 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.fernet_utils [None req-1e45ddfb-39d1-4c68-ae8d-640595e64c2f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80245) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 08:24:37.578899 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: [pid: 80245|app: 0|req: 738/1480] 10.222.0.47 () {58 vars in 1023 bytes} [Thu Jul 21 08:24:37 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 08:24:37.621641 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9676ca4-14de-4419-a79b-fdeb492c754e None None] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:24:37.623347 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.fernet_utils [None req-e9676ca4-14de-4419-a79b-fdeb492c754e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80244) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 08:24:37.634197 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG dbcounter [-] [80244] Writing DB stats keystone:SELECT=100 {{(pid=80244) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:24:37.770575 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9676ca4-14de-4419-a79b-fdeb492c754e None None] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:24:37.850183 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG dbcounter [-] [80244] Writing DB stats keystone:SELECT=100 {{(pid=80244) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:24:37.885688 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-627bb30c-c1ac-455e-9ef7-2c87ef43e197 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2b5b99a36442148fe743269054e991', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4faef13aa50e4e36b6c265fb1398fa14', 'project_domain_id': 'default', 'roles': ['member', 'service', 'ResellerAdmin', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:24:37.886339 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-627bb30c-c1ac-455e-9ef7-2c87ef43e197 service nova] REQUEST_METHOD: `GET` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:24:37.886617 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-627bb30c-c1ac-455e-9ef7-2c87ef43e197 service nova] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:24:37.886887 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-627bb30c-c1ac-455e-9ef7-2c87ef43e197 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:24:37.888581 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.fernet_utils [None req-627bb30c-c1ac-455e-9ef7-2c87ef43e197 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80244) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 08:24:37.986537 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-627bb30c-c1ac-455e-9ef7-2c87ef43e197 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:24:37.988816 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-627bb30c-c1ac-455e-9ef7-2c87ef43e197 service nova] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:24:38.006885 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 743/1481] 10.222.0.47 () {60 vars in 1365 bytes} [Thu Jul 21 08:24:37 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 386 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 08:24:40.369954 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f6d3494-7316-4360-8f4e-c1cfc7866613 None None] REQUEST_METHOD: `GET` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:24:40.370615 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f6d3494-7316-4360-8f4e-c1cfc7866613 None None] SCRIPT_NAME: `/identity` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:24:40.370615 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f6d3494-7316-4360-8f4e-c1cfc7866613 None None] PATH_INFO: `/` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:24:40.371910 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: [pid: 80245|app: 0|req: 739/1482] 10.222.0.47 () {54 vars in 926 bytes} [Thu Jul 21 08:24:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 08:24:40.377354 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-535f9dfe-2d96-4e29-9d35-07c85be1d457 None None] REQUEST_METHOD: `POST` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:24:40.377807 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-535f9dfe-2d96-4e29-9d35-07c85be1d457 None None] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:24:40.378156 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-535f9dfe-2d96-4e29-9d35-07c85be1d457 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:24:40.442727 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.auth.core [None req-535f9dfe-2d96-4e29-9d35-07c85be1d457 None None] MFA Rules not processed for user `2dce40b8d74c41ec94cc36e6ff05d412`. Rule list: `[]` (Enabled: `True`). {{(pid=80244) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 08:24:40.443954 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.fernet_utils [None req-535f9dfe-2d96-4e29-9d35-07c85be1d457 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80244) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 08:24:40.489854 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG dbcounter [-] [80244] Writing DB stats keystone:SELECT=100 {{(pid=80244) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:24:40.541041 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 744/1483] 10.222.0.47 () {58 vars in 1023 bytes} [Thu Jul 21 08:24:40 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 08:24:40.578349 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f6d3494-7316-4360-8f4e-c1cfc7866613 None None] Authenticating user token {{(pid=80245) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:24:40.580003 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.fernet_utils [None req-0f6d3494-7316-4360-8f4e-c1cfc7866613 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80245) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 08:24:40.603037 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG dbcounter [-] [80245] Writing DB stats keystone:SELECT=100 {{(pid=80245) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:24:40.684442 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f6d3494-7316-4360-8f4e-c1cfc7866613 None None] Validating token access rules against request {{(pid=80245) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:24:40.877473 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dd2e62f-c810-4c27-aa64-d4ebd168f6a1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20ca3c3bd3774691b3374cfbfec6e494', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4faef13aa50e4e36b6c265fb1398fa14', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80245) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:24:40.878131 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dd2e62f-c810-4c27-aa64-d4ebd168f6a1 service neutron] REQUEST_METHOD: `GET` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:24:40.878420 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dd2e62f-c810-4c27-aa64-d4ebd168f6a1 service neutron] SCRIPT_NAME: `/identity` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:24:40.878726 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dd2e62f-c810-4c27-aa64-d4ebd168f6a1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:24:40.880402 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.fernet_utils [None req-0dd2e62f-c810-4c27-aa64-d4ebd168f6a1 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80245) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 08:24:40.953950 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG dbcounter [-] [80245] Writing DB stats keystone:SELECT=100 {{(pid=80245) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:24:40.992819 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dd2e62f-c810-4c27-aa64-d4ebd168f6a1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:24:40.994261 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dd2e62f-c810-4c27-aa64-d4ebd168f6a1 service neutron] RBAC: Authorization granted {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:24:41.012499 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: [pid: 80245|app: 0|req: 740/1484] 10.222.0.47 () {60 vars in 1365 bytes} [Thu Jul 21 08:24:40 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 435 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 08:24:43.132119 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3757f1d-8f71-440d-9964-c2d65a46b2c1 None None] REQUEST_METHOD: `GET` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:24:43.132619 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3757f1d-8f71-440d-9964-c2d65a46b2c1 None None] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:24:43.132619 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3757f1d-8f71-440d-9964-c2d65a46b2c1 None None] PATH_INFO: `/` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:24:43.133990 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 745/1485] 10.222.0.47 () {54 vars in 926 bytes} [Thu Jul 21 08:24:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 08:24:43.139456 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4725baa-51c3-4a08-9b5d-f370c942f751 None None] REQUEST_METHOD: `POST` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:24:43.139852 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4725baa-51c3-4a08-9b5d-f370c942f751 None None] SCRIPT_NAME: `/identity` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:24:43.140011 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4725baa-51c3-4a08-9b5d-f370c942f751 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:24:43.204075 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.auth.core [None req-a4725baa-51c3-4a08-9b5d-f370c942f751 None None] MFA Rules not processed for user `2dce40b8d74c41ec94cc36e6ff05d412`. Rule list: `[]` (Enabled: `True`). {{(pid=80245) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 08:24:43.205273 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.fernet_utils [None req-a4725baa-51c3-4a08-9b5d-f370c942f751 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80245) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 08:24:43.301173 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: [pid: 80245|app: 0|req: 741/1486] 10.222.0.47 () {58 vars in 1023 bytes} [Thu Jul 21 08:24:43 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 08:24:43.339435 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3757f1d-8f71-440d-9964-c2d65a46b2c1 None None] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:24:43.340995 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.fernet_utils [None req-b3757f1d-8f71-440d-9964-c2d65a46b2c1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80244) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 08:24:43.442654 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3757f1d-8f71-440d-9964-c2d65a46b2c1 None None] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:24:43.516387 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG dbcounter [-] [80244] Writing DB stats keystone:SELECT=100 {{(pid=80244) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:24:43.518290 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1da84f09-f619-4efc-8a64-ee6203be5d3e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20ca3c3bd3774691b3374cfbfec6e494', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4faef13aa50e4e36b6c265fb1398fa14', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:24:43.518958 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1da84f09-f619-4efc-8a64-ee6203be5d3e service neutron] REQUEST_METHOD: `GET` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:24:43.519953 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1da84f09-f619-4efc-8a64-ee6203be5d3e service neutron] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:24:43.520439 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1da84f09-f619-4efc-8a64-ee6203be5d3e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:24:43.522328 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.fernet_utils [None req-1da84f09-f619-4efc-8a64-ee6203be5d3e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80244) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 08:24:43.630052 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1da84f09-f619-4efc-8a64-ee6203be5d3e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:24:43.631781 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1da84f09-f619-4efc-8a64-ee6203be5d3e service neutron] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:24:43.650431 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 746/1487] 10.222.0.47 () {60 vars in 1365 bytes} [Thu Jul 21 08:24:43 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 312 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 08:24:45.797680 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5796c457-fbd6-414c-8886-0dec41ab85e1 None None] REQUEST_METHOD: `GET` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:24:45.798223 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5796c457-fbd6-414c-8886-0dec41ab85e1 None None] SCRIPT_NAME: `/identity` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:24:45.798223 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5796c457-fbd6-414c-8886-0dec41ab85e1 None None] PATH_INFO: `/` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:24:45.799565 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: [pid: 80245|app: 0|req: 742/1488] 10.222.0.47 () {54 vars in 926 bytes} [Thu Jul 21 08:24:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 08:24:45.805381 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bdab67a-d6c5-40de-8d65-0a1431392351 None None] REQUEST_METHOD: `POST` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:24:45.805889 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bdab67a-d6c5-40de-8d65-0a1431392351 None None] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:24:45.806000 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bdab67a-d6c5-40de-8d65-0a1431392351 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:24:45.875516 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.auth.core [None req-5bdab67a-d6c5-40de-8d65-0a1431392351 None None] MFA Rules not processed for user `2dce40b8d74c41ec94cc36e6ff05d412`. Rule list: `[]` (Enabled: `True`). {{(pid=80244) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 08:24:45.876731 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.fernet_utils [None req-5bdab67a-d6c5-40de-8d65-0a1431392351 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80244) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 08:24:45.933628 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG dbcounter [-] [80244] Writing DB stats keystone:SELECT=100 {{(pid=80244) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:24:45.978297 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 747/1489] 10.222.0.47 () {58 vars in 1023 bytes} [Thu Jul 21 08:24:45 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 08:24:46.018837 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5796c457-fbd6-414c-8886-0dec41ab85e1 None None] Authenticating user token {{(pid=80245) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:24:46.020871 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.fernet_utils [None req-5796c457-fbd6-414c-8886-0dec41ab85e1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80245) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 08:24:46.056497 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG dbcounter [-] [80245] Writing DB stats keystone:SELECT=100 {{(pid=80245) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:24:46.139084 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5796c457-fbd6-414c-8886-0dec41ab85e1 None None] Validating token access rules against request {{(pid=80245) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:24:46.213545 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-723384d4-bf29-420e-a31d-4147c1356f95 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20ca3c3bd3774691b3374cfbfec6e494', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4faef13aa50e4e36b6c265fb1398fa14', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80245) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:24:46.214201 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-723384d4-bf29-420e-a31d-4147c1356f95 service neutron] REQUEST_METHOD: `GET` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:24:46.214479 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-723384d4-bf29-420e-a31d-4147c1356f95 service neutron] SCRIPT_NAME: `/identity` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:24:46.214760 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-723384d4-bf29-420e-a31d-4147c1356f95 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:24:46.216422 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.fernet_utils [None req-723384d4-bf29-420e-a31d-4147c1356f95 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80245) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 08:24:46.293137 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG dbcounter [-] [80245] Writing DB stats keystone:SELECT=100 {{(pid=80245) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:24:46.321968 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-723384d4-bf29-420e-a31d-4147c1356f95 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:24:46.323296 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-723384d4-bf29-420e-a31d-4147c1356f95 service neutron] RBAC: Authorization granted {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:24:46.342161 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: [pid: 80245|app: 0|req: 743/1490] 10.222.0.47 () {60 vars in 1365 bytes} [Thu Jul 21 08:24:46 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 324 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 08:24:48.661791 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c14cac4-d1f1-4c39-b6fd-34046a0c19d2 None None] REQUEST_METHOD: `GET` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:24:48.662366 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c14cac4-d1f1-4c39-b6fd-34046a0c19d2 None None] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:24:48.662366 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c14cac4-d1f1-4c39-b6fd-34046a0c19d2 None None] PATH_INFO: `/` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:24:48.663794 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 748/1491] 10.222.0.47 () {54 vars in 926 bytes} [Thu Jul 21 08:24:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 08:24:48.669080 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-378e216d-efee-4298-8e3e-15457d9f8487 None None] REQUEST_METHOD: `POST` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:24:48.670137 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-378e216d-efee-4298-8e3e-15457d9f8487 None None] SCRIPT_NAME: `/identity` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:24:48.670137 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-378e216d-efee-4298-8e3e-15457d9f8487 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:24:48.734205 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.auth.core [None req-378e216d-efee-4298-8e3e-15457d9f8487 None None] MFA Rules not processed for user `2dce40b8d74c41ec94cc36e6ff05d412`. Rule list: `[]` (Enabled: `True`). {{(pid=80245) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 08:24:48.735452 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.fernet_utils [None req-378e216d-efee-4298-8e3e-15457d9f8487 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80245) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 08:24:48.835007 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: [pid: 80245|app: 0|req: 744/1492] 10.222.0.47 () {58 vars in 1023 bytes} [Thu Jul 21 08:24:48 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 08:24:48.874787 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c14cac4-d1f1-4c39-b6fd-34046a0c19d2 None None] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:24:48.876381 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.fernet_utils [None req-2c14cac4-d1f1-4c39-b6fd-34046a0c19d2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80244) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 08:24:48.980768 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c14cac4-d1f1-4c39-b6fd-34046a0c19d2 None None] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:24:49.051691 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e85719b4-056a-421b-ae82-1925f894207a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20ca3c3bd3774691b3374cfbfec6e494', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4faef13aa50e4e36b6c265fb1398fa14', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:24:49.052329 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e85719b4-056a-421b-ae82-1925f894207a service neutron] REQUEST_METHOD: `GET` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:24:49.052607 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e85719b4-056a-421b-ae82-1925f894207a service neutron] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:24:49.052879 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e85719b4-056a-421b-ae82-1925f894207a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:24:49.054505 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.fernet_utils [None req-e85719b4-056a-421b-ae82-1925f894207a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80244) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 08:24:49.063293 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG dbcounter [-] [80244] Writing DB stats keystone:SELECT=100 {{(pid=80244) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:24:49.161420 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e85719b4-056a-421b-ae82-1925f894207a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:24:49.162892 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e85719b4-056a-421b-ae82-1925f894207a service neutron] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:24:49.180797 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 749/1493] 10.222.0.47 () {60 vars in 1365 bytes} [Thu Jul 21 08:24:48 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 306 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 08:24:58.833131 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG dbcounter [-] [80245] Writing DB stats keystone:SELECT=82 {{(pid=80245) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:24:59.178494 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG dbcounter [-] [80244] Writing DB stats keystone:SELECT=50 {{(pid=80244) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:26:20.567188 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d49dae0-68f1-4c36-8098-1703059ed30a None None] REQUEST_METHOD: `POST` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:26:20.567787 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d49dae0-68f1-4c36-8098-1703059ed30a None None] SCRIPT_NAME: `/identity` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:26:20.567787 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d49dae0-68f1-4c36-8098-1703059ed30a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:26:20.641495 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.auth.core [None req-5d49dae0-68f1-4c36-8098-1703059ed30a None None] MFA Rules not processed for user `2dce40b8d74c41ec94cc36e6ff05d412`. Rule list: `[]` (Enabled: `True`). {{(pid=80245) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 08:26:20.642734 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.fernet_utils [None req-5d49dae0-68f1-4c36-8098-1703059ed30a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80245) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 08:26:20.745921 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: [pid: 80245|app: 0|req: 745/1494] 10.222.0.47 () {58 vars in 957 bytes} [Thu Jul 21 08:26:20 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 180 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 08:26:20.753775 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54fa567a-b4ad-4695-9cec-be4c7c756f06 None None] REQUEST_METHOD: `POST` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:26:20.754270 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54fa567a-b4ad-4695-9cec-be4c7c756f06 None None] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:26:20.754375 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54fa567a-b4ad-4695-9cec-be4c7c756f06 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:26:20.829387 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.auth.core [None req-54fa567a-b4ad-4695-9cec-be4c7c756f06 None None] MFA Rules not processed for user `2dce40b8d74c41ec94cc36e6ff05d412`. Rule list: `[]` (Enabled: `True`). {{(pid=80244) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 08:26:20.830693 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.fernet_utils [None req-54fa567a-b4ad-4695-9cec-be4c7c756f06 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80244) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 08:26:20.926715 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 750/1495] 10.222.0.47 () {58 vars in 957 bytes} [Thu Jul 21 08:26:20 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 08:26:20.931305 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d49dae0-68f1-4c36-8098-1703059ed30a None None] Authenticating user token {{(pid=80245) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:26:20.932599 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.fernet_utils [None req-5d49dae0-68f1-4c36-8098-1703059ed30a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80245) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 08:26:20.943880 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d49dae0-68f1-4c36-8098-1703059ed30a None None] Validating token access rules against request {{(pid=80245) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:26:21.021113 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG dbcounter [-] [80245] Writing DB stats keystone:SELECT=100 {{(pid=80245) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:26:21.054727 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7202d89-45f5-46e8-b266-0d7027ca3630 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '42d15fc0515945acacbbffcf2356c7a5', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80245) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:26:21.055312 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7202d89-45f5-46e8-b266-0d7027ca3630 admin admin] REQUEST_METHOD: `GET` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:26:21.055630 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7202d89-45f5-46e8-b266-0d7027ca3630 admin admin] SCRIPT_NAME: `/identity` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:26:21.055906 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7202d89-45f5-46e8-b266-0d7027ca3630 admin admin] PATH_INFO: `/v3/domains` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:26:21.056629 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7202d89-45f5-46e8-b266-0d7027ca3630 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80245) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 08:26:21.056935 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7202d89-45f5-46e8-b266-0d7027ca3630 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:26:21.057947 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: WARNING py.warnings [None req-f7202d89-45f5-46e8-b266-0d7027ca3630 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_domains": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Jul 21 08:26:21.057947 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: warnings.warn(msg) Jul 21 08:26:21.057947 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: Jul 21 08:26:21.058841 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7202d89-45f5-46e8-b266-0d7027ca3630 admin admin] RBAC: Authorization granted {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:26:21.069939 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: [pid: 80245|app: 0|req: 746/1496] 10.222.0.47 () {58 vars in 1152 bytes} [Thu Jul 21 08:26:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 08:26:21.078674 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54fa567a-b4ad-4695-9cec-be4c7c756f06 None None] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:26:21.087702 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54fa567a-b4ad-4695-9cec-be4c7c756f06 None None] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:26:21.149370 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG dbcounter [-] [80244] Writing DB stats keystone:SELECT=100 {{(pid=80244) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:26:21.183258 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc05dda3-9cc2-4d7a-8a3c-4b6bc98e70f8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '42d15fc0515945acacbbffcf2356c7a5', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:26:21.183954 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc05dda3-9cc2-4d7a-8a3c-4b6bc98e70f8 admin admin] REQUEST_METHOD: `POST` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:26:21.184237 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc05dda3-9cc2-4d7a-8a3c-4b6bc98e70f8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:26:21.184509 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc05dda3-9cc2-4d7a-8a3c-4b6bc98e70f8 admin admin] PATH_INFO: `/v3/projects` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:26:21.185197 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc05dda3-9cc2-4d7a-8a3c-4b6bc98e70f8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:26:21.186961 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: WARNING py.warnings [None req-bc05dda3-9cc2-4d7a-8a3c-4b6bc98e70f8 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jul 21 08:26:21.186961 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: warnings.warn(msg) Jul 21 08:26:21.186961 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: Jul 21 08:26:21.188196 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc05dda3-9cc2-4d7a-8a3c-4b6bc98e70f8 admin admin] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:26:21.241930 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 751/1497] 10.222.0.47 () {60 vars in 1152 bytes} [Thu Jul 21 08:26:21 2022] POST /identity/v3/projects => generated 375 bytes in 164 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 08:26:21.246134 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7202d89-45f5-46e8-b266-0d7027ca3630 admin admin] Authenticating user token {{(pid=80245) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:26:21.247703 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.fernet_utils [None req-f7202d89-45f5-46e8-b266-0d7027ca3630 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80245) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 08:26:21.281527 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7202d89-45f5-46e8-b266-0d7027ca3630 admin admin] Validating token access rules against request {{(pid=80245) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:26:21.376537 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a5d372e-5a07-4069-83fb-ca505c6ab8be admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '42d15fc0515945acacbbffcf2356c7a5', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80245) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:26:21.377216 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a5d372e-5a07-4069-83fb-ca505c6ab8be admin admin] REQUEST_METHOD: `POST` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:26:21.377508 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a5d372e-5a07-4069-83fb-ca505c6ab8be admin admin] SCRIPT_NAME: `/identity` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:26:21.377777 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a5d372e-5a07-4069-83fb-ca505c6ab8be admin admin] PATH_INFO: `/v3/users` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:26:21.378469 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a5d372e-5a07-4069-83fb-ca505c6ab8be admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:26:21.379510 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: WARNING py.warnings [None req-2a5d372e-5a07-4069-83fb-ca505c6ab8be admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jul 21 08:26:21.379510 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: warnings.warn(msg) Jul 21 08:26:21.379510 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: Jul 21 08:26:21.380973 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a5d372e-5a07-4069-83fb-ca505c6ab8be admin admin] RBAC: Authorization granted {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:26:21.424505 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: [pid: 80245|app: 0|req: 747/1498] 10.222.0.47 () {60 vars in 1143 bytes} [Thu Jul 21 08:26:21 2022] POST /identity/v3/users => generated 288 bytes in 179 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 08:26:21.428683 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc05dda3-9cc2-4d7a-8a3c-4b6bc98e70f8 admin admin] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:26:21.430292 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.fernet_utils [None req-bc05dda3-9cc2-4d7a-8a3c-4b6bc98e70f8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80244) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 08:26:21.494216 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc05dda3-9cc2-4d7a-8a3c-4b6bc98e70f8 admin admin] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:26:21.580188 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG dbcounter [-] [80244] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80244) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:26:21.591146 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41686457-8574-48e1-a496-cfa57c3f45c3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '42d15fc0515945acacbbffcf2356c7a5', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:26:21.591859 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41686457-8574-48e1-a496-cfa57c3f45c3 admin admin] REQUEST_METHOD: `GET` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:26:21.592136 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41686457-8574-48e1-a496-cfa57c3f45c3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:26:21.592462 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41686457-8574-48e1-a496-cfa57c3f45c3 admin admin] PATH_INFO: `/v3/roles` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:26:21.593153 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41686457-8574-48e1-a496-cfa57c3f45c3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:26:21.594187 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: WARNING py.warnings [None req-41686457-8574-48e1-a496-cfa57c3f45c3 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_roles": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Jul 21 08:26:21.594187 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: warnings.warn(msg) Jul 21 08:26:21.594187 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: Jul 21 08:26:21.595070 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41686457-8574-48e1-a496-cfa57c3f45c3 admin admin] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:26:21.607281 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 752/1499] 10.222.0.47 () {58 vars in 1121 bytes} [Thu Jul 21 08:26:21 2022] GET /identity/v3/roles => generated 1420 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 21 08:26:21.611407 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a5d372e-5a07-4069-83fb-ca505c6ab8be admin admin] Authenticating user token {{(pid=80245) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:26:21.612935 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.fernet_utils [None req-2a5d372e-5a07-4069-83fb-ca505c6ab8be admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80245) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 08:26:21.624735 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a5d372e-5a07-4069-83fb-ca505c6ab8be admin admin] Validating token access rules against request {{(pid=80245) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:26:21.649963 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG dbcounter [-] [80245] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80245) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:26:21.721357 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb107326-ee36-4c8c-b7a8-72bc784e54a4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '42d15fc0515945acacbbffcf2356c7a5', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80245) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:26:21.722138 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb107326-ee36-4c8c-b7a8-72bc784e54a4 admin admin] REQUEST_METHOD: `GET` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:26:21.722321 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb107326-ee36-4c8c-b7a8-72bc784e54a4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:26:21.722632 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb107326-ee36-4c8c-b7a8-72bc784e54a4 admin admin] PATH_INFO: `/v3/roles` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:26:21.723339 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb107326-ee36-4c8c-b7a8-72bc784e54a4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:26:21.724390 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: WARNING py.warnings [None req-fb107326-ee36-4c8c-b7a8-72bc784e54a4 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_roles": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Jul 21 08:26:21.724390 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: warnings.warn(msg) Jul 21 08:26:21.724390 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: Jul 21 08:26:21.725277 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb107326-ee36-4c8c-b7a8-72bc784e54a4 admin admin] RBAC: Authorization granted {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:26:21.738341 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: [pid: 80245|app: 0|req: 748/1500] 10.222.0.47 () {58 vars in 1121 bytes} [Thu Jul 21 08:26:21 2022] GET /identity/v3/roles => generated 1420 bytes in 127 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 08:26:21.742695 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41686457-8574-48e1-a496-cfa57c3f45c3 admin admin] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:26:21.744310 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.fernet_utils [None req-41686457-8574-48e1-a496-cfa57c3f45c3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80244) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 08:26:21.777895 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41686457-8574-48e1-a496-cfa57c3f45c3 admin admin] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:26:21.872067 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74f19c1f-b8a2-4c16-8777-afcc6082cb31 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '42d15fc0515945acacbbffcf2356c7a5', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:26:21.872869 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74f19c1f-b8a2-4c16-8777-afcc6082cb31 admin admin] REQUEST_METHOD: `PUT` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:26:21.873153 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74f19c1f-b8a2-4c16-8777-afcc6082cb31 admin admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:26:21.873429 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74f19c1f-b8a2-4c16-8777-afcc6082cb31 admin admin] PATH_INFO: `/v3/projects/3879628fd4de457298006f1bffd67acb/users/8615fcb3ba1144a48e1a5265ff18474a/roles/3a074738221e4601b5205715ba87d240` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:26:21.896866 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74f19c1f-b8a2-4c16-8777-afcc6082cb31 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3879628fd4de457298006f1bffd67acb, user_id=8615fcb3ba1144a48e1a5265ff18474a, role_id=3a074738221e4601b5205715ba87d240)` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:26:21.897910 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: WARNING py.warnings [None req-74f19c1f-b8a2-4c16-8777-afcc6082cb31 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_grant": "(role:admin and system_scope:all) or ((role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.domain.id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.domain.id)s)) and (domain_id:%(target.role.domain_id)s or None:%(target.role.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jul 21 08:26:21.897910 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: warnings.warn(msg) Jul 21 08:26:21.897910 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: Jul 21 08:26:21.899888 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74f19c1f-b8a2-4c16-8777-afcc6082cb31 admin admin] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:26:21.924356 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 753/1501] 10.222.0.47 () {60 vars in 1482 bytes} [Thu Jul 21 08:26:21 2022] PUT /identity/v3/projects/3879628fd4de457298006f1bffd67acb/users/8615fcb3ba1144a48e1a5265ff18474a/roles/3a074738221e4601b5205715ba87d240 => generated 0 bytes in 182 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 08:26:21.937155 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c3f6440-b1fb-4f8a-81b4-22fd9b2c98d1 None None] REQUEST_METHOD: `POST` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:26:21.937441 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c3f6440-b1fb-4f8a-81b4-22fd9b2c98d1 None None] SCRIPT_NAME: `/identity` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:26:21.937725 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c3f6440-b1fb-4f8a-81b4-22fd9b2c98d1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:26:21.993420 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.auth.core [None req-5c3f6440-b1fb-4f8a-81b4-22fd9b2c98d1 None None] MFA Rules not processed for user `8615fcb3ba1144a48e1a5265ff18474a`. Rule list: `[]` (Enabled: `True`). {{(pid=80245) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 08:26:21.994638 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.fernet_utils [None req-5c3f6440-b1fb-4f8a-81b4-22fd9b2c98d1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80245) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 08:26:22.079648 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: [pid: 80245|app: 0|req: 749/1502] 10.222.0.47 () {58 vars in 957 bytes} [Thu Jul 21 08:26:21 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 144 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 08:26:22.093278 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-691ea58c-8ac7-43e9-83eb-cff2ca8dbb03 None None] REQUEST_METHOD: `GET` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:26:22.093567 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-691ea58c-8ac7-43e9-83eb-cff2ca8dbb03 None None] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:26:22.093841 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-691ea58c-8ac7-43e9-83eb-cff2ca8dbb03 None None] PATH_INFO: `/` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:26:22.095223 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 754/1503] 10.222.0.47 () {54 vars in 964 bytes} [Thu Jul 21 08:26:22 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 08:26:22.100219 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f135fbec-f447-4d6f-bbab-ea1496a3606a None None] REQUEST_METHOD: `POST` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:26:22.100511 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f135fbec-f447-4d6f-bbab-ea1496a3606a None None] SCRIPT_NAME: `/identity` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:26:22.100787 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f135fbec-f447-4d6f-bbab-ea1496a3606a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:26:22.130464 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG dbcounter [-] [80245] Writing DB stats keystone:SELECT=100 {{(pid=80245) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:26:22.175688 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.auth.core [None req-f135fbec-f447-4d6f-bbab-ea1496a3606a None None] MFA Rules not processed for user `b4f40410b7674687ab5362ac982248ef`. Rule list: `[]` (Enabled: `True`). {{(pid=80245) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 08:26:22.176887 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.fernet_utils [None req-f135fbec-f447-4d6f-bbab-ea1496a3606a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80245) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 08:26:22.251545 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: [pid: 80245|app: 0|req: 750/1504] 10.222.0.47 () {58 vars in 1061 bytes} [Thu Jul 21 08:26:22 2022] POST /identity/v3/auth/tokens => generated 3110 bytes in 152 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 08:26:22.259499 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-691ea58c-8ac7-43e9-83eb-cff2ca8dbb03 None None] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:26:22.261066 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.fernet_utils [None req-691ea58c-8ac7-43e9-83eb-cff2ca8dbb03 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80244) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 08:26:22.310761 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG dbcounter [-] [80244] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80244) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:26:22.369009 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-691ea58c-8ac7-43e9-83eb-cff2ca8dbb03 None None] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:26:22.442949 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ea995ff-6207-428b-8387-b29864490e7a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4f40410b7674687ab5362ac982248ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4faef13aa50e4e36b6c265fb1398fa14', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:26:22.443272 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ea995ff-6207-428b-8387-b29864490e7a service cinder] REQUEST_METHOD: `GET` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:26:22.443580 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ea995ff-6207-428b-8387-b29864490e7a service cinder] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:26:22.443853 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ea995ff-6207-428b-8387-b29864490e7a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:26:22.445524 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.fernet_utils [None req-4ea995ff-6207-428b-8387-b29864490e7a service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80244) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 08:26:22.542669 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ea995ff-6207-428b-8387-b29864490e7a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:26:22.544133 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ea995ff-6207-428b-8387-b29864490e7a service cinder] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:26:22.548172 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG dbcounter [-] [80244] Writing DB stats keystone:SELECT=100 {{(pid=80244) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:26:22.563235 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 755/1505] 10.222.0.47 () {60 vars in 1365 bytes} [Thu Jul 21 08:26:22 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 304 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 08:26:22.605376 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-470a19c6-4040-4496-8abf-df92d1542719 None None] REQUEST_METHOD: `GET` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:26:22.605670 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-470a19c6-4040-4496-8abf-df92d1542719 None None] SCRIPT_NAME: `/identity` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:26:22.605947 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-470a19c6-4040-4496-8abf-df92d1542719 None None] PATH_INFO: `/` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:26:22.607246 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: [pid: 80245|app: 0|req: 751/1506] 10.222.0.47 () {52 vars in 829 bytes} [Thu Jul 21 08:26:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 08:26:22.610833 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ea995ff-6207-428b-8387-b29864490e7a service cinder] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:26:22.612078 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.fernet_utils [None req-4ea995ff-6207-428b-8387-b29864490e7a service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80244) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 08:26:22.712958 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ea995ff-6207-428b-8387-b29864490e7a service cinder] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:26:22.808218 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f39848d4-642a-4aa4-8317-52671b09b4df admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '42d15fc0515945acacbbffcf2356c7a5', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:26:22.809087 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f39848d4-642a-4aa4-8317-52671b09b4df admin admin] REQUEST_METHOD: `DELETE` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:26:22.809371 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f39848d4-642a-4aa4-8317-52671b09b4df admin admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:26:22.809647 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f39848d4-642a-4aa4-8317-52671b09b4df admin admin] PATH_INFO: `/v3/users/8615fcb3ba1144a48e1a5265ff18474a` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:26:22.823901 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f39848d4-642a-4aa4-8317-52671b09b4df admin admin] RBAC: Authorizing `identity:delete_user(user_id=8615fcb3ba1144a48e1a5265ff18474a)` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:26:22.824950 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: WARNING py.warnings [None req-f39848d4-642a-4aa4-8317-52671b09b4df admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jul 21 08:26:22.824950 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: warnings.warn(msg) Jul 21 08:26:22.824950 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: Jul 21 08:26:22.826175 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f39848d4-642a-4aa4-8317-52671b09b4df admin admin] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:26:22.836613 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG dbcounter [-] [80244] Writing DB stats keystone:SELECT=100 {{(pid=80244) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:26:22.892002 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.notifications [None req-f39848d4-642a-4aa4-8317-52671b09b4df admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8615fcb3ba1144a48e1a5265ff18474a'} {{(pid=80244) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 08:26:22.902708 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.notifications [None req-f39848d4-642a-4aa4-8317-52671b09b4df admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8615fcb3ba1144a48e1a5265ff18474a'} {{(pid=80244) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 08:26:22.920540 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.notifications [None req-f39848d4-642a-4aa4-8317-52671b09b4df admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8615fcb3ba1144a48e1a5265ff18474a'} {{(pid=80244) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 08:26:22.920977 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.notifications [None req-f39848d4-642a-4aa4-8317-52671b09b4df admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8615fcb3ba1144a48e1a5265ff18474a'} {{(pid=80244) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 08:26:22.933002 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.notifications [None req-f39848d4-642a-4aa4-8317-52671b09b4df admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8615fcb3ba1144a48e1a5265ff18474a'} {{(pid=80244) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 08:26:22.939312 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 756/1507] 10.222.0.47 () {58 vars in 1223 bytes} [Thu Jul 21 08:26:22 2022] DELETE /identity/v3/users/8615fcb3ba1144a48e1a5265ff18474a => generated 0 bytes in 329 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 08:26:22.943743 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef674e8a-d6a9-4622-946d-1ece583c6ae8 None None] REQUEST_METHOD: `POST` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:26:22.944219 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef674e8a-d6a9-4622-946d-1ece583c6ae8 None None] SCRIPT_NAME: `/identity` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:26:22.944333 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef674e8a-d6a9-4622-946d-1ece583c6ae8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:26:22.998683 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.auth.core [None req-ef674e8a-d6a9-4622-946d-1ece583c6ae8 None None] MFA Rules not processed for user `2dce40b8d74c41ec94cc36e6ff05d412`. Rule list: `[]` (Enabled: `True`). {{(pid=80245) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 08:26:23.000371 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.fernet_utils [None req-ef674e8a-d6a9-4622-946d-1ece583c6ae8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80245) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 08:26:23.077260 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG dbcounter [-] [80245] Writing DB stats keystone:SELECT=100 {{(pid=80245) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:26:23.090096 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: [pid: 80245|app: 0|req: 752/1508] 10.222.0.47 () {58 vars in 957 bytes} [Thu Jul 21 08:26:22 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 147 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 08:26:23.100216 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f39848d4-642a-4aa4-8317-52671b09b4df admin admin] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:26:23.101451 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.fernet_utils [None req-f39848d4-642a-4aa4-8317-52671b09b4df admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80244) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 08:26:23.201053 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f39848d4-642a-4aa4-8317-52671b09b4df admin admin] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:26:23.255964 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG dbcounter [-] [80244] Writing DB stats keystone:DELETE=10,keystone:SELECT=89,keystone:INSERT=1 {{(pid=80244) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:26:23.277241 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd1df18e-2c88-4139-931d-6b36603534ae service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20ca3c3bd3774691b3374cfbfec6e494', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4faef13aa50e4e36b6c265fb1398fa14', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:26:23.277890 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd1df18e-2c88-4139-931d-6b36603534ae service neutron] REQUEST_METHOD: `GET` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:26:23.278173 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd1df18e-2c88-4139-931d-6b36603534ae service neutron] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:26:23.278445 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd1df18e-2c88-4139-931d-6b36603534ae service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:26:23.280121 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.fernet_utils [None req-bd1df18e-2c88-4139-931d-6b36603534ae service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80244) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 08:26:23.402572 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd1df18e-2c88-4139-931d-6b36603534ae service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:26:23.405866 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd1df18e-2c88-4139-931d-6b36603534ae service neutron] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:26:23.433441 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 757/1509] 10.222.0.47 () {60 vars in 1365 bytes} [Thu Jul 21 08:26:23 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 334 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 08:26:23.763162 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef674e8a-d6a9-4622-946d-1ece583c6ae8 None None] Authenticating user token {{(pid=80245) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:26:23.764431 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.fernet_utils [None req-ef674e8a-d6a9-4622-946d-1ece583c6ae8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80245) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 08:26:23.776521 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef674e8a-d6a9-4622-946d-1ece583c6ae8 None None] Validating token access rules against request {{(pid=80245) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:26:23.873287 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ece4a070-c3d2-4ec3-aa2f-dc9fafef4e96 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '42d15fc0515945acacbbffcf2356c7a5', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80245) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:26:23.874686 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ece4a070-c3d2-4ec3-aa2f-dc9fafef4e96 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:26:23.874991 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ece4a070-c3d2-4ec3-aa2f-dc9fafef4e96 admin admin] SCRIPT_NAME: `/identity` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:26:23.875264 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ece4a070-c3d2-4ec3-aa2f-dc9fafef4e96 admin admin] PATH_INFO: `/v3/projects/3879628fd4de457298006f1bffd67acb` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:26:23.885708 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ece4a070-c3d2-4ec3-aa2f-dc9fafef4e96 admin admin] RBAC: Authorizing `identity:delete_project(project_id=3879628fd4de457298006f1bffd67acb)` {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:26:23.886740 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: WARNING py.warnings [None req-ece4a070-c3d2-4ec3-aa2f-dc9fafef4e96 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jul 21 08:26:23.886740 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: warnings.warn(msg) Jul 21 08:26:23.886740 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: Jul 21 08:26:23.887986 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ece4a070-c3d2-4ec3-aa2f-dc9fafef4e96 admin admin] RBAC: Authorization granted {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:26:23.946522 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.notifications [None req-ece4a070-c3d2-4ec3-aa2f-dc9fafef4e96 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3879628fd4de457298006f1bffd67acb', 'request_id': 'req-ece4a070-c3d2-4ec3-aa2f-dc9fafef4e96'} {{(pid=80245) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 08:26:23.959275 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.notifications [None req-ece4a070-c3d2-4ec3-aa2f-dc9fafef4e96 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3879628fd4de457298006f1bffd67acb', 'request_id': 'req-ece4a070-c3d2-4ec3-aa2f-dc9fafef4e96'} {{(pid=80245) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 08:26:23.965229 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.notifications [None req-ece4a070-c3d2-4ec3-aa2f-dc9fafef4e96 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3879628fd4de457298006f1bffd67acb', 'request_id': 'req-ece4a070-c3d2-4ec3-aa2f-dc9fafef4e96'} {{(pid=80245) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 08:26:23.990611 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.notifications [None req-ece4a070-c3d2-4ec3-aa2f-dc9fafef4e96 admin admin] The token cache is being invalidate because project 3879628fd4de457298006f1bffd67acb was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80245) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 21 08:26:23.990891 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.notifications [None req-ece4a070-c3d2-4ec3-aa2f-dc9fafef4e96 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80245) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 08:26:23.991310 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.notifications [None req-ece4a070-c3d2-4ec3-aa2f-dc9fafef4e96 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80245) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 08:26:23.992782 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: [pid: 80245|app: 0|req: 753/1510] 10.222.0.47 () {58 vars in 1232 bytes} [Thu Jul 21 08:26:23 2022] DELETE /identity/v3/projects/3879628fd4de457298006f1bffd67acb => generated 0 bytes in 230 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 08:26:33.430999 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG dbcounter [-] [80244] Writing DB stats keystone:SELECT=62 {{(pid=80244) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:26:33.988458 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG dbcounter [-] [80245] Writing DB stats keystone:SELECT=84,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80245) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:28:07.105480 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a558f28-8788-476e-8592-3d9bc0848a97 None None] REQUEST_METHOD: `GET` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:28:07.106106 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a558f28-8788-476e-8592-3d9bc0848a97 None None] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:28:07.106106 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a558f28-8788-476e-8592-3d9bc0848a97 None None] PATH_INFO: `/` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:28:07.107307 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 758/1511] 127.0.0.1 () {54 vars in 901 bytes} [Thu Jul 21 08:28:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 08:28:07.112475 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ca523d8-523e-420b-b2fd-7aeb0af64b1b None None] REQUEST_METHOD: `POST` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:28:07.112999 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ca523d8-523e-420b-b2fd-7aeb0af64b1b None None] SCRIPT_NAME: `/identity` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:28:07.113119 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ca523d8-523e-420b-b2fd-7aeb0af64b1b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:28:07.194378 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.auth.core [None req-7ca523d8-523e-420b-b2fd-7aeb0af64b1b None None] MFA Rules not processed for user `2dce40b8d74c41ec94cc36e6ff05d412`. Rule list: `[]` (Enabled: `True`). {{(pid=80245) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 08:28:07.195533 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.fernet_utils [None req-7ca523d8-523e-420b-b2fd-7aeb0af64b1b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80245) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 08:28:07.303669 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: [pid: 80245|app: 0|req: 754/1512] 127.0.0.1 () {58 vars in 998 bytes} [Thu Jul 21 08:28:07 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 192 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 08:28:07.461945 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a558f28-8788-476e-8592-3d9bc0848a97 None None] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:28:07.464158 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.fernet_utils [None req-8a558f28-8788-476e-8592-3d9bc0848a97 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80244) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 08:28:07.591780 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a558f28-8788-476e-8592-3d9bc0848a97 None None] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:28:07.597368 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1710826-364e-4424-92d4-31f39b81cd48 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2b5b99a36442148fe743269054e991', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4faef13aa50e4e36b6c265fb1398fa14', 'project_domain_id': 'default', 'roles': ['member', 'service', 'ResellerAdmin', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:28:07.598022 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1710826-364e-4424-92d4-31f39b81cd48 service nova] REQUEST_METHOD: `GET` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:28:07.598307 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1710826-364e-4424-92d4-31f39b81cd48 service nova] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:28:07.598586 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1710826-364e-4424-92d4-31f39b81cd48 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:28:07.607023 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1710826-364e-4424-92d4-31f39b81cd48 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:28:07.608653 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1710826-364e-4424-92d4-31f39b81cd48 service nova] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:28:07.618178 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 759/1513] 10.222.0.47 () {60 vars in 1365 bytes} [Thu Jul 21 08:28:07 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 157 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 08:28:08.459104 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b79ca00-3d32-4eda-b765-9d5971412d8c None None] REQUEST_METHOD: `GET` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:28:08.459684 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b79ca00-3d32-4eda-b765-9d5971412d8c None None] SCRIPT_NAME: `/identity` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:28:08.459684 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b79ca00-3d32-4eda-b765-9d5971412d8c None None] PATH_INFO: `/` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:28:08.461127 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: [pid: 80245|app: 0|req: 755/1514] 127.0.0.1 () {54 vars in 901 bytes} [Thu Jul 21 08:28:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 08:28:08.466053 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9f40331-2169-4847-889f-c89c763686e1 None None] REQUEST_METHOD: `POST` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:28:08.466474 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9f40331-2169-4847-889f-c89c763686e1 None None] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:28:08.466622 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9f40331-2169-4847-889f-c89c763686e1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:28:08.485573 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.auth.core [None req-a9f40331-2169-4847-889f-c89c763686e1 None None] MFA Rules not processed for user `2dce40b8d74c41ec94cc36e6ff05d412`. Rule list: `[]` (Enabled: `True`). {{(pid=80244) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 08:28:08.486834 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.fernet_utils [None req-a9f40331-2169-4847-889f-c89c763686e1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80244) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 08:28:08.494853 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 760/1515] 127.0.0.1 () {58 vars in 998 bytes} [Thu Jul 21 08:28:08 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 08:28:08.679847 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b79ca00-3d32-4eda-b765-9d5971412d8c None None] Authenticating user token {{(pid=80245) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:28:08.694630 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b79ca00-3d32-4eda-b765-9d5971412d8c None None] Validating token access rules against request {{(pid=80245) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:28:08.700331 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-751b7770-1bd2-43b5-8c4c-5182f3133330 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2b5b99a36442148fe743269054e991', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4faef13aa50e4e36b6c265fb1398fa14', 'project_domain_id': 'default', 'roles': ['member', 'service', 'ResellerAdmin', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80245) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:28:08.700986 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-751b7770-1bd2-43b5-8c4c-5182f3133330 service nova] REQUEST_METHOD: `GET` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:28:08.701384 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-751b7770-1bd2-43b5-8c4c-5182f3133330 service nova] SCRIPT_NAME: `/identity` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:28:08.701587 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-751b7770-1bd2-43b5-8c4c-5182f3133330 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:28:08.710711 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-751b7770-1bd2-43b5-8c4c-5182f3133330 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:28:08.712340 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-751b7770-1bd2-43b5-8c4c-5182f3133330 service nova] RBAC: Authorization granted {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:28:08.721956 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: [pid: 80245|app: 0|req: 756/1516] 10.222.0.47 () {60 vars in 1365 bytes} [Thu Jul 21 08:28:08 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 21 08:28:10.782789 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-479964f7-c169-44d3-a7b4-6ac6183bd748 None None] REQUEST_METHOD: `GET` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:28:10.783447 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-479964f7-c169-44d3-a7b4-6ac6183bd748 None None] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:28:10.783447 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-479964f7-c169-44d3-a7b4-6ac6183bd748 None None] PATH_INFO: `/v3` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:28:10.785134 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 761/1517] 10.222.0.18 () {54 vars in 925 bytes} [Thu Jul 21 08:28:10 2022] GET /identity/v3 => generated 255 bytes in 4 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 21 08:28:10.811991 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e4875f9-68e3-4663-9a3a-bd08a7da3aad None None] REQUEST_METHOD: `POST` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:28:10.811991 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e4875f9-68e3-4663-9a3a-bd08a7da3aad None None] SCRIPT_NAME: `/identity` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:28:10.812586 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e4875f9-68e3-4663-9a3a-bd08a7da3aad None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:28:10.869951 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.auth.core [None req-2e4875f9-68e3-4663-9a3a-bd08a7da3aad None None] MFA Rules not processed for user `3379b32543474d0a8e897484e85c7d04`. Rule list: `[]` (Enabled: `True`). {{(pid=80245) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 08:28:10.871166 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.fernet_utils [None req-2e4875f9-68e3-4663-9a3a-bd08a7da3aad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80245) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 08:28:10.888930 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG dbcounter [-] [80245] Writing DB stats keystone:SELECT=100 {{(pid=80245) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:28:10.944412 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: [pid: 80245|app: 0|req: 757/1518] 10.222.0.18 () {58 vars in 1015 bytes} [Thu Jul 21 08:28:10 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 134 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 08:28:15.281796 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c79f78a-c9d3-460b-9fce-9a84590627d0 None None] REQUEST_METHOD: `GET` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:28:15.282330 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c79f78a-c9d3-460b-9fce-9a84590627d0 None None] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:28:15.282454 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c79f78a-c9d3-460b-9fce-9a84590627d0 None None] PATH_INFO: `/` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:28:15.283912 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 762/1519] 10.222.0.47 () {54 vars in 961 bytes} [Thu Jul 21 08:28:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 08:28:15.289032 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4ea5c56-13eb-4d0b-a6f9-5d63226e3904 None None] REQUEST_METHOD: `POST` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:28:15.289470 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4ea5c56-13eb-4d0b-a6f9-5d63226e3904 None None] SCRIPT_NAME: `/identity` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:28:15.289623 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4ea5c56-13eb-4d0b-a6f9-5d63226e3904 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:28:15.308636 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.auth.core [None req-f4ea5c56-13eb-4d0b-a6f9-5d63226e3904 None None] MFA Rules not processed for user `3379b32543474d0a8e897484e85c7d04`. Rule list: `[]` (Enabled: `True`). {{(pid=80245) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 08:28:15.309887 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.fernet_utils [None req-f4ea5c56-13eb-4d0b-a6f9-5d63226e3904 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80245) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 08:28:15.319183 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: [pid: 80245|app: 0|req: 758/1520] 10.222.0.47 () {58 vars in 1058 bytes} [Thu Jul 21 08:28:15 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 08:28:15.326756 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c79f78a-c9d3-460b-9fce-9a84590627d0 None None] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:28:15.340498 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c79f78a-c9d3-460b-9fce-9a84590627d0 None None] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:28:15.349864 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da88b2d0-b137-4c15-8d07-b33368657669 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3379b32543474d0a8e897484e85c7d04', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4faef13aa50e4e36b6c265fb1398fa14', 'project_domain_id': 'default', 'roles': ['service', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:28:15.350539 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da88b2d0-b137-4c15-8d07-b33368657669 service placement] REQUEST_METHOD: `GET` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:28:15.350934 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da88b2d0-b137-4c15-8d07-b33368657669 service placement] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:28:15.351169 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da88b2d0-b137-4c15-8d07-b33368657669 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:28:15.360072 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da88b2d0-b137-4c15-8d07-b33368657669 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:28:15.362156 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da88b2d0-b137-4c15-8d07-b33368657669 service placement] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:28:15.375221 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 763/1521] 10.222.0.47 () {60 vars in 1365 bytes} [Thu Jul 21 08:28:15 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 49 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 08:28:19.976619 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5af68a7b-d4ca-40c6-bf7a-5731ea236b64 None None] REQUEST_METHOD: `GET` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:28:19.977223 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5af68a7b-d4ca-40c6-bf7a-5731ea236b64 None None] SCRIPT_NAME: `/identity` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:28:19.977223 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5af68a7b-d4ca-40c6-bf7a-5731ea236b64 None None] PATH_INFO: `/` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:28:19.978582 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: [pid: 80245|app: 0|req: 759/1522] 127.0.0.1 () {54 vars in 901 bytes} [Thu Jul 21 08:28:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 08:28:19.983564 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42ab76f8-0776-49b4-92e9-2ee3ba702ba8 None None] REQUEST_METHOD: `POST` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:28:19.983995 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42ab76f8-0776-49b4-92e9-2ee3ba702ba8 None None] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:28:19.984127 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42ab76f8-0776-49b4-92e9-2ee3ba702ba8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:28:20.001978 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.auth.core [None req-42ab76f8-0776-49b4-92e9-2ee3ba702ba8 None None] MFA Rules not processed for user `2dce40b8d74c41ec94cc36e6ff05d412`. Rule list: `[]` (Enabled: `True`). {{(pid=80244) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 08:28:20.003175 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.fernet_utils [None req-42ab76f8-0776-49b4-92e9-2ee3ba702ba8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80244) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 08:28:20.011486 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 764/1523] 127.0.0.1 () {58 vars in 998 bytes} [Thu Jul 21 08:28:19 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 08:28:20.165447 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5af68a7b-d4ca-40c6-bf7a-5731ea236b64 None None] Authenticating user token {{(pid=80245) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:28:20.180057 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5af68a7b-d4ca-40c6-bf7a-5731ea236b64 None None] Validating token access rules against request {{(pid=80245) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:28:20.185361 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-076e0366-883f-453e-a674-3fa74efc88aa service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2b5b99a36442148fe743269054e991', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4faef13aa50e4e36b6c265fb1398fa14', 'project_domain_id': 'default', 'roles': ['member', 'service', 'ResellerAdmin', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80245) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:28:20.185996 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-076e0366-883f-453e-a674-3fa74efc88aa service nova] REQUEST_METHOD: `GET` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:28:20.186275 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-076e0366-883f-453e-a674-3fa74efc88aa service nova] SCRIPT_NAME: `/identity` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:28:20.186545 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-076e0366-883f-453e-a674-3fa74efc88aa service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:28:20.195088 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-076e0366-883f-453e-a674-3fa74efc88aa service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:28:20.196812 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-076e0366-883f-453e-a674-3fa74efc88aa service nova] RBAC: Authorization granted {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:28:20.207023 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: [pid: 80245|app: 0|req: 760/1524] 10.222.0.47 () {60 vars in 1365 bytes} [Thu Jul 21 08:28:20 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 08:28:21.025215 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6064ab0-76f9-47f7-9ec9-1364de4a7cd0 None None] REQUEST_METHOD: `GET` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:28:21.025753 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6064ab0-76f9-47f7-9ec9-1364de4a7cd0 None None] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:28:21.025858 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6064ab0-76f9-47f7-9ec9-1364de4a7cd0 None None] PATH_INFO: `/` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:28:21.027170 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 765/1525] 127.0.0.1 () {54 vars in 901 bytes} [Thu Jul 21 08:28:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 08:28:21.032017 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-923913fa-a6cb-4e28-96c4-7c32c28bb4c7 None None] REQUEST_METHOD: `POST` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:28:21.032416 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-923913fa-a6cb-4e28-96c4-7c32c28bb4c7 None None] SCRIPT_NAME: `/identity` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:28:21.032571 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-923913fa-a6cb-4e28-96c4-7c32c28bb4c7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:28:21.052594 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.auth.core [None req-923913fa-a6cb-4e28-96c4-7c32c28bb4c7 None None] MFA Rules not processed for user `2dce40b8d74c41ec94cc36e6ff05d412`. Rule list: `[]` (Enabled: `True`). {{(pid=80245) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 08:28:21.053786 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.fernet_utils [None req-923913fa-a6cb-4e28-96c4-7c32c28bb4c7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80245) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 08:28:21.061596 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: [pid: 80245|app: 0|req: 761/1526] 127.0.0.1 () {58 vars in 998 bytes} [Thu Jul 21 08:28:21 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 08:28:21.211963 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6064ab0-76f9-47f7-9ec9-1364de4a7cd0 None None] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:28:21.227012 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6064ab0-76f9-47f7-9ec9-1364de4a7cd0 None None] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:28:21.232514 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f46edc1c-4400-4dd5-ba34-f80ffe290148 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2b5b99a36442148fe743269054e991', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4faef13aa50e4e36b6c265fb1398fa14', 'project_domain_id': 'default', 'roles': ['member', 'service', 'ResellerAdmin', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:28:21.237338 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f46edc1c-4400-4dd5-ba34-f80ffe290148 service nova] REQUEST_METHOD: `GET` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:28:21.237338 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f46edc1c-4400-4dd5-ba34-f80ffe290148 service nova] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:28:21.237338 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f46edc1c-4400-4dd5-ba34-f80ffe290148 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:28:21.242413 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f46edc1c-4400-4dd5-ba34-f80ffe290148 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:28:21.244083 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f46edc1c-4400-4dd5-ba34-f80ffe290148 service nova] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:28:21.253941 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 766/1527] 10.222.0.47 () {60 vars in 1365 bytes} [Thu Jul 21 08:28:21 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 08:28:25.432045 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-509d3dbf-9315-4813-b124-2dc6b7f27e7c None None] REQUEST_METHOD: `GET` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:28:25.432628 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-509d3dbf-9315-4813-b124-2dc6b7f27e7c None None] SCRIPT_NAME: `/identity` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:28:25.432628 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-509d3dbf-9315-4813-b124-2dc6b7f27e7c None None] PATH_INFO: `/` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:28:25.434016 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: [pid: 80245|app: 0|req: 762/1528] 127.0.0.1 () {54 vars in 916 bytes} [Thu Jul 21 08:28:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 08:28:25.439371 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9de9a2a0-4ec7-4d6d-8f1b-e04171c523c4 None None] REQUEST_METHOD: `POST` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:28:25.439811 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9de9a2a0-4ec7-4d6d-8f1b-e04171c523c4 None None] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:28:25.440444 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9de9a2a0-4ec7-4d6d-8f1b-e04171c523c4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:28:25.458905 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.auth.core [None req-9de9a2a0-4ec7-4d6d-8f1b-e04171c523c4 None None] MFA Rules not processed for user `2dce40b8d74c41ec94cc36e6ff05d412`. Rule list: `[]` (Enabled: `True`). {{(pid=80244) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 08:28:25.460143 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.fernet_utils [None req-9de9a2a0-4ec7-4d6d-8f1b-e04171c523c4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80244) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 08:28:25.468003 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 767/1529] 127.0.0.1 () {58 vars in 1013 bytes} [Thu Jul 21 08:28:25 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 08:28:25.545140 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-509d3dbf-9315-4813-b124-2dc6b7f27e7c None None] Authenticating user token {{(pid=80245) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:28:25.546857 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.fernet_utils [None req-509d3dbf-9315-4813-b124-2dc6b7f27e7c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80245) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 08:28:25.625320 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-509d3dbf-9315-4813-b124-2dc6b7f27e7c None None] Validating token access rules against request {{(pid=80245) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:28:25.629314 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae125cbe-8db8-4b63-ab89-4df7ea033908 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20ca3c3bd3774691b3374cfbfec6e494', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4faef13aa50e4e36b6c265fb1398fa14', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80245) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:28:25.630210 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae125cbe-8db8-4b63-ab89-4df7ea033908 service neutron] REQUEST_METHOD: `GET` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:28:25.630494 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae125cbe-8db8-4b63-ab89-4df7ea033908 service neutron] SCRIPT_NAME: `/identity` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:28:25.630770 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae125cbe-8db8-4b63-ab89-4df7ea033908 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:28:25.639587 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae125cbe-8db8-4b63-ab89-4df7ea033908 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:28:25.641005 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae125cbe-8db8-4b63-ab89-4df7ea033908 service neutron] RBAC: Authorization granted {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:28:25.650965 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: [pid: 80245|app: 0|req: 763/1530] 10.222.0.47 () {60 vars in 1365 bytes} [Thu Jul 21 08:28:25 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 106 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 08:28:27.745872 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5046d4bb-c0ef-4a60-9017-cb799b39988c None None] REQUEST_METHOD: `GET` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:28:27.746391 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5046d4bb-c0ef-4a60-9017-cb799b39988c None None] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:28:27.746391 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5046d4bb-c0ef-4a60-9017-cb799b39988c None None] PATH_INFO: `/` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:28:27.747841 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 768/1531] 127.0.0.1 () {54 vars in 916 bytes} [Thu Jul 21 08:28:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 08:28:27.753061 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90b5652c-94a8-49a1-a9aa-632501cf6cef None None] REQUEST_METHOD: `POST` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:28:27.753489 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90b5652c-94a8-49a1-a9aa-632501cf6cef None None] SCRIPT_NAME: `/identity` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:28:27.753612 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90b5652c-94a8-49a1-a9aa-632501cf6cef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:28:27.771807 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.auth.core [None req-90b5652c-94a8-49a1-a9aa-632501cf6cef None None] MFA Rules not processed for user `2dce40b8d74c41ec94cc36e6ff05d412`. Rule list: `[]` (Enabled: `True`). {{(pid=80245) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 08:28:27.773001 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.fernet_utils [None req-90b5652c-94a8-49a1-a9aa-632501cf6cef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80245) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 08:28:27.780822 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: [pid: 80245|app: 0|req: 764/1532] 127.0.0.1 () {58 vars in 1013 bytes} [Thu Jul 21 08:28:27 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 08:28:27.819025 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5046d4bb-c0ef-4a60-9017-cb799b39988c None None] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:28:27.820775 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.fernet_utils [None req-5046d4bb-c0ef-4a60-9017-cb799b39988c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80244) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 08:28:27.833887 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5046d4bb-c0ef-4a60-9017-cb799b39988c None None] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:28:27.837385 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-499bd96f-85c4-4de5-8982-36dcb9266148 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20ca3c3bd3774691b3374cfbfec6e494', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4faef13aa50e4e36b6c265fb1398fa14', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:28:27.838028 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-499bd96f-85c4-4de5-8982-36dcb9266148 service neutron] REQUEST_METHOD: `GET` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:28:27.838315 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-499bd96f-85c4-4de5-8982-36dcb9266148 service neutron] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:28:27.838597 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-499bd96f-85c4-4de5-8982-36dcb9266148 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:28:27.847978 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-499bd96f-85c4-4de5-8982-36dcb9266148 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:28:27.849405 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-499bd96f-85c4-4de5-8982-36dcb9266148 service neutron] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:28:27.858722 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 769/1533] 10.222.0.47 () {60 vars in 1365 bytes} [Thu Jul 21 08:28:27 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 08:28:31.528300 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ed4697f-fb12-407e-a6e6-4488375e04c4 None None] REQUEST_METHOD: `GET` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:28:31.528804 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ed4697f-fb12-407e-a6e6-4488375e04c4 None None] SCRIPT_NAME: `/identity` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:28:31.528920 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ed4697f-fb12-407e-a6e6-4488375e04c4 None None] PATH_INFO: `/` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:28:31.530223 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: [pid: 80245|app: 0|req: 765/1534] 127.0.0.1 () {54 vars in 916 bytes} [Thu Jul 21 08:28:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 08:28:31.535864 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f3c082a-04da-46ad-98f9-df3d9f99c3c6 None None] REQUEST_METHOD: `POST` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:28:31.536268 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f3c082a-04da-46ad-98f9-df3d9f99c3c6 None None] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:28:31.536430 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f3c082a-04da-46ad-98f9-df3d9f99c3c6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:28:31.555210 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.auth.core [None req-1f3c082a-04da-46ad-98f9-df3d9f99c3c6 None None] MFA Rules not processed for user `2dce40b8d74c41ec94cc36e6ff05d412`. Rule list: `[]` (Enabled: `True`). {{(pid=80244) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 08:28:31.556467 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.fernet_utils [None req-1f3c082a-04da-46ad-98f9-df3d9f99c3c6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80244) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 08:28:31.564457 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 770/1535] 127.0.0.1 () {58 vars in 1013 bytes} [Thu Jul 21 08:28:31 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 08:28:31.878197 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ed4697f-fb12-407e-a6e6-4488375e04c4 None None] Authenticating user token {{(pid=80245) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:28:31.880200 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.fernet_utils [None req-7ed4697f-fb12-407e-a6e6-4488375e04c4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80245) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 08:28:31.945330 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG dbcounter [-] [80245] Writing DB stats keystone:SELECT=100 {{(pid=80245) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:28:31.959361 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ed4697f-fb12-407e-a6e6-4488375e04c4 None None] Validating token access rules against request {{(pid=80245) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:28:31.963011 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6a401da-b402-43ac-95ce-7f52219fc203 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4f40410b7674687ab5362ac982248ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4faef13aa50e4e36b6c265fb1398fa14', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80245) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:28:31.963736 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6a401da-b402-43ac-95ce-7f52219fc203 service cinder] REQUEST_METHOD: `GET` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:28:31.964024 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6a401da-b402-43ac-95ce-7f52219fc203 service cinder] SCRIPT_NAME: `/identity` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:28:31.964322 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6a401da-b402-43ac-95ce-7f52219fc203 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:28:31.974121 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6a401da-b402-43ac-95ce-7f52219fc203 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:28:31.975599 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6a401da-b402-43ac-95ce-7f52219fc203 service cinder] RBAC: Authorization granted {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:28:31.984904 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: [pid: 80245|app: 0|req: 766/1536] 10.222.0.47 () {60 vars in 1365 bytes} [Thu Jul 21 08:28:31 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 107 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 08:28:34.063775 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f780820d-b540-479e-aa2d-3dec6fc1aa6b None None] REQUEST_METHOD: `GET` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:28:34.064333 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f780820d-b540-479e-aa2d-3dec6fc1aa6b None None] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:28:34.064333 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f780820d-b540-479e-aa2d-3dec6fc1aa6b None None] PATH_INFO: `/` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:28:34.066662 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 771/1537] 127.0.0.1 () {54 vars in 916 bytes} [Thu Jul 21 08:28:34 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 08:28:34.071889 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72fe9fd0-1876-4d0b-965c-f05c77dc2177 None None] REQUEST_METHOD: `POST` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:28:34.072294 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72fe9fd0-1876-4d0b-965c-f05c77dc2177 None None] SCRIPT_NAME: `/identity` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:28:34.072442 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72fe9fd0-1876-4d0b-965c-f05c77dc2177 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:28:34.090188 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.auth.core [None req-72fe9fd0-1876-4d0b-965c-f05c77dc2177 None None] MFA Rules not processed for user `2dce40b8d74c41ec94cc36e6ff05d412`. Rule list: `[]` (Enabled: `True`). {{(pid=80245) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 08:28:34.091416 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.fernet_utils [None req-72fe9fd0-1876-4d0b-965c-f05c77dc2177 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80245) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 08:28:34.099746 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: [pid: 80245|app: 0|req: 767/1538] 127.0.0.1 () {58 vars in 1013 bytes} [Thu Jul 21 08:28:34 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 08:28:34.421539 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f780820d-b540-479e-aa2d-3dec6fc1aa6b None None] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:28:34.423235 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.fernet_utils [None req-f780820d-b540-479e-aa2d-3dec6fc1aa6b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80244) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 08:28:34.431620 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG dbcounter [-] [80244] Writing DB stats keystone:SELECT=84 {{(pid=80244) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:28:34.437552 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f780820d-b540-479e-aa2d-3dec6fc1aa6b None None] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:28:34.441225 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dda775b8-ae2f-4210-8b97-ffe79d25307f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4f40410b7674687ab5362ac982248ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4faef13aa50e4e36b6c265fb1398fa14', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:28:34.441868 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dda775b8-ae2f-4210-8b97-ffe79d25307f service cinder] REQUEST_METHOD: `GET` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:28:34.442141 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dda775b8-ae2f-4210-8b97-ffe79d25307f service cinder] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:28:34.442432 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dda775b8-ae2f-4210-8b97-ffe79d25307f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:28:34.451318 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dda775b8-ae2f-4210-8b97-ffe79d25307f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:28:34.452764 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dda775b8-ae2f-4210-8b97-ffe79d25307f service cinder] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:28:34.462540 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 772/1539] 10.222.0.47 () {60 vars in 1365 bytes} [Thu Jul 21 08:28:34 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 08:28:44.085496 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG dbcounter [-] [80245] Writing DB stats keystone:SELECT=11 {{(pid=80245) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:28:44.458447 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG dbcounter [-] [80244] Writing DB stats keystone:SELECT=5 {{(pid=80244) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:30:05.204859 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d00a53eb-c7df-4d20-b785-ddf1805fad28 None None] REQUEST_METHOD: `POST` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:30:05.211784 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d00a53eb-c7df-4d20-b785-ddf1805fad28 None None] SCRIPT_NAME: `/identity` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:30:05.211784 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d00a53eb-c7df-4d20-b785-ddf1805fad28 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:30:05.233383 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.auth.core [None req-d00a53eb-c7df-4d20-b785-ddf1805fad28 None None] MFA Rules not processed for user `2dce40b8d74c41ec94cc36e6ff05d412`. Rule list: `[]` (Enabled: `True`). {{(pid=80245) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 08:30:05.236646 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.fernet_utils [None req-d00a53eb-c7df-4d20-b785-ddf1805fad28 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80245) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 08:30:05.256822 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: [pid: 80245|app: 0|req: 768/1540] 10.222.0.47 () {58 vars in 957 bytes} [Thu Jul 21 08:30:05 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 53 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 08:30:05.265611 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c537d916-ddc3-4e7a-b7f3-0c96d0f71fc0 None None] REQUEST_METHOD: `POST` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:30:05.266232 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c537d916-ddc3-4e7a-b7f3-0c96d0f71fc0 None None] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:30:05.266232 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c537d916-ddc3-4e7a-b7f3-0c96d0f71fc0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:30:05.306898 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.auth.core [None req-c537d916-ddc3-4e7a-b7f3-0c96d0f71fc0 None None] MFA Rules not processed for user `2dce40b8d74c41ec94cc36e6ff05d412`. Rule list: `[]` (Enabled: `True`). {{(pid=80244) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 08:30:05.308189 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.fernet_utils [None req-c537d916-ddc3-4e7a-b7f3-0c96d0f71fc0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80244) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 08:30:05.314840 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abdc0be3-a0fd-44dd-a7ca-a9bdf6ca11d7 None None] REQUEST_METHOD: `POST` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:30:05.315136 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abdc0be3-a0fd-44dd-a7ca-a9bdf6ca11d7 None None] SCRIPT_NAME: `/identity` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:30:05.315432 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abdc0be3-a0fd-44dd-a7ca-a9bdf6ca11d7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:30:05.341330 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.auth.core [None req-abdc0be3-a0fd-44dd-a7ca-a9bdf6ca11d7 None None] MFA Rules not processed for user `2dce40b8d74c41ec94cc36e6ff05d412`. Rule list: `[]` (Enabled: `True`). {{(pid=80245) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 08:30:05.342572 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.fernet_utils [None req-abdc0be3-a0fd-44dd-a7ca-a9bdf6ca11d7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80245) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 08:30:05.361365 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 773/1541] 10.222.0.47 () {58 vars in 957 bytes} [Thu Jul 21 08:30:05 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 08:30:05.371347 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: [pid: 80245|app: 0|req: 769/1542] 10.222.0.47 () {58 vars in 957 bytes} [Thu Jul 21 08:30:05 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 54 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 08:30:05.371558 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c537d916-ddc3-4e7a-b7f3-0c96d0f71fc0 None None] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:30:05.378662 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb6047c4-e1b5-45ca-9449-fd3304df1300 None None] REQUEST_METHOD: `POST` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:30:05.379011 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb6047c4-e1b5-45ca-9449-fd3304df1300 None None] SCRIPT_NAME: `/identity` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:30:05.379242 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb6047c4-e1b5-45ca-9449-fd3304df1300 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:30:05.401394 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c537d916-ddc3-4e7a-b7f3-0c96d0f71fc0 None None] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:30:05.420591 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.auth.core [None req-eb6047c4-e1b5-45ca-9449-fd3304df1300 None None] MFA Rules not processed for user `2dce40b8d74c41ec94cc36e6ff05d412`. Rule list: `[]` (Enabled: `True`). {{(pid=80245) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 08:30:05.426606 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.fernet_utils [None req-eb6047c4-e1b5-45ca-9449-fd3304df1300 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80245) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 08:30:05.437980 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: [pid: 80245|app: 0|req: 770/1543] 10.222.0.47 () {58 vars in 957 bytes} [Thu Jul 21 08:30:05 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 60 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 08:30:05.439807 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cedab282-d263-43e1-996a-8676e2a088e2 None None] REQUEST_METHOD: `POST` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:30:05.439953 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-287db76e-0f9d-469c-b7d4-c27078796628 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '42d15fc0515945acacbbffcf2356c7a5', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:30:05.440742 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cedab282-d263-43e1-996a-8676e2a088e2 None None] SCRIPT_NAME: `/identity` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:30:05.441487 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cedab282-d263-43e1-996a-8676e2a088e2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:30:05.443680 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-287db76e-0f9d-469c-b7d4-c27078796628 admin admin] REQUEST_METHOD: `GET` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:30:05.443827 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-287db76e-0f9d-469c-b7d4-c27078796628 admin admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:30:05.444103 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-287db76e-0f9d-469c-b7d4-c27078796628 admin admin] PATH_INFO: `/v3/domains` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:30:05.444893 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-287db76e-0f9d-469c-b7d4-c27078796628 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80244) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 08:30:05.445161 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-287db76e-0f9d-469c-b7d4-c27078796628 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:30:05.446192 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: WARNING py.warnings [None req-287db76e-0f9d-469c-b7d4-c27078796628 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_domains": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Jul 21 08:30:05.446192 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: warnings.warn(msg) Jul 21 08:30:05.446192 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: Jul 21 08:30:05.447098 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-287db76e-0f9d-469c-b7d4-c27078796628 admin admin] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:30:05.477104 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.auth.core [None req-cedab282-d263-43e1-996a-8676e2a088e2 None None] MFA Rules not processed for user `2dce40b8d74c41ec94cc36e6ff05d412`. Rule list: `[]` (Enabled: `True`). {{(pid=80245) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 08:30:05.479055 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 774/1544] 10.222.0.47 () {58 vars in 1152 bytes} [Thu Jul 21 08:30:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 111 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 21 08:30:05.482697 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f75ede86-9b6f-4734-883b-1be961abd08c None None] REQUEST_METHOD: `POST` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:30:05.483132 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.fernet_utils [None req-cedab282-d263-43e1-996a-8676e2a088e2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80245) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 08:30:05.485163 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f75ede86-9b6f-4734-883b-1be961abd08c None None] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:30:05.485696 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f75ede86-9b6f-4734-883b-1be961abd08c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:30:05.512819 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: [pid: 80245|app: 0|req: 771/1545] 10.222.0.47 () {58 vars in 957 bytes} [Thu Jul 21 08:30:05 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 74 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 08:30:05.514259 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78ea3198-e454-4373-a50f-da96a725fe36 None None] REQUEST_METHOD: `POST` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:30:05.514556 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78ea3198-e454-4373-a50f-da96a725fe36 None None] SCRIPT_NAME: `/identity` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:30:05.514841 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78ea3198-e454-4373-a50f-da96a725fe36 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:30:05.536895 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.auth.core [None req-f75ede86-9b6f-4734-883b-1be961abd08c None None] MFA Rules not processed for user `2dce40b8d74c41ec94cc36e6ff05d412`. Rule list: `[]` (Enabled: `True`). {{(pid=80244) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 08:30:05.538131 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.fernet_utils [None req-f75ede86-9b6f-4734-883b-1be961abd08c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80244) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 08:30:05.544713 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.auth.core [None req-78ea3198-e454-4373-a50f-da96a725fe36 None None] MFA Rules not processed for user `2dce40b8d74c41ec94cc36e6ff05d412`. Rule list: `[]` (Enabled: `True`). {{(pid=80245) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 08:30:05.545900 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.fernet_utils [None req-78ea3198-e454-4373-a50f-da96a725fe36 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80245) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 08:30:05.548061 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 775/1546] 10.222.0.47 () {58 vars in 957 bytes} [Thu Jul 21 08:30:05 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 66 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 08:30:05.548691 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f75ede86-9b6f-4734-883b-1be961abd08c None None] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:30:05.558758 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f75ede86-9b6f-4734-883b-1be961abd08c None None] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:30:05.564603 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: [pid: 80245|app: 0|req: 772/1547] 10.222.0.47 () {58 vars in 957 bytes} [Thu Jul 21 08:30:05 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 51 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 08:30:05.565439 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78ea3198-e454-4373-a50f-da96a725fe36 None None] Authenticating user token {{(pid=80245) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:30:05.571678 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cb8b2ba-cbba-4ce4-b3ce-0a88478b4f96 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '42d15fc0515945acacbbffcf2356c7a5', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:30:05.572366 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cb8b2ba-cbba-4ce4-b3ce-0a88478b4f96 admin admin] REQUEST_METHOD: `GET` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:30:05.572652 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cb8b2ba-cbba-4ce4-b3ce-0a88478b4f96 admin admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:30:05.572933 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cb8b2ba-cbba-4ce4-b3ce-0a88478b4f96 admin admin] PATH_INFO: `/v3/domains` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:30:05.573684 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cb8b2ba-cbba-4ce4-b3ce-0a88478b4f96 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80244) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 08:30:05.573992 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cb8b2ba-cbba-4ce4-b3ce-0a88478b4f96 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:30:05.575810 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cb8b2ba-cbba-4ce4-b3ce-0a88478b4f96 admin admin] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:30:05.577024 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78ea3198-e454-4373-a50f-da96a725fe36 None None] Validating token access rules against request {{(pid=80245) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:30:05.582710 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c5756d6-37ef-46eb-a5bc-ae7cc885f616 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '42d15fc0515945acacbbffcf2356c7a5', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80245) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:30:05.583519 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c5756d6-37ef-46eb-a5bc-ae7cc885f616 admin admin] REQUEST_METHOD: `POST` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:30:05.583810 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c5756d6-37ef-46eb-a5bc-ae7cc885f616 admin admin] SCRIPT_NAME: `/identity` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:30:05.584091 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c5756d6-37ef-46eb-a5bc-ae7cc885f616 admin admin] PATH_INFO: `/v3/projects` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:30:05.584781 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c5756d6-37ef-46eb-a5bc-ae7cc885f616 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:30:05.585796 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: WARNING py.warnings [None req-8c5756d6-37ef-46eb-a5bc-ae7cc885f616 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jul 21 08:30:05.585796 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: warnings.warn(msg) Jul 21 08:30:05.585796 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: Jul 21 08:30:05.586980 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c5756d6-37ef-46eb-a5bc-ae7cc885f616 admin admin] RBAC: Authorization granted {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:30:05.591373 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 776/1548] 10.222.0.47 () {58 vars in 1152 bytes} [Thu Jul 21 08:30:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 08:30:05.591373 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b98abacd-202a-4584-b1eb-b6d32a9b21de None None] REQUEST_METHOD: `POST` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:30:05.591373 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b98abacd-202a-4584-b1eb-b6d32a9b21de None None] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:30:05.591373 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b98abacd-202a-4584-b1eb-b6d32a9b21de None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:30:05.611825 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.auth.core [None req-b98abacd-202a-4584-b1eb-b6d32a9b21de None None] MFA Rules not processed for user `2dce40b8d74c41ec94cc36e6ff05d412`. Rule list: `[]` (Enabled: `True`). {{(pid=80244) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 08:30:05.613111 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.fernet_utils [None req-b98abacd-202a-4584-b1eb-b6d32a9b21de None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80244) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 08:30:05.630799 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 777/1549] 10.222.0.47 () {58 vars in 957 bytes} [Thu Jul 21 08:30:05 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 08:30:05.630799 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f50d67e-289a-4ad1-81ef-29a808a7bbce None None] REQUEST_METHOD: `POST` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:30:05.630799 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f50d67e-289a-4ad1-81ef-29a808a7bbce None None] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:30:05.630799 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f50d67e-289a-4ad1-81ef-29a808a7bbce None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:30:05.637431 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: [pid: 80245|app: 0|req: 773/1550] 10.222.0.47 () {60 vars in 1152 bytes} [Thu Jul 21 08:30:05 2022] POST /identity/v3/projects => generated 387 bytes in 69 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 08:30:05.637431 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55327e12-3588-4ef8-a6b8-b70ea675c1c6 None None] REQUEST_METHOD: `POST` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:30:05.637431 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55327e12-3588-4ef8-a6b8-b70ea675c1c6 None None] SCRIPT_NAME: `/identity` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:30:05.637431 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55327e12-3588-4ef8-a6b8-b70ea675c1c6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:30:05.648940 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.auth.core [None req-7f50d67e-289a-4ad1-81ef-29a808a7bbce None None] MFA Rules not processed for user `2dce40b8d74c41ec94cc36e6ff05d412`. Rule list: `[]` (Enabled: `True`). {{(pid=80244) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 08:30:05.651019 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.fernet_utils [None req-7f50d67e-289a-4ad1-81ef-29a808a7bbce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80244) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 08:30:05.656544 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.auth.core [None req-55327e12-3588-4ef8-a6b8-b70ea675c1c6 None None] MFA Rules not processed for user `2dce40b8d74c41ec94cc36e6ff05d412`. Rule list: `[]` (Enabled: `True`). {{(pid=80245) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 08:30:05.657971 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.fernet_utils [None req-55327e12-3588-4ef8-a6b8-b70ea675c1c6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80245) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 08:30:05.725749 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 778/1551] 10.222.0.47 () {58 vars in 957 bytes} [Thu Jul 21 08:30:05 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 08:30:05.725749 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ab50bde-7e4e-47a2-a4fc-f72cfbd5a139 None None] REQUEST_METHOD: `POST` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:30:05.725749 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ab50bde-7e4e-47a2-a4fc-f72cfbd5a139 None None] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:30:05.725749 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ab50bde-7e4e-47a2-a4fc-f72cfbd5a139 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:30:05.730160 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: [pid: 80245|app: 0|req: 774/1552] 10.222.0.47 () {58 vars in 957 bytes} [Thu Jul 21 08:30:05 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 08:30:05.730160 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a11d427b-f6f3-4f65-99a5-87fb9f793d4c None None] REQUEST_METHOD: `POST` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:30:05.730160 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a11d427b-f6f3-4f65-99a5-87fb9f793d4c None None] SCRIPT_NAME: `/identity` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:30:05.730160 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a11d427b-f6f3-4f65-99a5-87fb9f793d4c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:30:05.741243 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.auth.core [None req-3ab50bde-7e4e-47a2-a4fc-f72cfbd5a139 None None] MFA Rules not processed for user `2dce40b8d74c41ec94cc36e6ff05d412`. Rule list: `[]` (Enabled: `True`). {{(pid=80244) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 08:30:05.742769 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.fernet_utils [None req-3ab50bde-7e4e-47a2-a4fc-f72cfbd5a139 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80244) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 08:30:05.753154 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 779/1553] 10.222.0.47 () {58 vars in 957 bytes} [Thu Jul 21 08:30:05 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 08:30:05.754842 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf78cd4f-79ca-4411-b208-ec46eac70f75 None None] REQUEST_METHOD: `POST` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:30:05.755131 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf78cd4f-79ca-4411-b208-ec46eac70f75 None None] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:30:05.755441 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf78cd4f-79ca-4411-b208-ec46eac70f75 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:30:05.757293 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.auth.core [None req-a11d427b-f6f3-4f65-99a5-87fb9f793d4c None None] MFA Rules not processed for user `2dce40b8d74c41ec94cc36e6ff05d412`. Rule list: `[]` (Enabled: `True`). {{(pid=80245) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 08:30:05.758527 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.fernet_utils [None req-a11d427b-f6f3-4f65-99a5-87fb9f793d4c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80245) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 08:30:05.766998 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: [pid: 80245|app: 0|req: 775/1554] 10.222.0.47 () {58 vars in 957 bytes} [Thu Jul 21 08:30:05 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 40 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 08:30:05.767649 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a11d427b-f6f3-4f65-99a5-87fb9f793d4c None None] Authenticating user token {{(pid=80245) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:30:05.776187 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.auth.core [None req-cf78cd4f-79ca-4411-b208-ec46eac70f75 None None] MFA Rules not processed for user `2dce40b8d74c41ec94cc36e6ff05d412`. Rule list: `[]` (Enabled: `True`). {{(pid=80244) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 08:30:05.777079 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a11d427b-f6f3-4f65-99a5-87fb9f793d4c None None] Validating token access rules against request {{(pid=80245) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:30:05.777395 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.fernet_utils [None req-cf78cd4f-79ca-4411-b208-ec46eac70f75 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80244) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 08:30:05.782198 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85dd010f-acd9-4104-8b6a-226c9ce84c2f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '42d15fc0515945acacbbffcf2356c7a5', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80245) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:30:05.782873 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85dd010f-acd9-4104-8b6a-226c9ce84c2f admin admin] REQUEST_METHOD: `POST` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:30:05.783144 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85dd010f-acd9-4104-8b6a-226c9ce84c2f admin admin] SCRIPT_NAME: `/identity` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:30:05.783412 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85dd010f-acd9-4104-8b6a-226c9ce84c2f admin admin] PATH_INFO: `/v3/projects` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:30:05.784102 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85dd010f-acd9-4104-8b6a-226c9ce84c2f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:30:05.786037 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85dd010f-acd9-4104-8b6a-226c9ce84c2f admin admin] RBAC: Authorization granted {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:30:05.786570 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 780/1555] 10.222.0.47 () {58 vars in 957 bytes} [Thu Jul 21 08:30:05 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 08:30:05.787928 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf5f7ffc-9d95-44fa-a999-5979979e4521 None None] REQUEST_METHOD: `POST` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:30:05.788214 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf5f7ffc-9d95-44fa-a999-5979979e4521 None None] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:30:05.788490 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf5f7ffc-9d95-44fa-a999-5979979e4521 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:30:05.809210 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.auth.core [None req-cf5f7ffc-9d95-44fa-a999-5979979e4521 None None] MFA Rules not processed for user `2dce40b8d74c41ec94cc36e6ff05d412`. Rule list: `[]` (Enabled: `True`). {{(pid=80244) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 08:30:05.810757 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.fernet_utils [None req-cf5f7ffc-9d95-44fa-a999-5979979e4521 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80244) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 08:30:05.816984 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: [pid: 80245|app: 0|req: 776/1556] 10.222.0.47 () {60 vars in 1152 bytes} [Thu Jul 21 08:30:05 2022] POST /identity/v3/projects => generated 369 bytes in 50 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 08:30:05.817789 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85dd010f-acd9-4104-8b6a-226c9ce84c2f admin admin] Authenticating user token {{(pid=80245) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:30:05.822529 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 781/1557] 10.222.0.47 () {58 vars in 957 bytes} [Thu Jul 21 08:30:05 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 08:30:05.823953 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91ad96e7-1b2a-4ed6-8e22-fe4f53dade73 None None] REQUEST_METHOD: `POST` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:30:05.824507 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91ad96e7-1b2a-4ed6-8e22-fe4f53dade73 None None] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:30:05.824992 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91ad96e7-1b2a-4ed6-8e22-fe4f53dade73 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:30:05.844512 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.auth.core [None req-91ad96e7-1b2a-4ed6-8e22-fe4f53dade73 None None] MFA Rules not processed for user `2dce40b8d74c41ec94cc36e6ff05d412`. Rule list: `[]` (Enabled: `True`). {{(pid=80244) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 08:30:05.845743 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.fernet_utils [None req-91ad96e7-1b2a-4ed6-8e22-fe4f53dade73 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80244) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 08:30:05.883460 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85dd010f-acd9-4104-8b6a-226c9ce84c2f admin admin] Validating token access rules against request {{(pid=80245) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:30:05.891262 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae7bbf95-df89-4e79-8c49-d4618c2b1c4b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '42d15fc0515945acacbbffcf2356c7a5', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80245) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:30:05.891969 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae7bbf95-df89-4e79-8c49-d4618c2b1c4b admin admin] REQUEST_METHOD: `POST` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:30:05.892257 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae7bbf95-df89-4e79-8c49-d4618c2b1c4b admin admin] SCRIPT_NAME: `/identity` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:30:05.892545 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae7bbf95-df89-4e79-8c49-d4618c2b1c4b admin admin] PATH_INFO: `/v3/users` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:30:05.893237 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae7bbf95-df89-4e79-8c49-d4618c2b1c4b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:30:05.895215 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae7bbf95-df89-4e79-8c49-d4618c2b1c4b admin admin] RBAC: Authorization granted {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:30:05.906928 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 782/1558] 10.222.0.47 () {58 vars in 957 bytes} [Thu Jul 21 08:30:05 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 08:30:05.907984 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91ad96e7-1b2a-4ed6-8e22-fe4f53dade73 None None] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:30:05.912870 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG dbcounter [-] [80245] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80245) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:30:05.918188 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91ad96e7-1b2a-4ed6-8e22-fe4f53dade73 None None] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:30:05.920153 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: [pid: 80245|app: 0|req: 777/1559] 10.222.0.47 () {60 vars in 1143 bytes} [Thu Jul 21 08:30:05 2022] POST /identity/v3/users => generated 294 bytes in 103 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 08:30:05.921583 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aef4214-d83a-43dc-a519-963997160cfb None None] REQUEST_METHOD: `POST` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:30:05.921837 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aef4214-d83a-43dc-a519-963997160cfb None None] SCRIPT_NAME: `/identity` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:30:05.922481 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aef4214-d83a-43dc-a519-963997160cfb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:30:05.923178 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96de13e2-0f06-4097-a382-7b4afc93b9bb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '42d15fc0515945acacbbffcf2356c7a5', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:30:05.923993 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96de13e2-0f06-4097-a382-7b4afc93b9bb admin admin] REQUEST_METHOD: `GET` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:30:05.924280 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96de13e2-0f06-4097-a382-7b4afc93b9bb admin admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:30:05.924684 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96de13e2-0f06-4097-a382-7b4afc93b9bb admin admin] PATH_INFO: `/v3/domains` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:30:05.925413 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96de13e2-0f06-4097-a382-7b4afc93b9bb admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80244) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 08:30:05.925721 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96de13e2-0f06-4097-a382-7b4afc93b9bb admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:30:05.927392 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96de13e2-0f06-4097-a382-7b4afc93b9bb admin admin] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:30:05.939834 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 783/1560] 10.222.0.47 () {58 vars in 1152 bytes} [Thu Jul 21 08:30:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 08:30:05.940649 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96de13e2-0f06-4097-a382-7b4afc93b9bb admin admin] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:30:05.945849 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.auth.core [None req-7aef4214-d83a-43dc-a519-963997160cfb None None] MFA Rules not processed for user `2dce40b8d74c41ec94cc36e6ff05d412`. Rule list: `[]` (Enabled: `True`). {{(pid=80245) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 08:30:05.947710 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.fernet_utils [None req-7aef4214-d83a-43dc-a519-963997160cfb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80245) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 08:30:05.950120 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96de13e2-0f06-4097-a382-7b4afc93b9bb admin admin] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:30:05.955310 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a4f9e6f-896f-4898-b7b6-8cdb97a3aa1e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '42d15fc0515945acacbbffcf2356c7a5', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:30:05.955986 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a4f9e6f-896f-4898-b7b6-8cdb97a3aa1e admin admin] REQUEST_METHOD: `GET` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:30:05.956268 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a4f9e6f-896f-4898-b7b6-8cdb97a3aa1e admin admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:30:05.956548 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a4f9e6f-896f-4898-b7b6-8cdb97a3aa1e admin admin] PATH_INFO: `/v3/domains` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:30:05.956734 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: [pid: 80245|app: 0|req: 778/1561] 10.222.0.47 () {58 vars in 957 bytes} [Thu Jul 21 08:30:05 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 08:30:05.957269 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a4f9e6f-896f-4898-b7b6-8cdb97a3aa1e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80244) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 08:30:05.957477 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aef4214-d83a-43dc-a519-963997160cfb None None] Authenticating user token {{(pid=80245) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:30:05.957592 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a4f9e6f-896f-4898-b7b6-8cdb97a3aa1e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:30:05.959159 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a4f9e6f-896f-4898-b7b6-8cdb97a3aa1e admin admin] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:30:05.966697 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aef4214-d83a-43dc-a519-963997160cfb None None] Validating token access rules against request {{(pid=80245) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:30:05.979221 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 784/1562] 10.222.0.47 () {58 vars in 1152 bytes} [Thu Jul 21 08:30:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 08:30:05.979532 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c76809db-e51d-465c-b4ae-fd231109cd22 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '42d15fc0515945acacbbffcf2356c7a5', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80245) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:30:05.979532 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c76809db-e51d-465c-b4ae-fd231109cd22 admin admin] REQUEST_METHOD: `GET` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:30:05.979756 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a4f9e6f-896f-4898-b7b6-8cdb97a3aa1e admin admin] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:30:05.979857 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c76809db-e51d-465c-b4ae-fd231109cd22 admin admin] SCRIPT_NAME: `/identity` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:30:05.979857 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c76809db-e51d-465c-b4ae-fd231109cd22 admin admin] PATH_INFO: `/v3/domains` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:30:05.979857 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c76809db-e51d-465c-b4ae-fd231109cd22 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80245) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 08:30:05.979857 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c76809db-e51d-465c-b4ae-fd231109cd22 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:30:05.979857 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c76809db-e51d-465c-b4ae-fd231109cd22 admin admin] RBAC: Authorization granted {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:30:05.982915 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a4f9e6f-896f-4898-b7b6-8cdb97a3aa1e admin admin] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:30:05.987647 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c916b726-0e36-4bd2-9e85-521307aa7b55 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '42d15fc0515945acacbbffcf2356c7a5', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:30:05.988342 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c916b726-0e36-4bd2-9e85-521307aa7b55 admin admin] REQUEST_METHOD: `POST` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:30:05.988625 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c916b726-0e36-4bd2-9e85-521307aa7b55 admin admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:30:05.988822 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: [pid: 80245|app: 0|req: 779/1563] 10.222.0.47 () {58 vars in 1152 bytes} [Thu Jul 21 08:30:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 08:30:05.988945 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c916b726-0e36-4bd2-9e85-521307aa7b55 admin admin] PATH_INFO: `/v3/users` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:30:05.989640 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c916b726-0e36-4bd2-9e85-521307aa7b55 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:30:05.990654 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: WARNING py.warnings [None req-c916b726-0e36-4bd2-9e85-521307aa7b55 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jul 21 08:30:05.990654 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: warnings.warn(msg) Jul 21 08:30:05.990654 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: Jul 21 08:30:05.991895 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c916b726-0e36-4bd2-9e85-521307aa7b55 admin admin] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:30:05.994938 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c76809db-e51d-465c-b4ae-fd231109cd22 admin admin] Authenticating user token {{(pid=80245) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:30:06.000469 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG dbcounter [-] [80244] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80244) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:30:06.006015 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c76809db-e51d-465c-b4ae-fd231109cd22 admin admin] Validating token access rules against request {{(pid=80245) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:30:06.011631 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae939197-f3b0-416e-9cb4-44ab0dd23147 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '42d15fc0515945acacbbffcf2356c7a5', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80245) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:30:06.012299 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae939197-f3b0-416e-9cb4-44ab0dd23147 admin admin] REQUEST_METHOD: `GET` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:30:06.012582 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae939197-f3b0-416e-9cb4-44ab0dd23147 admin admin] SCRIPT_NAME: `/identity` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:30:06.012868 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae939197-f3b0-416e-9cb4-44ab0dd23147 admin admin] PATH_INFO: `/v3/domains` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:30:06.013593 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae939197-f3b0-416e-9cb4-44ab0dd23147 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80245) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 08:30:06.013897 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae939197-f3b0-416e-9cb4-44ab0dd23147 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:30:06.018518 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 785/1564] 10.222.0.47 () {60 vars in 1143 bytes} [Thu Jul 21 08:30:05 2022] POST /identity/v3/users => generated 285 bytes in 42 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 08:30:06.018518 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c916b726-0e36-4bd2-9e85-521307aa7b55 admin admin] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:30:06.018765 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae939197-f3b0-416e-9cb4-44ab0dd23147 admin admin] RBAC: Authorization granted {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:30:06.029242 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c916b726-0e36-4bd2-9e85-521307aa7b55 admin admin] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:30:06.029671 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: [pid: 80245|app: 0|req: 780/1565] 10.222.0.47 () {58 vars in 1152 bytes} [Thu Jul 21 08:30:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 08:30:06.035351 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae939197-f3b0-416e-9cb4-44ab0dd23147 admin admin] Authenticating user token {{(pid=80245) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:30:06.039536 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7598240f-3fb4-4737-b707-586200c46f37 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '42d15fc0515945acacbbffcf2356c7a5', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:30:06.040369 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7598240f-3fb4-4737-b707-586200c46f37 admin admin] REQUEST_METHOD: `GET` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:30:06.040833 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7598240f-3fb4-4737-b707-586200c46f37 admin admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:30:06.040962 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae939197-f3b0-416e-9cb4-44ab0dd23147 admin admin] Validating token access rules against request {{(pid=80245) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:30:06.041118 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7598240f-3fb4-4737-b707-586200c46f37 admin admin] PATH_INFO: `/v3/domains` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:30:06.041853 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7598240f-3fb4-4737-b707-586200c46f37 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80244) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 08:30:06.042159 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7598240f-3fb4-4737-b707-586200c46f37 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:30:06.043801 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7598240f-3fb4-4737-b707-586200c46f37 admin admin] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:30:06.045518 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56828fa3-61cd-449a-bc53-abf705436b91 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '42d15fc0515945acacbbffcf2356c7a5', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80245) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:30:06.046169 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56828fa3-61cd-449a-bc53-abf705436b91 admin admin] REQUEST_METHOD: `GET` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:30:06.046493 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56828fa3-61cd-449a-bc53-abf705436b91 admin admin] SCRIPT_NAME: `/identity` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:30:06.046898 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56828fa3-61cd-449a-bc53-abf705436b91 admin admin] PATH_INFO: `/v3/roles` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:30:06.047656 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56828fa3-61cd-449a-bc53-abf705436b91 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:30:06.049235 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56828fa3-61cd-449a-bc53-abf705436b91 admin admin] RBAC: Authorization granted {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:30:06.056775 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 786/1566] 10.222.0.47 () {58 vars in 1152 bytes} [Thu Jul 21 08:30:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 08:30:06.057779 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7598240f-3fb4-4737-b707-586200c46f37 admin admin] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:30:06.059578 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: [pid: 80245|app: 0|req: 781/1567] 10.222.0.47 () {58 vars in 1121 bytes} [Thu Jul 21 08:30:06 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 08:30:06.060246 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56828fa3-61cd-449a-bc53-abf705436b91 admin admin] Authenticating user token {{(pid=80245) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:30:06.069166 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7598240f-3fb4-4737-b707-586200c46f37 admin admin] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:30:06.073267 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56828fa3-61cd-449a-bc53-abf705436b91 admin admin] Validating token access rules against request {{(pid=80245) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:30:06.073829 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49397036-0f12-4bfa-b874-a4e7b5d2037e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '42d15fc0515945acacbbffcf2356c7a5', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:30:06.074483 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49397036-0f12-4bfa-b874-a4e7b5d2037e admin admin] REQUEST_METHOD: `POST` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:30:06.074763 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49397036-0f12-4bfa-b874-a4e7b5d2037e admin admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:30:06.075036 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49397036-0f12-4bfa-b874-a4e7b5d2037e admin admin] PATH_INFO: `/v3/projects` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:30:06.075754 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49397036-0f12-4bfa-b874-a4e7b5d2037e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:30:06.078525 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49397036-0f12-4bfa-b874-a4e7b5d2037e admin admin] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:30:06.078682 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2303c14-5fd9-4f31-b8bf-7383a343e27d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '42d15fc0515945acacbbffcf2356c7a5', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80245) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:30:06.079289 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2303c14-5fd9-4f31-b8bf-7383a343e27d admin admin] REQUEST_METHOD: `GET` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:30:06.079607 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2303c14-5fd9-4f31-b8bf-7383a343e27d admin admin] SCRIPT_NAME: `/identity` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:30:06.079884 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2303c14-5fd9-4f31-b8bf-7383a343e27d admin admin] PATH_INFO: `/v3/domains` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:30:06.080603 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2303c14-5fd9-4f31-b8bf-7383a343e27d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80245) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 08:30:06.080911 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2303c14-5fd9-4f31-b8bf-7383a343e27d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:30:06.082498 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2303c14-5fd9-4f31-b8bf-7383a343e27d admin admin] RBAC: Authorization granted {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:30:06.097172 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: [pid: 80245|app: 0|req: 782/1568] 10.222.0.47 () {58 vars in 1152 bytes} [Thu Jul 21 08:30:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 08:30:06.097725 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2303c14-5fd9-4f31-b8bf-7383a343e27d admin admin] Authenticating user token {{(pid=80245) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:30:06.109219 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2303c14-5fd9-4f31-b8bf-7383a343e27d admin admin] Validating token access rules against request {{(pid=80245) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:30:06.111674 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 787/1569] 10.222.0.47 () {60 vars in 1152 bytes} [Thu Jul 21 08:30:06 2022] POST /identity/v3/projects => generated 385 bytes in 54 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 08:30:06.112640 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49397036-0f12-4bfa-b874-a4e7b5d2037e admin admin] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:30:06.169081 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-855d85a3-988f-4412-82a8-662ed7419817 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '42d15fc0515945acacbbffcf2356c7a5', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80245) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:30:06.169760 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-855d85a3-988f-4412-82a8-662ed7419817 admin admin] REQUEST_METHOD: `POST` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:30:06.170046 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-855d85a3-988f-4412-82a8-662ed7419817 admin admin] SCRIPT_NAME: `/identity` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:30:06.170326 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-855d85a3-988f-4412-82a8-662ed7419817 admin admin] PATH_INFO: `/v3/projects` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:30:06.171019 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-855d85a3-988f-4412-82a8-662ed7419817 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:30:06.173209 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-855d85a3-988f-4412-82a8-662ed7419817 admin admin] RBAC: Authorization granted {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:30:06.177417 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49397036-0f12-4bfa-b874-a4e7b5d2037e admin admin] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:30:06.182060 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e41fccb-b40f-42b3-9b77-76bfda0a4f8a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '42d15fc0515945acacbbffcf2356c7a5', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:30:06.183016 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e41fccb-b40f-42b3-9b77-76bfda0a4f8a admin admin] REQUEST_METHOD: `POST` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:30:06.183016 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e41fccb-b40f-42b3-9b77-76bfda0a4f8a admin admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:30:06.183270 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e41fccb-b40f-42b3-9b77-76bfda0a4f8a admin admin] PATH_INFO: `/v3/projects` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:30:06.183999 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e41fccb-b40f-42b3-9b77-76bfda0a4f8a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:30:06.186022 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e41fccb-b40f-42b3-9b77-76bfda0a4f8a admin admin] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:30:06.198391 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: [pid: 80245|app: 0|req: 783/1570] 10.222.0.47 () {60 vars in 1152 bytes} [Thu Jul 21 08:30:06 2022] POST /identity/v3/projects => generated 371 bytes in 101 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 08:30:06.200471 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-855d85a3-988f-4412-82a8-662ed7419817 admin admin] Authenticating user token {{(pid=80245) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:30:06.213606 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 788/1571] 10.222.0.47 () {60 vars in 1152 bytes} [Thu Jul 21 08:30:06 2022] POST /identity/v3/projects => generated 379 bytes in 101 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 08:30:06.214476 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e41fccb-b40f-42b3-9b77-76bfda0a4f8a admin admin] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:30:06.270774 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-855d85a3-988f-4412-82a8-662ed7419817 admin admin] Validating token access rules against request {{(pid=80245) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:30:06.275884 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0b212d9-094b-4a6f-bbd6-19dfe3234b15 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '42d15fc0515945acacbbffcf2356c7a5', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80245) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:30:06.276780 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0b212d9-094b-4a6f-bbd6-19dfe3234b15 admin admin] REQUEST_METHOD: `GET` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:30:06.277417 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0b212d9-094b-4a6f-bbd6-19dfe3234b15 admin admin] SCRIPT_NAME: `/identity` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:30:06.277417 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0b212d9-094b-4a6f-bbd6-19dfe3234b15 admin admin] PATH_INFO: `/v3/roles` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:30:06.278069 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0b212d9-094b-4a6f-bbd6-19dfe3234b15 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:30:06.280036 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0b212d9-094b-4a6f-bbd6-19dfe3234b15 admin admin] RBAC: Authorization granted {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:30:06.281378 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e41fccb-b40f-42b3-9b77-76bfda0a4f8a admin admin] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:30:06.286388 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b27527fd-9d95-40bd-a4c5-1d3820cbfc6b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '42d15fc0515945acacbbffcf2356c7a5', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:30:06.287059 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b27527fd-9d95-40bd-a4c5-1d3820cbfc6b admin admin] REQUEST_METHOD: `POST` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:30:06.287399 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b27527fd-9d95-40bd-a4c5-1d3820cbfc6b admin admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:30:06.287674 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b27527fd-9d95-40bd-a4c5-1d3820cbfc6b admin admin] PATH_INFO: `/v3/projects` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:30:06.288374 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b27527fd-9d95-40bd-a4c5-1d3820cbfc6b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:30:06.290447 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b27527fd-9d95-40bd-a4c5-1d3820cbfc6b admin admin] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:30:06.291740 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: [pid: 80245|app: 0|req: 784/1572] 10.222.0.47 () {58 vars in 1121 bytes} [Thu Jul 21 08:30:06 2022] GET /identity/v3/roles => generated 1420 bytes in 92 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 08:30:06.292409 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0b212d9-094b-4a6f-bbd6-19dfe3234b15 admin admin] Authenticating user token {{(pid=80245) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:30:06.301890 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0b212d9-094b-4a6f-bbd6-19dfe3234b15 admin admin] Validating token access rules against request {{(pid=80245) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:30:06.307001 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9148f9c4-7f35-4a60-b933-f73bdb57ce03 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '42d15fc0515945acacbbffcf2356c7a5', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80245) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:30:06.307706 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9148f9c4-7f35-4a60-b933-f73bdb57ce03 admin admin] REQUEST_METHOD: `GET` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:30:06.307993 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9148f9c4-7f35-4a60-b933-f73bdb57ce03 admin admin] SCRIPT_NAME: `/identity` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:30:06.308272 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9148f9c4-7f35-4a60-b933-f73bdb57ce03 admin admin] PATH_INFO: `/v3/roles` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:30:06.309145 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9148f9c4-7f35-4a60-b933-f73bdb57ce03 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:30:06.311169 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9148f9c4-7f35-4a60-b933-f73bdb57ce03 admin admin] RBAC: Authorization granted {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:30:06.320803 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 789/1573] 10.222.0.47 () {60 vars in 1152 bytes} [Thu Jul 21 08:30:06 2022] POST /identity/v3/projects => generated 355 bytes in 107 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 08:30:06.321726 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b27527fd-9d95-40bd-a4c5-1d3820cbfc6b admin admin] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:30:06.325861 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: [pid: 80245|app: 0|req: 785/1574] 10.222.0.47 () {58 vars in 1121 bytes} [Thu Jul 21 08:30:06 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 08:30:06.325861 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9148f9c4-7f35-4a60-b933-f73bdb57ce03 admin admin] Authenticating user token {{(pid=80245) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:30:06.335511 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9148f9c4-7f35-4a60-b933-f73bdb57ce03 admin admin] Validating token access rules against request {{(pid=80245) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:30:06.343200 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG dbcounter [-] [80245] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80245) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:30:06.356263 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG dbcounter [-] [80244] Writing DB stats keystone:INSERT=5,keystone:SELECT=95 {{(pid=80244) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:30:06.391884 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b27527fd-9d95-40bd-a4c5-1d3820cbfc6b admin admin] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:30:06.394719 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d7843ab-ba8b-41ea-8575-953f905ac1d7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '42d15fc0515945acacbbffcf2356c7a5', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80245) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:30:06.395421 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d7843ab-ba8b-41ea-8575-953f905ac1d7 admin admin] REQUEST_METHOD: `POST` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:30:06.395711 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d7843ab-ba8b-41ea-8575-953f905ac1d7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:30:06.395986 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d7843ab-ba8b-41ea-8575-953f905ac1d7 admin admin] PATH_INFO: `/v3/projects` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:30:06.396535 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fd2722f-70cd-441a-82ec-1c3086bf4344 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '42d15fc0515945acacbbffcf2356c7a5', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:30:06.396672 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d7843ab-ba8b-41ea-8575-953f905ac1d7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:30:06.397180 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fd2722f-70cd-441a-82ec-1c3086bf4344 admin admin] REQUEST_METHOD: `POST` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:30:06.397494 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fd2722f-70cd-441a-82ec-1c3086bf4344 admin admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:30:06.397744 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fd2722f-70cd-441a-82ec-1c3086bf4344 admin admin] PATH_INFO: `/v3/projects` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:30:06.398439 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fd2722f-70cd-441a-82ec-1c3086bf4344 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:30:06.398599 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d7843ab-ba8b-41ea-8575-953f905ac1d7 admin admin] RBAC: Authorization granted {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:30:06.400394 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fd2722f-70cd-441a-82ec-1c3086bf4344 admin admin] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:30:06.428107 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: [pid: 80245|app: 0|req: 786/1575] 10.222.0.47 () {60 vars in 1152 bytes} [Thu Jul 21 08:30:06 2022] POST /identity/v3/projects => generated 373 bytes in 102 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 08:30:06.428107 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d7843ab-ba8b-41ea-8575-953f905ac1d7 admin admin] Authenticating user token {{(pid=80245) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:30:06.430228 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 790/1576] 10.222.0.47 () {60 vars in 1152 bytes} [Thu Jul 21 08:30:06 2022] POST /identity/v3/projects => generated 393 bytes in 109 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 08:30:06.431130 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fd2722f-70cd-441a-82ec-1c3086bf4344 admin admin] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:30:06.490003 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d7843ab-ba8b-41ea-8575-953f905ac1d7 admin admin] Validating token access rules against request {{(pid=80245) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:30:06.494703 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a683b62-6750-457a-8011-82dc119493c3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '42d15fc0515945acacbbffcf2356c7a5', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80245) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:30:06.495412 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a683b62-6750-457a-8011-82dc119493c3 admin admin] REQUEST_METHOD: `POST` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:30:06.495704 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a683b62-6750-457a-8011-82dc119493c3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:30:06.495985 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a683b62-6750-457a-8011-82dc119493c3 admin admin] PATH_INFO: `/v3/users` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:30:06.496685 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a683b62-6750-457a-8011-82dc119493c3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:30:06.497499 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fd2722f-70cd-441a-82ec-1c3086bf4344 admin admin] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:30:06.498653 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a683b62-6750-457a-8011-82dc119493c3 admin admin] RBAC: Authorization granted {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:30:06.504085 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9f8ea35-a818-41b1-b2e7-c4c2ae3e7a63 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '42d15fc0515945acacbbffcf2356c7a5', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:30:06.505276 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9f8ea35-a818-41b1-b2e7-c4c2ae3e7a63 admin admin] REQUEST_METHOD: `POST` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:30:06.505564 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9f8ea35-a818-41b1-b2e7-c4c2ae3e7a63 admin admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:30:06.505838 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9f8ea35-a818-41b1-b2e7-c4c2ae3e7a63 admin admin] PATH_INFO: `/v3/users` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:30:06.506536 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9f8ea35-a818-41b1-b2e7-c4c2ae3e7a63 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:30:06.508536 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9f8ea35-a818-41b1-b2e7-c4c2ae3e7a63 admin admin] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:30:06.531955 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: [pid: 80245|app: 0|req: 787/1577] 10.222.0.47 () {60 vars in 1143 bytes} [Thu Jul 21 08:30:06 2022] POST /identity/v3/users => generated 293 bytes in 103 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 08:30:06.531955 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a683b62-6750-457a-8011-82dc119493c3 admin admin] Authenticating user token {{(pid=80245) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:30:06.532287 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 791/1578] 10.222.0.47 () {60 vars in 1143 bytes} [Thu Jul 21 08:30:06 2022] POST /identity/v3/users => generated 286 bytes in 101 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 08:30:06.533056 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9f8ea35-a818-41b1-b2e7-c4c2ae3e7a63 admin admin] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:30:06.542534 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a683b62-6750-457a-8011-82dc119493c3 admin admin] Validating token access rules against request {{(pid=80245) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:30:06.543386 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9f8ea35-a818-41b1-b2e7-c4c2ae3e7a63 admin admin] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:30:06.547995 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e44282ce-f2b7-425b-b8b9-cf4825ecb934 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '42d15fc0515945acacbbffcf2356c7a5', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80245) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:30:06.548831 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e44282ce-f2b7-425b-b8b9-cf4825ecb934 admin admin] REQUEST_METHOD: `POST` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:30:06.549117 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e44282ce-f2b7-425b-b8b9-cf4825ecb934 admin admin] SCRIPT_NAME: `/identity` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:30:06.549328 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6307257a-0e2d-4a78-9f5d-8fd24ee2c714 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '42d15fc0515945acacbbffcf2356c7a5', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:30:06.549480 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e44282ce-f2b7-425b-b8b9-cf4825ecb934 admin admin] PATH_INFO: `/v3/users` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:30:06.550042 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6307257a-0e2d-4a78-9f5d-8fd24ee2c714 admin admin] REQUEST_METHOD: `GET` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:30:06.550147 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e44282ce-f2b7-425b-b8b9-cf4825ecb934 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:30:06.550329 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6307257a-0e2d-4a78-9f5d-8fd24ee2c714 admin admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:30:06.550615 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6307257a-0e2d-4a78-9f5d-8fd24ee2c714 admin admin] PATH_INFO: `/v3/roles` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:30:06.551378 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6307257a-0e2d-4a78-9f5d-8fd24ee2c714 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:30:06.552357 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e44282ce-f2b7-425b-b8b9-cf4825ecb934 admin admin] RBAC: Authorization granted {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:30:06.553020 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6307257a-0e2d-4a78-9f5d-8fd24ee2c714 admin admin] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:30:06.568729 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 792/1579] 10.222.0.47 () {58 vars in 1121 bytes} [Thu Jul 21 08:30:06 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 08:30:06.568729 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6307257a-0e2d-4a78-9f5d-8fd24ee2c714 admin admin] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:30:06.577138 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6307257a-0e2d-4a78-9f5d-8fd24ee2c714 admin admin] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:30:06.578477 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: [pid: 80245|app: 0|req: 788/1580] 10.222.0.47 () {60 vars in 1143 bytes} [Thu Jul 21 08:30:06 2022] POST /identity/v3/users => generated 290 bytes in 49 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 08:30:06.579125 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e44282ce-f2b7-425b-b8b9-cf4825ecb934 admin admin] Authenticating user token {{(pid=80245) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:30:06.585241 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adf16259-d85e-4f06-8e1f-41d65fd91421 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '42d15fc0515945acacbbffcf2356c7a5', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:30:06.585913 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adf16259-d85e-4f06-8e1f-41d65fd91421 admin admin] REQUEST_METHOD: `POST` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:30:06.586201 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adf16259-d85e-4f06-8e1f-41d65fd91421 admin admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:30:06.586483 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adf16259-d85e-4f06-8e1f-41d65fd91421 admin admin] PATH_INFO: `/v3/users` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:30:06.587173 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adf16259-d85e-4f06-8e1f-41d65fd91421 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:30:06.588793 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e44282ce-f2b7-425b-b8b9-cf4825ecb934 admin admin] Validating token access rules against request {{(pid=80245) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:30:06.589509 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adf16259-d85e-4f06-8e1f-41d65fd91421 admin admin] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:30:06.593408 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f6a394f-2867-48eb-bcb4-efa9dca38f2a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '42d15fc0515945acacbbffcf2356c7a5', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80245) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:30:06.594426 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f6a394f-2867-48eb-bcb4-efa9dca38f2a admin admin] REQUEST_METHOD: `PUT` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:30:06.594706 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f6a394f-2867-48eb-bcb4-efa9dca38f2a admin admin] SCRIPT_NAME: `/identity` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:30:06.594979 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f6a394f-2867-48eb-bcb4-efa9dca38f2a admin admin] PATH_INFO: `/v3/projects/1bae82711d354d908d096edaae82c2c5/users/ad56309db7044cb2a86651d4149cd090/roles/3a074738221e4601b5205715ba87d240` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:30:06.613908 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 793/1581] 10.222.0.47 () {60 vars in 1143 bytes} [Thu Jul 21 08:30:06 2022] POST /identity/v3/users => generated 278 bytes in 45 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 08:30:06.614140 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f6a394f-2867-48eb-bcb4-efa9dca38f2a admin admin] RBAC: Authorizing `identity:create_grant(project_id=1bae82711d354d908d096edaae82c2c5, user_id=ad56309db7044cb2a86651d4149cd090, role_id=3a074738221e4601b5205715ba87d240)` {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:30:06.614246 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adf16259-d85e-4f06-8e1f-41d65fd91421 admin admin] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:30:06.614337 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: WARNING py.warnings [None req-3f6a394f-2867-48eb-bcb4-efa9dca38f2a admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_grant": "(role:admin and system_scope:all) or ((role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.domain.id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.domain.id)s)) and (domain_id:%(target.role.domain_id)s or None:%(target.role.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jul 21 08:30:06.614337 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: warnings.warn(msg) Jul 21 08:30:06.614337 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: Jul 21 08:30:06.614337 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f6a394f-2867-48eb-bcb4-efa9dca38f2a admin admin] RBAC: Authorization granted {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:30:06.626125 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adf16259-d85e-4f06-8e1f-41d65fd91421 admin admin] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:30:06.630856 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: [pid: 80245|app: 0|req: 789/1582] 10.222.0.47 () {60 vars in 1482 bytes} [Thu Jul 21 08:30:06 2022] PUT /identity/v3/projects/1bae82711d354d908d096edaae82c2c5/users/ad56309db7044cb2a86651d4149cd090/roles/3a074738221e4601b5205715ba87d240 => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 08:30:06.631754 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f6a394f-2867-48eb-bcb4-efa9dca38f2a admin admin] Authenticating user token {{(pid=80245) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:30:06.684775 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG dbcounter [-] [80245] Writing DB stats keystone:SELECT=92,keystone:INSERT=8 {{(pid=80245) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:30:06.698308 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5aa9fd2-dd38-4850-b936-45bd2c1e6668 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '42d15fc0515945acacbbffcf2356c7a5', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:30:06.698994 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5aa9fd2-dd38-4850-b936-45bd2c1e6668 admin admin] REQUEST_METHOD: `POST` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:30:06.699274 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5aa9fd2-dd38-4850-b936-45bd2c1e6668 admin admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:30:06.699580 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5aa9fd2-dd38-4850-b936-45bd2c1e6668 admin admin] PATH_INFO: `/v3/users` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:30:06.700274 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5aa9fd2-dd38-4850-b936-45bd2c1e6668 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:30:06.702195 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5aa9fd2-dd38-4850-b936-45bd2c1e6668 admin admin] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:30:06.707945 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f6a394f-2867-48eb-bcb4-efa9dca38f2a admin admin] Validating token access rules against request {{(pid=80245) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:30:06.713129 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a50a59dc-a956-4e86-8ee8-f8c6c64def0c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '42d15fc0515945acacbbffcf2356c7a5', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80245) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:30:06.713863 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a50a59dc-a956-4e86-8ee8-f8c6c64def0c admin admin] REQUEST_METHOD: `POST` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:30:06.714222 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a50a59dc-a956-4e86-8ee8-f8c6c64def0c admin admin] SCRIPT_NAME: `/identity` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:30:06.714599 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a50a59dc-a956-4e86-8ee8-f8c6c64def0c admin admin] PATH_INFO: `/v3/users` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:30:06.715404 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a50a59dc-a956-4e86-8ee8-f8c6c64def0c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:30:06.717806 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a50a59dc-a956-4e86-8ee8-f8c6c64def0c admin admin] RBAC: Authorization granted {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:30:06.727426 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 794/1583] 10.222.0.47 () {60 vars in 1143 bytes} [Thu Jul 21 08:30:06 2022] POST /identity/v3/users => generated 287 bytes in 113 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 08:30:06.727426 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5aa9fd2-dd38-4850-b936-45bd2c1e6668 admin admin] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:30:06.733614 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG dbcounter [-] [80244] Writing DB stats keystone:SELECT=90,keystone:INSERT=10 {{(pid=80244) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:30:06.739820 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5aa9fd2-dd38-4850-b936-45bd2c1e6668 admin admin] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:30:06.744138 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: [pid: 80245|app: 0|req: 790/1584] 10.222.0.47 () {60 vars in 1143 bytes} [Thu Jul 21 08:30:06 2022] POST /identity/v3/users => generated 297 bytes in 109 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 08:30:06.744138 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a50a59dc-a956-4e86-8ee8-f8c6c64def0c admin admin] Authenticating user token {{(pid=80245) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:30:06.747862 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4df0d5d2-1ab6-4d7f-b572-874ab17b5837 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '42d15fc0515945acacbbffcf2356c7a5', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:30:06.748539 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4df0d5d2-1ab6-4d7f-b572-874ab17b5837 admin admin] REQUEST_METHOD: `GET` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:30:06.748813 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4df0d5d2-1ab6-4d7f-b572-874ab17b5837 admin admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:30:06.749199 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4df0d5d2-1ab6-4d7f-b572-874ab17b5837 admin admin] PATH_INFO: `/v3/roles` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:30:06.750510 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4df0d5d2-1ab6-4d7f-b572-874ab17b5837 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:30:06.752082 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a50a59dc-a956-4e86-8ee8-f8c6c64def0c admin admin] Validating token access rules against request {{(pid=80245) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:30:06.752286 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4df0d5d2-1ab6-4d7f-b572-874ab17b5837 admin admin] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:30:06.757338 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4618d6d2-9577-4878-a5bf-ada5e67efc61 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '42d15fc0515945acacbbffcf2356c7a5', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80245) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:30:06.758021 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4618d6d2-9577-4878-a5bf-ada5e67efc61 admin admin] REQUEST_METHOD: `GET` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:30:06.758305 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4618d6d2-9577-4878-a5bf-ada5e67efc61 admin admin] SCRIPT_NAME: `/identity` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:30:06.758584 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4618d6d2-9577-4878-a5bf-ada5e67efc61 admin admin] PATH_INFO: `/v3/roles` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:30:06.759315 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4618d6d2-9577-4878-a5bf-ada5e67efc61 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:30:06.761037 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4618d6d2-9577-4878-a5bf-ada5e67efc61 admin admin] RBAC: Authorization granted {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:30:06.763484 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 795/1585] 10.222.0.47 () {58 vars in 1121 bytes} [Thu Jul 21 08:30:06 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 08:30:06.764751 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4df0d5d2-1ab6-4d7f-b572-874ab17b5837 admin admin] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:30:06.776249 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: [pid: 80245|app: 0|req: 791/1586] 10.222.0.47 () {58 vars in 1121 bytes} [Thu Jul 21 08:30:06 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 08:30:06.776628 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4df0d5d2-1ab6-4d7f-b572-874ab17b5837 admin admin] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:30:06.776957 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4618d6d2-9577-4878-a5bf-ada5e67efc61 admin admin] Authenticating user token {{(pid=80245) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:30:06.783627 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be1bfdfe-c5cc-4c78-ac35-34b0ff4128e6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '42d15fc0515945acacbbffcf2356c7a5', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:30:06.784530 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be1bfdfe-c5cc-4c78-ac35-34b0ff4128e6 admin admin] REQUEST_METHOD: `PUT` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:30:06.784811 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be1bfdfe-c5cc-4c78-ac35-34b0ff4128e6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:30:06.785629 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be1bfdfe-c5cc-4c78-ac35-34b0ff4128e6 admin admin] PATH_INFO: `/v3/projects/b4b35ea61009452684dbd4b39032bed1/users/8fff6db7bd92410f8d6c680c9b24f305/roles/3a074738221e4601b5205715ba87d240` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:30:06.789254 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4618d6d2-9577-4878-a5bf-ada5e67efc61 admin admin] Validating token access rules against request {{(pid=80245) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:30:06.793872 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-995e5cbd-06c6-4ff2-8dc5-3579ebe1b7a6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '42d15fc0515945acacbbffcf2356c7a5', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80245) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:30:06.794542 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-995e5cbd-06c6-4ff2-8dc5-3579ebe1b7a6 admin admin] REQUEST_METHOD: `GET` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:30:06.794827 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-995e5cbd-06c6-4ff2-8dc5-3579ebe1b7a6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:30:06.795092 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-995e5cbd-06c6-4ff2-8dc5-3579ebe1b7a6 admin admin] PATH_INFO: `/v3/roles` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:30:06.795851 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-995e5cbd-06c6-4ff2-8dc5-3579ebe1b7a6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:30:06.797983 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-995e5cbd-06c6-4ff2-8dc5-3579ebe1b7a6 admin admin] RBAC: Authorization granted {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:30:06.800818 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be1bfdfe-c5cc-4c78-ac35-34b0ff4128e6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b4b35ea61009452684dbd4b39032bed1, user_id=8fff6db7bd92410f8d6c680c9b24f305, role_id=3a074738221e4601b5205715ba87d240)` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:30:06.803493 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be1bfdfe-c5cc-4c78-ac35-34b0ff4128e6 admin admin] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:30:06.815424 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: [pid: 80245|app: 0|req: 792/1587] 10.222.0.47 () {58 vars in 1121 bytes} [Thu Jul 21 08:30:06 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 08:30:06.815424 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-995e5cbd-06c6-4ff2-8dc5-3579ebe1b7a6 admin admin] Authenticating user token {{(pid=80245) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:30:06.821810 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 796/1588] 10.222.0.47 () {60 vars in 1482 bytes} [Thu Jul 21 08:30:06 2022] PUT /identity/v3/projects/b4b35ea61009452684dbd4b39032bed1/users/8fff6db7bd92410f8d6c680c9b24f305/roles/3a074738221e4601b5205715ba87d240 => generated 0 bytes in 58 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 08:30:06.823236 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfc9411c-2031-4184-af2d-2d64e638b293 None None] REQUEST_METHOD: `POST` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:30:06.823623 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfc9411c-2031-4184-af2d-2d64e638b293 None None] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:30:06.823731 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-995e5cbd-06c6-4ff2-8dc5-3579ebe1b7a6 admin admin] Validating token access rules against request {{(pid=80245) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:30:06.823973 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfc9411c-2031-4184-af2d-2d64e638b293 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:30:06.845036 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.auth.core [None req-cfc9411c-2031-4184-af2d-2d64e638b293 None None] MFA Rules not processed for user `2dce40b8d74c41ec94cc36e6ff05d412`. Rule list: `[]` (Enabled: `True`). {{(pid=80244) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 08:30:06.846242 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.fernet_utils [None req-cfc9411c-2031-4184-af2d-2d64e638b293 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80244) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 08:30:06.883972 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbe42f4c-3af1-4676-98dc-8239a4a70e7b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '42d15fc0515945acacbbffcf2356c7a5', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80245) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:30:06.884658 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbe42f4c-3af1-4676-98dc-8239a4a70e7b admin admin] REQUEST_METHOD: `GET` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:30:06.884943 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbe42f4c-3af1-4676-98dc-8239a4a70e7b admin admin] SCRIPT_NAME: `/identity` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:30:06.885218 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbe42f4c-3af1-4676-98dc-8239a4a70e7b admin admin] PATH_INFO: `/v3/roles` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:30:06.885947 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbe42f4c-3af1-4676-98dc-8239a4a70e7b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:30:06.887580 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbe42f4c-3af1-4676-98dc-8239a4a70e7b admin admin] RBAC: Authorization granted {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:30:06.898089 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: [pid: 80245|app: 0|req: 793/1589] 10.222.0.47 () {58 vars in 1121 bytes} [Thu Jul 21 08:30:06 2022] GET /identity/v3/roles => generated 1420 bytes in 86 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 08:30:06.898895 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbe42f4c-3af1-4676-98dc-8239a4a70e7b admin admin] Authenticating user token {{(pid=80245) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:30:06.908982 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbe42f4c-3af1-4676-98dc-8239a4a70e7b admin admin] Validating token access rules against request {{(pid=80245) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:30:06.909815 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 797/1590] 10.222.0.47 () {58 vars in 957 bytes} [Thu Jul 21 08:30:06 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 08:30:06.910793 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfc9411c-2031-4184-af2d-2d64e638b293 None None] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:30:06.913944 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc716470-6799-4847-be36-86a1dcae8ffc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '42d15fc0515945acacbbffcf2356c7a5', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80245) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:30:06.914604 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc716470-6799-4847-be36-86a1dcae8ffc admin admin] REQUEST_METHOD: `GET` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:30:06.914878 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc716470-6799-4847-be36-86a1dcae8ffc admin admin] SCRIPT_NAME: `/identity` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:30:06.915161 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc716470-6799-4847-be36-86a1dcae8ffc admin admin] PATH_INFO: `/v3/roles` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:30:06.915911 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc716470-6799-4847-be36-86a1dcae8ffc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:30:06.917500 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc716470-6799-4847-be36-86a1dcae8ffc admin admin] RBAC: Authorization granted {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:30:06.920330 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfc9411c-2031-4184-af2d-2d64e638b293 None None] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:30:06.925825 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6e07875-3061-4e58-8bf9-c0b1347d9831 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '42d15fc0515945acacbbffcf2356c7a5', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:30:06.926507 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6e07875-3061-4e58-8bf9-c0b1347d9831 admin admin] REQUEST_METHOD: `GET` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:30:06.926796 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6e07875-3061-4e58-8bf9-c0b1347d9831 admin admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:30:06.927076 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6e07875-3061-4e58-8bf9-c0b1347d9831 admin admin] PATH_INFO: `/v3/roles` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:30:06.930833 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: [pid: 80245|app: 0|req: 794/1591] 10.222.0.47 () {58 vars in 1121 bytes} [Thu Jul 21 08:30:06 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 08:30:06.930970 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6e07875-3061-4e58-8bf9-c0b1347d9831 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:30:06.931082 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc716470-6799-4847-be36-86a1dcae8ffc admin admin] Authenticating user token {{(pid=80245) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:30:06.931187 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6e07875-3061-4e58-8bf9-c0b1347d9831 admin admin] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:30:06.937927 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc716470-6799-4847-be36-86a1dcae8ffc admin admin] Validating token access rules against request {{(pid=80245) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:30:06.939691 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 798/1592] 10.222.0.47 () {58 vars in 1121 bytes} [Thu Jul 21 08:30:06 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 08:30:06.940496 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6e07875-3061-4e58-8bf9-c0b1347d9831 admin admin] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:30:06.943761 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d24bf069-a32d-4922-852b-5ecc77b5646b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '42d15fc0515945acacbbffcf2356c7a5', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80245) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:30:06.944641 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d24bf069-a32d-4922-852b-5ecc77b5646b admin admin] REQUEST_METHOD: `GET` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:30:06.944924 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d24bf069-a32d-4922-852b-5ecc77b5646b admin admin] SCRIPT_NAME: `/identity` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:30:06.945199 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d24bf069-a32d-4922-852b-5ecc77b5646b admin admin] PATH_INFO: `/v3/roles` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:30:06.946013 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d24bf069-a32d-4922-852b-5ecc77b5646b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:30:06.947635 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d24bf069-a32d-4922-852b-5ecc77b5646b admin admin] RBAC: Authorization granted {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:30:06.950399 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6e07875-3061-4e58-8bf9-c0b1347d9831 admin admin] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:30:06.956117 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-644eb980-e0fc-49ba-8723-06dd6d7a8e90 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '42d15fc0515945acacbbffcf2356c7a5', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:30:06.956770 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-644eb980-e0fc-49ba-8723-06dd6d7a8e90 admin admin] REQUEST_METHOD: `GET` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:30:06.957095 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-644eb980-e0fc-49ba-8723-06dd6d7a8e90 admin admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:30:06.957374 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-644eb980-e0fc-49ba-8723-06dd6d7a8e90 admin admin] PATH_INFO: `/v3/roles` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:30:06.958095 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-644eb980-e0fc-49ba-8723-06dd6d7a8e90 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:30:06.958269 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: [pid: 80245|app: 0|req: 795/1593] 10.222.0.47 () {58 vars in 1121 bytes} [Thu Jul 21 08:30:06 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 08:30:06.961386 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d24bf069-a32d-4922-852b-5ecc77b5646b admin admin] Authenticating user token {{(pid=80245) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:30:06.961507 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-644eb980-e0fc-49ba-8723-06dd6d7a8e90 admin admin] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:30:06.969691 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d24bf069-a32d-4922-852b-5ecc77b5646b admin admin] Validating token access rules against request {{(pid=80245) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:30:06.972158 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 799/1594] 10.222.0.47 () {58 vars in 1121 bytes} [Thu Jul 21 08:30:06 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 08:30:06.972158 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d6bf5e3-d35d-4bde-8c34-e63a160c8183 None None] REQUEST_METHOD: `POST` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:30:06.972438 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d6bf5e3-d35d-4bde-8c34-e63a160c8183 None None] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:30:06.972719 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d6bf5e3-d35d-4bde-8c34-e63a160c8183 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:30:06.975276 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41b398b7-72f2-47fb-b39a-96e5cf104d4c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '42d15fc0515945acacbbffcf2356c7a5', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80245) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:30:06.975965 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41b398b7-72f2-47fb-b39a-96e5cf104d4c admin admin] REQUEST_METHOD: `GET` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:30:06.976246 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41b398b7-72f2-47fb-b39a-96e5cf104d4c admin admin] SCRIPT_NAME: `/identity` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:30:06.976524 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41b398b7-72f2-47fb-b39a-96e5cf104d4c admin admin] PATH_INFO: `/v3/roles` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:30:06.977236 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41b398b7-72f2-47fb-b39a-96e5cf104d4c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:30:06.978845 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41b398b7-72f2-47fb-b39a-96e5cf104d4c admin admin] RBAC: Authorization granted {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:30:06.990188 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: [pid: 80245|app: 0|req: 796/1595] 10.222.0.47 () {58 vars in 1121 bytes} [Thu Jul 21 08:30:06 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 08:30:06.991017 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41b398b7-72f2-47fb-b39a-96e5cf104d4c admin admin] Authenticating user token {{(pid=80245) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:30:07.000467 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.auth.core [None req-4d6bf5e3-d35d-4bde-8c34-e63a160c8183 None None] MFA Rules not processed for user `8fff6db7bd92410f8d6c680c9b24f305`. Rule list: `[]` (Enabled: `True`). {{(pid=80244) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 08:30:07.001679 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.fernet_utils [None req-4d6bf5e3-d35d-4bde-8c34-e63a160c8183 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80244) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 08:30:07.002200 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41b398b7-72f2-47fb-b39a-96e5cf104d4c admin admin] Validating token access rules against request {{(pid=80245) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:30:07.007304 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b65060c7-c917-4145-b9ee-e8792eac7ca1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '42d15fc0515945acacbbffcf2356c7a5', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80245) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:30:07.007995 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b65060c7-c917-4145-b9ee-e8792eac7ca1 admin admin] REQUEST_METHOD: `GET` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:30:07.008337 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b65060c7-c917-4145-b9ee-e8792eac7ca1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:30:07.008622 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b65060c7-c917-4145-b9ee-e8792eac7ca1 admin admin] PATH_INFO: `/v3/roles` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:30:07.009380 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b65060c7-c917-4145-b9ee-e8792eac7ca1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:30:07.010970 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b65060c7-c917-4145-b9ee-e8792eac7ca1 admin admin] RBAC: Authorization granted {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:30:07.021880 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: [pid: 80245|app: 0|req: 797/1596] 10.222.0.47 () {58 vars in 1121 bytes} [Thu Jul 21 08:30:06 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 08:30:07.025838 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b65060c7-c917-4145-b9ee-e8792eac7ca1 admin admin] Authenticating user token {{(pid=80245) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:30:07.072171 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 800/1597] 10.222.0.47 () {58 vars in 957 bytes} [Thu Jul 21 08:30:06 2022] POST /identity/v3/auth/tokens => generated 3236 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 08:30:07.072171 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d6bf5e3-d35d-4bde-8c34-e63a160c8183 None None] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:30:07.080211 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d6bf5e3-d35d-4bde-8c34-e63a160c8183 None None] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:30:07.084729 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92103e79-b3bb-4bff-9586-84c3889105fd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '42d15fc0515945acacbbffcf2356c7a5', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:30:07.085252 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b65060c7-c917-4145-b9ee-e8792eac7ca1 admin admin] Validating token access rules against request {{(pid=80245) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:30:07.085382 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92103e79-b3bb-4bff-9586-84c3889105fd admin admin] REQUEST_METHOD: `GET` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:30:07.085657 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92103e79-b3bb-4bff-9586-84c3889105fd admin admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:30:07.085933 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92103e79-b3bb-4bff-9586-84c3889105fd admin admin] PATH_INFO: `/v3/roles` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:30:07.086656 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92103e79-b3bb-4bff-9586-84c3889105fd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:30:07.088316 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92103e79-b3bb-4bff-9586-84c3889105fd admin admin] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:30:07.089020 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a2e43cb-b09a-47bb-9738-5acdc78d2a8e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20ca3c3bd3774691b3374cfbfec6e494', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4faef13aa50e4e36b6c265fb1398fa14', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80245) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:30:07.089663 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a2e43cb-b09a-47bb-9738-5acdc78d2a8e service neutron] REQUEST_METHOD: `GET` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:30:07.089942 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a2e43cb-b09a-47bb-9738-5acdc78d2a8e service neutron] SCRIPT_NAME: `/identity` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:30:07.090218 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a2e43cb-b09a-47bb-9738-5acdc78d2a8e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:30:07.098398 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 801/1598] 10.222.0.47 () {58 vars in 1121 bytes} [Thu Jul 21 08:30:07 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 08:30:07.099291 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92103e79-b3bb-4bff-9586-84c3889105fd admin admin] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:30:07.099796 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a2e43cb-b09a-47bb-9738-5acdc78d2a8e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:30:07.101656 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a2e43cb-b09a-47bb-9738-5acdc78d2a8e service neutron] RBAC: Authorization granted {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:30:07.109739 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92103e79-b3bb-4bff-9586-84c3889105fd admin admin] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:30:07.111682 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: [pid: 80245|app: 0|req: 798/1599] 10.222.0.47 () {60 vars in 1365 bytes} [Thu Jul 21 08:30:07 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 08:30:07.112578 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a2e43cb-b09a-47bb-9738-5acdc78d2a8e service neutron] Authenticating user token {{(pid=80245) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:30:07.114793 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b78c76d-e382-40bf-bb59-128ae2bb1bb7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '42d15fc0515945acacbbffcf2356c7a5', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:30:07.115603 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b78c76d-e382-40bf-bb59-128ae2bb1bb7 admin admin] REQUEST_METHOD: `GET` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:30:07.115963 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b78c76d-e382-40bf-bb59-128ae2bb1bb7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:30:07.116312 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b78c76d-e382-40bf-bb59-128ae2bb1bb7 admin admin] PATH_INFO: `/v3/roles` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:30:07.117298 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b78c76d-e382-40bf-bb59-128ae2bb1bb7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:30:07.119913 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b78c76d-e382-40bf-bb59-128ae2bb1bb7 admin admin] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:30:07.121729 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a2e43cb-b09a-47bb-9738-5acdc78d2a8e service neutron] Validating token access rules against request {{(pid=80245) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:30:07.127057 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba2dd571-f36c-4608-9fa8-7494ae7e1be1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '42d15fc0515945acacbbffcf2356c7a5', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80245) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:30:07.127982 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba2dd571-f36c-4608-9fa8-7494ae7e1be1 admin admin] REQUEST_METHOD: `PUT` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:30:07.128272 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba2dd571-f36c-4608-9fa8-7494ae7e1be1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:30:07.128549 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba2dd571-f36c-4608-9fa8-7494ae7e1be1 admin admin] PATH_INFO: `/v3/projects/84f16ebf991048df944d6c5d23b48195/users/4ec042eca40246e6b32756eafe3ac5bb/roles/3a074738221e4601b5205715ba87d240` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:30:07.131256 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG dbcounter [-] [80245] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80245) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:30:07.136366 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 802/1600] 10.222.0.47 () {58 vars in 1121 bytes} [Thu Jul 21 08:30:07 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 08:30:07.136366 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b78c76d-e382-40bf-bb59-128ae2bb1bb7 admin admin] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:30:07.148105 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b78c76d-e382-40bf-bb59-128ae2bb1bb7 admin admin] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:30:07.148378 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba2dd571-f36c-4608-9fa8-7494ae7e1be1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=84f16ebf991048df944d6c5d23b48195, user_id=4ec042eca40246e6b32756eafe3ac5bb, role_id=3a074738221e4601b5205715ba87d240)` {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:30:07.151165 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba2dd571-f36c-4608-9fa8-7494ae7e1be1 admin admin] RBAC: Authorization granted {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:30:07.152737 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b98fe66a-26ee-4a0b-b9f0-bebf02ad0632 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '42d15fc0515945acacbbffcf2356c7a5', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:30:07.153633 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b98fe66a-26ee-4a0b-b9f0-bebf02ad0632 admin admin] REQUEST_METHOD: `PUT` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:30:07.153919 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b98fe66a-26ee-4a0b-b9f0-bebf02ad0632 admin admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:30:07.154194 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b98fe66a-26ee-4a0b-b9f0-bebf02ad0632 admin admin] PATH_INFO: `/v3/projects/a9d8ff00b3ae4c18971814d983d93e05/users/976a5d6aedea4670827416763642c61d/roles/3a074738221e4601b5205715ba87d240` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:30:07.166803 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: [pid: 80245|app: 0|req: 799/1601] 10.222.0.47 () {60 vars in 1482 bytes} [Thu Jul 21 08:30:07 2022] PUT /identity/v3/projects/84f16ebf991048df944d6c5d23b48195/users/4ec042eca40246e6b32756eafe3ac5bb/roles/3a074738221e4601b5205715ba87d240 => generated 0 bytes in 54 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 08:30:07.167491 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba2dd571-f36c-4608-9fa8-7494ae7e1be1 admin admin] Authenticating user token {{(pid=80245) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:30:07.168538 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG dbcounter [-] [80244] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80244) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:30:07.181748 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b98fe66a-26ee-4a0b-b9f0-bebf02ad0632 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a9d8ff00b3ae4c18971814d983d93e05, user_id=976a5d6aedea4670827416763642c61d, role_id=3a074738221e4601b5205715ba87d240)` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:30:07.184833 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b98fe66a-26ee-4a0b-b9f0-bebf02ad0632 admin admin] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:30:07.200327 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 803/1602] 10.222.0.47 () {60 vars in 1482 bytes} [Thu Jul 21 08:30:07 2022] PUT /identity/v3/projects/a9d8ff00b3ae4c18971814d983d93e05/users/976a5d6aedea4670827416763642c61d/roles/3a074738221e4601b5205715ba87d240 => generated 0 bytes in 67 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 08:30:07.201309 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b98fe66a-26ee-4a0b-b9f0-bebf02ad0632 admin admin] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:30:07.262506 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba2dd571-f36c-4608-9fa8-7494ae7e1be1 admin admin] Validating token access rules against request {{(pid=80245) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:30:07.284199 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b98fe66a-26ee-4a0b-b9f0-bebf02ad0632 admin admin] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:30:07.291100 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4f5cb8b-4436-4cea-bea4-aacf91fb6e51 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '42d15fc0515945acacbbffcf2356c7a5', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:30:07.292558 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4f5cb8b-4436-4cea-bea4-aacf91fb6e51 admin admin] REQUEST_METHOD: `PUT` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:30:07.292850 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4f5cb8b-4436-4cea-bea4-aacf91fb6e51 admin admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:30:07.293214 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4f5cb8b-4436-4cea-bea4-aacf91fb6e51 admin admin] PATH_INFO: `/v3/projects/4af2477817324d4cb2e5e86cb5d3d103/users/8b727a8c76f141b79e9d996019da54a5/roles/3a074738221e4601b5205715ba87d240` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:30:07.309655 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4f5cb8b-4436-4cea-bea4-aacf91fb6e51 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4af2477817324d4cb2e5e86cb5d3d103, user_id=8b727a8c76f141b79e9d996019da54a5, role_id=3a074738221e4601b5205715ba87d240)` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:30:07.313079 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4f5cb8b-4436-4cea-bea4-aacf91fb6e51 admin admin] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:30:07.326333 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71673f8f-d028-475c-b2f5-985b2925266d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '42d15fc0515945acacbbffcf2356c7a5', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80245) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:30:07.326903 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 804/1603] 10.222.0.47 () {60 vars in 1482 bytes} [Thu Jul 21 08:30:07 2022] PUT /identity/v3/projects/4af2477817324d4cb2e5e86cb5d3d103/users/8b727a8c76f141b79e9d996019da54a5/roles/3a074738221e4601b5205715ba87d240 => generated 0 bytes in 126 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 08:30:07.327547 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71673f8f-d028-475c-b2f5-985b2925266d admin admin] REQUEST_METHOD: `PUT` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:30:07.328357 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71673f8f-d028-475c-b2f5-985b2925266d admin admin] SCRIPT_NAME: `/identity` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:30:07.328640 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71673f8f-d028-475c-b2f5-985b2925266d admin admin] PATH_INFO: `/v3/projects/90d95f75c84d445f9ec31e4a0e8d3953/users/2465817c3ba340f4870debf937929499/roles/3a074738221e4601b5205715ba87d240` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:30:07.329323 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4f5cb8b-4436-4cea-bea4-aacf91fb6e51 admin admin] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:30:07.345800 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71673f8f-d028-475c-b2f5-985b2925266d admin admin] RBAC: Authorizing `identity:create_grant(project_id=90d95f75c84d445f9ec31e4a0e8d3953, user_id=2465817c3ba340f4870debf937929499, role_id=3a074738221e4601b5205715ba87d240)` {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:30:07.348542 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71673f8f-d028-475c-b2f5-985b2925266d admin admin] RBAC: Authorization granted {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:30:07.370058 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: [pid: 80245|app: 0|req: 800/1604] 10.222.0.47 () {60 vars in 1482 bytes} [Thu Jul 21 08:30:07 2022] PUT /identity/v3/projects/90d95f75c84d445f9ec31e4a0e8d3953/users/2465817c3ba340f4870debf937929499/roles/3a074738221e4601b5205715ba87d240 => generated 0 bytes in 199 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 08:30:07.370058 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71673f8f-d028-475c-b2f5-985b2925266d admin admin] Authenticating user token {{(pid=80245) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:30:07.424299 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4f5cb8b-4436-4cea-bea4-aacf91fb6e51 admin admin] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:30:07.436040 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71673f8f-d028-475c-b2f5-985b2925266d admin admin] Validating token access rules against request {{(pid=80245) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:30:07.443670 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40eeba0f-0174-432f-af2a-b3a0001f8bba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '42d15fc0515945acacbbffcf2356c7a5', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80245) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:30:07.444732 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40eeba0f-0174-432f-af2a-b3a0001f8bba admin admin] REQUEST_METHOD: `PUT` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:30:07.445031 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40eeba0f-0174-432f-af2a-b3a0001f8bba admin admin] SCRIPT_NAME: `/identity` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:30:07.445372 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40eeba0f-0174-432f-af2a-b3a0001f8bba admin admin] PATH_INFO: `/v3/projects/d0c25b6392a740d9b324c8a780a08830/users/ffe1270415354f1caff259ad7bc7b956/roles/3a074738221e4601b5205715ba87d240` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:30:07.464797 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40eeba0f-0174-432f-af2a-b3a0001f8bba admin admin] RBAC: Authorizing `identity:create_grant(project_id=d0c25b6392a740d9b324c8a780a08830, user_id=ffe1270415354f1caff259ad7bc7b956, role_id=3a074738221e4601b5205715ba87d240)` {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:30:07.467581 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40eeba0f-0174-432f-af2a-b3a0001f8bba admin admin] RBAC: Authorization granted {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:30:07.475664 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG dbcounter [-] [80245] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80245) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:30:07.489235 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: [pid: 80245|app: 0|req: 801/1605] 10.222.0.47 () {60 vars in 1482 bytes} [Thu Jul 21 08:30:07 2022] PUT /identity/v3/projects/d0c25b6392a740d9b324c8a780a08830/users/ffe1270415354f1caff259ad7bc7b956/roles/3a074738221e4601b5205715ba87d240 => generated 0 bytes in 122 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 08:30:07.490700 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40eeba0f-0174-432f-af2a-b3a0001f8bba admin admin] Authenticating user token {{(pid=80245) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:30:07.498953 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d2bf869-0a86-41ec-80f9-de351d1515b5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '42d15fc0515945acacbbffcf2356c7a5', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:30:07.500323 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d2bf869-0a86-41ec-80f9-de351d1515b5 admin admin] REQUEST_METHOD: `POST` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:30:07.500684 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d2bf869-0a86-41ec-80f9-de351d1515b5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:30:07.501084 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d2bf869-0a86-41ec-80f9-de351d1515b5 admin admin] PATH_INFO: `/v3/projects` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:30:07.501855 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d2bf869-0a86-41ec-80f9-de351d1515b5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:30:07.506357 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d2bf869-0a86-41ec-80f9-de351d1515b5 admin admin] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:30:07.538106 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 805/1606] 10.222.0.47 () {60 vars in 1152 bytes} [Thu Jul 21 08:30:07 2022] POST /identity/v3/projects => generated 371 bytes in 211 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 08:30:07.539873 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d6f0b95-95f1-48c9-bfd0-996ec9145b37 None None] REQUEST_METHOD: `POST` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:30:07.540170 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d6f0b95-95f1-48c9-bfd0-996ec9145b37 None None] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:30:07.540456 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d6f0b95-95f1-48c9-bfd0-996ec9145b37 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:30:07.550843 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG dbcounter [-] [80244] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80244) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:30:07.581238 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.auth.core [None req-0d6f0b95-95f1-48c9-bfd0-996ec9145b37 None None] MFA Rules not processed for user `4ec042eca40246e6b32756eafe3ac5bb`. Rule list: `[]` (Enabled: `True`). {{(pid=80244) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 08:30:07.582918 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40eeba0f-0174-432f-af2a-b3a0001f8bba admin admin] Validating token access rules against request {{(pid=80245) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:30:07.583201 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.fernet_utils [None req-0d6f0b95-95f1-48c9-bfd0-996ec9145b37 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80244) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 08:30:07.675727 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9220628b-ace5-4716-971f-e551bf8d2c11 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '42d15fc0515945acacbbffcf2356c7a5', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80245) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:30:07.676618 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 806/1607] 10.222.0.47 () {58 vars in 957 bytes} [Thu Jul 21 08:30:07 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 138 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 08:30:07.677013 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9220628b-ace5-4716-971f-e551bf8d2c11 admin admin] REQUEST_METHOD: `PUT` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:30:07.677511 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9220628b-ace5-4716-971f-e551bf8d2c11 admin admin] SCRIPT_NAME: `/identity` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:30:07.678011 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d580551f-3e8e-4d4b-b3cb-ad80e6298bdf None None] REQUEST_METHOD: `POST` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:30:07.678136 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9220628b-ace5-4716-971f-e551bf8d2c11 admin admin] PATH_INFO: `/v3/projects/b27a1d777abf4d139048716280a4bb4f/users/3528104f3e914d62a10e80e13d3c97de/roles/3a074738221e4601b5205715ba87d240` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:30:07.678304 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d580551f-3e8e-4d4b-b3cb-ad80e6298bdf None None] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:30:07.678572 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d580551f-3e8e-4d4b-b3cb-ad80e6298bdf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:30:07.704047 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9220628b-ace5-4716-971f-e551bf8d2c11 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b27a1d777abf4d139048716280a4bb4f, user_id=3528104f3e914d62a10e80e13d3c97de, role_id=3a074738221e4601b5205715ba87d240)` {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:30:07.707939 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9220628b-ace5-4716-971f-e551bf8d2c11 admin admin] RBAC: Authorization granted {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:30:07.709117 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.auth.core [None req-d580551f-3e8e-4d4b-b3cb-ad80e6298bdf None None] MFA Rules not processed for user `976a5d6aedea4670827416763642c61d`. Rule list: `[]` (Enabled: `True`). {{(pid=80244) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 08:30:07.710332 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.fernet_utils [None req-d580551f-3e8e-4d4b-b3cb-ad80e6298bdf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80244) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 08:30:07.728601 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: [pid: 80245|app: 0|req: 802/1608] 10.222.0.47 () {60 vars in 1482 bytes} [Thu Jul 21 08:30:07 2022] PUT /identity/v3/projects/b27a1d777abf4d139048716280a4bb4f/users/3528104f3e914d62a10e80e13d3c97de/roles/3a074738221e4601b5205715ba87d240 => generated 0 bytes in 239 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 08:30:07.730729 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aef8dca4-c591-4b9a-a2e7-9959aa8895c7 None None] REQUEST_METHOD: `POST` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:30:07.731305 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aef8dca4-c591-4b9a-a2e7-9959aa8895c7 None None] SCRIPT_NAME: `/identity` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:30:07.732591 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aef8dca4-c591-4b9a-a2e7-9959aa8895c7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:30:07.771112 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.auth.core [None req-aef8dca4-c591-4b9a-a2e7-9959aa8895c7 None None] MFA Rules not processed for user `8b727a8c76f141b79e9d996019da54a5`. Rule list: `[]` (Enabled: `True`). {{(pid=80245) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 08:30:07.772315 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.fernet_utils [None req-aef8dca4-c591-4b9a-a2e7-9959aa8895c7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80245) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 08:30:07.792561 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 807/1609] 10.222.0.47 () {58 vars in 957 bytes} [Thu Jul 21 08:30:07 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 08:30:07.795433 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffbc9c9e-537a-49be-b0ff-c33e69c01d2b None None] REQUEST_METHOD: `POST` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:30:07.796934 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffbc9c9e-537a-49be-b0ff-c33e69c01d2b None None] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:30:07.798143 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffbc9c9e-537a-49be-b0ff-c33e69c01d2b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:30:07.828382 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.auth.core [None req-ffbc9c9e-537a-49be-b0ff-c33e69c01d2b None None] MFA Rules not processed for user `2dce40b8d74c41ec94cc36e6ff05d412`. Rule list: `[]` (Enabled: `True`). {{(pid=80244) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 08:30:07.829821 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.fernet_utils [None req-ffbc9c9e-537a-49be-b0ff-c33e69c01d2b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80244) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 08:30:07.872965 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: [pid: 80245|app: 0|req: 803/1610] 10.222.0.47 () {58 vars in 957 bytes} [Thu Jul 21 08:30:07 2022] POST /identity/v3/auth/tokens => generated 3222 bytes in 144 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 08:30:07.877077 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-349b4791-7fcf-4dbf-b73b-86701bdcbab7 None None] REQUEST_METHOD: `POST` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:30:07.877575 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-349b4791-7fcf-4dbf-b73b-86701bdcbab7 None None] SCRIPT_NAME: `/identity` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:30:07.878014 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-349b4791-7fcf-4dbf-b73b-86701bdcbab7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:30:07.910900 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.auth.core [None req-349b4791-7fcf-4dbf-b73b-86701bdcbab7 None None] MFA Rules not processed for user `ffe1270415354f1caff259ad7bc7b956`. Rule list: `[]` (Enabled: `True`). {{(pid=80245) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 08:30:07.912582 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.fernet_utils [None req-349b4791-7fcf-4dbf-b73b-86701bdcbab7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80245) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 08:30:07.917742 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG dbcounter [-] [80245] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80245) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:30:07.927920 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 808/1611] 10.222.0.47 () {58 vars in 957 bytes} [Thu Jul 21 08:30:07 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 134 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 08:30:07.929461 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffbc9c9e-537a-49be-b0ff-c33e69c01d2b None None] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:30:07.954741 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffbc9c9e-537a-49be-b0ff-c33e69c01d2b None None] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:30:07.966071 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ad91642-1aac-4b4b-9b65-e7495734d14f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '42d15fc0515945acacbbffcf2356c7a5', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:30:07.967232 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ad91642-1aac-4b4b-9b65-e7495734d14f admin admin] REQUEST_METHOD: `POST` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:30:07.967748 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ad91642-1aac-4b4b-9b65-e7495734d14f admin admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:30:07.969009 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ad91642-1aac-4b4b-9b65-e7495734d14f admin admin] PATH_INFO: `/v3/users` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:30:07.970099 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ad91642-1aac-4b4b-9b65-e7495734d14f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:30:07.972914 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ad91642-1aac-4b4b-9b65-e7495734d14f admin admin] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:30:08.002546 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 809/1612] 10.222.0.47 () {60 vars in 1143 bytes} [Thu Jul 21 08:30:07 2022] POST /identity/v3/users => generated 292 bytes in 74 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 08:30:08.003667 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ad91642-1aac-4b4b-9b65-e7495734d14f admin admin] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:30:08.015438 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ad91642-1aac-4b4b-9b65-e7495734d14f admin admin] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:30:08.021613 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adc9d2e6-5c70-4834-886f-eadb64bbefd5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '42d15fc0515945acacbbffcf2356c7a5', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:30:08.022295 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adc9d2e6-5c70-4834-886f-eadb64bbefd5 admin admin] REQUEST_METHOD: `POST` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:30:08.022574 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adc9d2e6-5c70-4834-886f-eadb64bbefd5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:30:08.023579 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adc9d2e6-5c70-4834-886f-eadb64bbefd5 admin admin] PATH_INFO: `/v3/projects` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:30:08.024289 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adc9d2e6-5c70-4834-886f-eadb64bbefd5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:30:08.025595 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: [pid: 80245|app: 0|req: 804/1613] 10.222.0.47 () {58 vars in 957 bytes} [Thu Jul 21 08:30:07 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 151 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 08:30:08.026500 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adc9d2e6-5c70-4834-886f-eadb64bbefd5 admin admin] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:30:08.028410 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac91c1f9-f727-4b67-81f8-2cef23294e0b None None] REQUEST_METHOD: `POST` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:30:08.029009 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac91c1f9-f727-4b67-81f8-2cef23294e0b None None] SCRIPT_NAME: `/identity` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:30:08.029497 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac91c1f9-f727-4b67-81f8-2cef23294e0b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:30:08.031342 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG dbcounter [-] [80244] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80244) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:30:08.055116 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.auth.core [None req-ac91c1f9-f727-4b67-81f8-2cef23294e0b None None] MFA Rules not processed for user `2dce40b8d74c41ec94cc36e6ff05d412`. Rule list: `[]` (Enabled: `True`). {{(pid=80245) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 08:30:08.056802 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.fernet_utils [None req-ac91c1f9-f727-4b67-81f8-2cef23294e0b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80245) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 08:30:08.062411 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 810/1614] 10.222.0.47 () {60 vars in 1152 bytes} [Thu Jul 21 08:30:08 2022] POST /identity/v3/projects => generated 385 bytes in 59 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 08:30:08.063852 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acea0841-fd00-41d0-987b-22c0cd30670f None None] REQUEST_METHOD: `GET` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:30:08.064131 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acea0841-fd00-41d0-987b-22c0cd30670f None None] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:30:08.064408 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acea0841-fd00-41d0-987b-22c0cd30670f None None] PATH_INFO: `/` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:30:08.066571 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 811/1615] 10.222.0.47 () {54 vars in 962 bytes} [Thu Jul 21 08:30:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 08:30:08.067362 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acea0841-fd00-41d0-987b-22c0cd30670f None None] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:30:08.072844 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: [pid: 80245|app: 0|req: 805/1616] 10.222.0.47 () {58 vars in 957 bytes} [Thu Jul 21 08:30:08 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 44 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 08:30:08.072844 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac91c1f9-f727-4b67-81f8-2cef23294e0b None None] Authenticating user token {{(pid=80245) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:30:08.154123 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac91c1f9-f727-4b67-81f8-2cef23294e0b None None] Validating token access rules against request {{(pid=80245) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:30:08.169845 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acea0841-fd00-41d0-987b-22c0cd30670f None None] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:30:08.177942 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-979588e0-7c7b-4f1e-b693-f289bf04db5d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '42d15fc0515945acacbbffcf2356c7a5', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:30:08.179268 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-979588e0-7c7b-4f1e-b693-f289bf04db5d admin admin] REQUEST_METHOD: `POST` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:30:08.179731 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-979588e0-7c7b-4f1e-b693-f289bf04db5d admin admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:30:08.180314 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-979588e0-7c7b-4f1e-b693-f289bf04db5d admin admin] PATH_INFO: `/v3/projects` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:30:08.181357 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-979588e0-7c7b-4f1e-b693-f289bf04db5d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:30:08.188080 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-979588e0-7c7b-4f1e-b693-f289bf04db5d admin admin] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:30:08.213978 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36c17260-0f4e-4fd2-a4d9-77a24c3709e6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20ca3c3bd3774691b3374cfbfec6e494', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4faef13aa50e4e36b6c265fb1398fa14', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80245) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:30:08.215118 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36c17260-0f4e-4fd2-a4d9-77a24c3709e6 service neutron] REQUEST_METHOD: `GET` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:30:08.215467 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36c17260-0f4e-4fd2-a4d9-77a24c3709e6 service neutron] SCRIPT_NAME: `/identity` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:30:08.215831 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36c17260-0f4e-4fd2-a4d9-77a24c3709e6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:30:08.230737 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 812/1617] 10.222.0.47 () {60 vars in 1152 bytes} [Thu Jul 21 08:30:08 2022] POST /identity/v3/projects => generated 355 bytes in 163 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 08:30:08.231691 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-979588e0-7c7b-4f1e-b693-f289bf04db5d admin admin] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:30:08.245908 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36c17260-0f4e-4fd2-a4d9-77a24c3709e6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:30:08.247391 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36c17260-0f4e-4fd2-a4d9-77a24c3709e6 service neutron] RBAC: Authorization granted {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:30:08.259268 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: [pid: 80245|app: 0|req: 806/1618] 10.222.0.47 () {60 vars in 1365 bytes} [Thu Jul 21 08:30:08 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 188 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 08:30:08.259748 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36c17260-0f4e-4fd2-a4d9-77a24c3709e6 service neutron] Authenticating user token {{(pid=80245) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:30:08.270108 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36c17260-0f4e-4fd2-a4d9-77a24c3709e6 service neutron] Validating token access rules against request {{(pid=80245) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:30:08.342975 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG dbcounter [-] [80245] Writing DB stats keystone:SELECT=100 {{(pid=80245) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:30:08.353777 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b1c90b5-77c2-4cd9-8ffa-733eac4cd878 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '42d15fc0515945acacbbffcf2356c7a5', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80245) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:30:08.354469 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b1c90b5-77c2-4cd9-8ffa-733eac4cd878 admin admin] REQUEST_METHOD: `POST` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:30:08.354955 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b1c90b5-77c2-4cd9-8ffa-733eac4cd878 admin admin] SCRIPT_NAME: `/identity` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:30:08.355087 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b1c90b5-77c2-4cd9-8ffa-733eac4cd878 admin admin] PATH_INFO: `/v3/projects` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:30:08.355828 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b1c90b5-77c2-4cd9-8ffa-733eac4cd878 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:30:08.358646 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b1c90b5-77c2-4cd9-8ffa-733eac4cd878 admin admin] RBAC: Authorization granted {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:30:08.376785 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-979588e0-7c7b-4f1e-b693-f289bf04db5d admin admin] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:30:08.383014 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c0533cd-ca83-4d0f-9be4-d7da98357a9c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '42d15fc0515945acacbbffcf2356c7a5', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:30:08.383894 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c0533cd-ca83-4d0f-9be4-d7da98357a9c admin admin] REQUEST_METHOD: `GET` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:30:08.384190 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c0533cd-ca83-4d0f-9be4-d7da98357a9c admin admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:30:08.384519 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c0533cd-ca83-4d0f-9be4-d7da98357a9c admin admin] PATH_INFO: `/v3/roles` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:30:08.385306 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c0533cd-ca83-4d0f-9be4-d7da98357a9c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:30:08.387204 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c0533cd-ca83-4d0f-9be4-d7da98357a9c admin admin] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:30:08.399740 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 813/1619] 10.222.0.47 () {58 vars in 1121 bytes} [Thu Jul 21 08:30:08 2022] GET /identity/v3/roles => generated 1420 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 08:30:08.404495 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: [pid: 80245|app: 0|req: 807/1620] 10.222.0.47 () {60 vars in 1152 bytes} [Thu Jul 21 08:30:08 2022] POST /identity/v3/projects => generated 369 bytes in 145 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 08:30:08.406218 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b1c90b5-77c2-4cd9-8ffa-733eac4cd878 admin admin] Authenticating user token {{(pid=80245) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:30:08.407385 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-998f6051-f633-4fac-8de6-74a4f7786425 None None] REQUEST_METHOD: `POST` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:30:08.407519 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-998f6051-f633-4fac-8de6-74a4f7786425 None None] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:30:08.407762 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-998f6051-f633-4fac-8de6-74a4f7786425 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:30:08.495476 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.auth.core [None req-998f6051-f633-4fac-8de6-74a4f7786425 None None] MFA Rules not processed for user `ac2b5b99a36442148fe743269054e991`. Rule list: `[]` (Enabled: `True`). {{(pid=80244) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 08:30:08.496955 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.fernet_utils [None req-998f6051-f633-4fac-8de6-74a4f7786425 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80244) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 08:30:08.522714 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b1c90b5-77c2-4cd9-8ffa-733eac4cd878 admin admin] Validating token access rules against request {{(pid=80245) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:30:08.530678 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e632e8c6-ec9b-4c8a-bedb-193c002b5701 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '42d15fc0515945acacbbffcf2356c7a5', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80245) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:30:08.531600 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e632e8c6-ec9b-4c8a-bedb-193c002b5701 admin admin] REQUEST_METHOD: `POST` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:30:08.531919 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e632e8c6-ec9b-4c8a-bedb-193c002b5701 admin admin] SCRIPT_NAME: `/identity` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:30:08.532294 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e632e8c6-ec9b-4c8a-bedb-193c002b5701 admin admin] PATH_INFO: `/v3/users` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:30:08.533390 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e632e8c6-ec9b-4c8a-bedb-193c002b5701 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:30:08.536233 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e632e8c6-ec9b-4c8a-bedb-193c002b5701 admin admin] RBAC: Authorization granted {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:30:08.569120 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: [pid: 80245|app: 0|req: 808/1621] 10.222.0.47 () {60 vars in 1143 bytes} [Thu Jul 21 08:30:08 2022] POST /identity/v3/users => generated 299 bytes in 164 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 08:30:08.570388 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e632e8c6-ec9b-4c8a-bedb-193c002b5701 admin admin] Authenticating user token {{(pid=80245) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:30:08.601486 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG dbcounter [-] [80244] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80244) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:30:08.650385 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 814/1622] 10.222.0.47 () {58 vars in 1059 bytes} [Thu Jul 21 08:30:08 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 244 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 08:30:08.653182 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-998f6051-f633-4fac-8de6-74a4f7786425 None None] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:30:08.677231 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-998f6051-f633-4fac-8de6-74a4f7786425 None None] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:30:08.677945 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e632e8c6-ec9b-4c8a-bedb-193c002b5701 admin admin] Validating token access rules against request {{(pid=80245) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:30:08.685147 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24745444-d436-4a4c-b4fb-d2b10ff977c9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20ca3c3bd3774691b3374cfbfec6e494', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4faef13aa50e4e36b6c265fb1398fa14', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80245) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:30:08.686644 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24745444-d436-4a4c-b4fb-d2b10ff977c9 service neutron] REQUEST_METHOD: `GET` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:30:08.688176 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a573140-1b91-4d67-a69a-7f4d87ced373 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '42d15fc0515945acacbbffcf2356c7a5', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:30:08.689484 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a573140-1b91-4d67-a69a-7f4d87ced373 admin admin] REQUEST_METHOD: `POST` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:30:08.689484 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a573140-1b91-4d67-a69a-7f4d87ced373 admin admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:30:08.690005 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24745444-d436-4a4c-b4fb-d2b10ff977c9 service neutron] SCRIPT_NAME: `/identity` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:30:08.690334 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24745444-d436-4a4c-b4fb-d2b10ff977c9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:30:08.690726 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a573140-1b91-4d67-a69a-7f4d87ced373 admin admin] PATH_INFO: `/v3/users` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:30:08.693208 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a573140-1b91-4d67-a69a-7f4d87ced373 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:30:08.695310 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a573140-1b91-4d67-a69a-7f4d87ced373 admin admin] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:30:08.725683 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24745444-d436-4a4c-b4fb-d2b10ff977c9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:30:08.728142 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24745444-d436-4a4c-b4fb-d2b10ff977c9 service neutron] RBAC: Authorization granted {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:30:08.737072 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 815/1623] 10.222.0.47 () {60 vars in 1143 bytes} [Thu Jul 21 08:30:08 2022] POST /identity/v3/users => generated 284 bytes in 86 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 08:30:08.738419 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a573140-1b91-4d67-a69a-7f4d87ced373 admin admin] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:30:08.763827 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a573140-1b91-4d67-a69a-7f4d87ced373 admin admin] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:30:08.769676 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db3f8a8e-94d5-4f3d-a3a8-2653324637a2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '42d15fc0515945acacbbffcf2356c7a5', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:30:08.770375 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db3f8a8e-94d5-4f3d-a3a8-2653324637a2 admin admin] REQUEST_METHOD: `PUT` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:30:08.770721 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db3f8a8e-94d5-4f3d-a3a8-2653324637a2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:30:08.771062 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db3f8a8e-94d5-4f3d-a3a8-2653324637a2 admin admin] PATH_INFO: `/v3/projects/85733045af3649ca8b35ffd2eb635bae/users/78013433f60b4d3680e01b7fbaba39b0/roles/e29870f6e9e2411a9db0d5bf45f32ca5` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:30:08.773301 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: [pid: 80245|app: 0|req: 809/1624] 10.222.0.47 () {60 vars in 1365 bytes} [Thu Jul 21 08:30:08 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 204 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 08:30:08.776330 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24745444-d436-4a4c-b4fb-d2b10ff977c9 service neutron] Authenticating user token {{(pid=80245) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:30:08.791438 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db3f8a8e-94d5-4f3d-a3a8-2653324637a2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=85733045af3649ca8b35ffd2eb635bae, user_id=78013433f60b4d3680e01b7fbaba39b0, role_id=e29870f6e9e2411a9db0d5bf45f32ca5)` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:30:08.794274 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24745444-d436-4a4c-b4fb-d2b10ff977c9 service neutron] Validating token access rules against request {{(pid=80245) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:30:08.796213 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db3f8a8e-94d5-4f3d-a3a8-2653324637a2 admin admin] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:30:08.801811 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a53ca84-7edf-4ab8-82c1-10c7b212432c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '42d15fc0515945acacbbffcf2356c7a5', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80245) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:30:08.802810 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a53ca84-7edf-4ab8-82c1-10c7b212432c admin admin] REQUEST_METHOD: `POST` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:30:08.803365 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a53ca84-7edf-4ab8-82c1-10c7b212432c admin admin] SCRIPT_NAME: `/identity` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:30:08.803936 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a53ca84-7edf-4ab8-82c1-10c7b212432c admin admin] PATH_INFO: `/v3/users` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:30:08.804970 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a53ca84-7edf-4ab8-82c1-10c7b212432c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:30:08.807478 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a53ca84-7edf-4ab8-82c1-10c7b212432c admin admin] RBAC: Authorization granted {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:30:08.819158 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 816/1625] 10.222.0.47 () {60 vars in 1482 bytes} [Thu Jul 21 08:30:08 2022] PUT /identity/v3/projects/85733045af3649ca8b35ffd2eb635bae/users/78013433f60b4d3680e01b7fbaba39b0/roles/e29870f6e9e2411a9db0d5bf45f32ca5 => generated 0 bytes in 79 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 08:30:08.819158 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db3f8a8e-94d5-4f3d-a3a8-2653324637a2 admin admin] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:30:08.856601 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: [pid: 80245|app: 0|req: 810/1626] 10.222.0.47 () {60 vars in 1143 bytes} [Thu Jul 21 08:30:08 2022] POST /identity/v3/users => generated 291 bytes in 81 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 08:30:08.858645 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a53ca84-7edf-4ab8-82c1-10c7b212432c admin admin] Authenticating user token {{(pid=80245) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:30:08.890717 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a53ca84-7edf-4ab8-82c1-10c7b212432c admin admin] Validating token access rules against request {{(pid=80245) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:30:08.925924 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db3f8a8e-94d5-4f3d-a3a8-2653324637a2 admin admin] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:30:08.935251 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-115cdab8-d22e-47e4-bcc6-0175406fa1b0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '42d15fc0515945acacbbffcf2356c7a5', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:30:08.935898 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-115cdab8-d22e-47e4-bcc6-0175406fa1b0 admin admin] REQUEST_METHOD: `GET` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:30:08.936179 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-115cdab8-d22e-47e4-bcc6-0175406fa1b0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:30:08.936452 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-115cdab8-d22e-47e4-bcc6-0175406fa1b0 admin admin] PATH_INFO: `/v3/roles` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:30:08.937549 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-115cdab8-d22e-47e4-bcc6-0175406fa1b0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:30:08.939205 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-115cdab8-d22e-47e4-bcc6-0175406fa1b0 admin admin] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:30:08.958585 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 817/1627] 10.222.0.47 () {58 vars in 1121 bytes} [Thu Jul 21 08:30:08 2022] GET /identity/v3/roles => generated 1420 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 08:30:08.959596 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-115cdab8-d22e-47e4-bcc6-0175406fa1b0 admin admin] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:30:08.975187 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-115cdab8-d22e-47e4-bcc6-0175406fa1b0 admin admin] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:30:08.983950 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1846dd7-a72b-470c-b42b-c619b1bd9d96 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '42d15fc0515945acacbbffcf2356c7a5', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:30:08.984639 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1846dd7-a72b-470c-b42b-c619b1bd9d96 admin admin] REQUEST_METHOD: `GET` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:30:08.985457 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1846dd7-a72b-470c-b42b-c619b1bd9d96 admin admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:30:08.985743 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1846dd7-a72b-470c-b42b-c619b1bd9d96 admin admin] PATH_INFO: `/v3/roles` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:30:08.987057 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1846dd7-a72b-470c-b42b-c619b1bd9d96 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:30:08.991339 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1846dd7-a72b-470c-b42b-c619b1bd9d96 admin admin] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:30:09.011196 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 818/1628] 10.222.0.47 () {58 vars in 1121 bytes} [Thu Jul 21 08:30:08 2022] GET /identity/v3/roles => generated 1420 bytes in 49 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 08:30:09.011196 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1846dd7-a72b-470c-b42b-c619b1bd9d96 admin admin] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:30:09.011786 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG dbcounter [-] [80245] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=80245) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:30:09.029182 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1846dd7-a72b-470c-b42b-c619b1bd9d96 admin admin] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:30:09.040596 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef91cf42-7ee6-487b-a5bf-3701bac52eb2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '42d15fc0515945acacbbffcf2356c7a5', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:30:09.043377 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef91cf42-7ee6-487b-a5bf-3701bac52eb2 admin admin] REQUEST_METHOD: `GET` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:30:09.045447 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef91cf42-7ee6-487b-a5bf-3701bac52eb2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:30:09.047024 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef91cf42-7ee6-487b-a5bf-3701bac52eb2 admin admin] PATH_INFO: `/v3/roles` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:30:09.047858 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef91cf42-7ee6-487b-a5bf-3701bac52eb2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:30:09.052943 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef91cf42-7ee6-487b-a5bf-3701bac52eb2 admin admin] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:30:09.053568 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-103eccb2-1f14-43d9-9275-5d42f8328ad0 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2b5b99a36442148fe743269054e991', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4faef13aa50e4e36b6c265fb1398fa14', 'project_domain_id': 'default', 'roles': ['member', 'service', 'ResellerAdmin', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80245) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:30:09.054695 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-103eccb2-1f14-43d9-9275-5d42f8328ad0 service nova] REQUEST_METHOD: `GET` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:30:09.056371 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-103eccb2-1f14-43d9-9275-5d42f8328ad0 service nova] SCRIPT_NAME: `/identity` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:30:09.057669 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-103eccb2-1f14-43d9-9275-5d42f8328ad0 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:30:09.076860 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 819/1629] 10.222.0.47 () {58 vars in 1121 bytes} [Thu Jul 21 08:30:09 2022] GET /identity/v3/roles => generated 1420 bytes in 68 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 08:30:09.078344 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef91cf42-7ee6-487b-a5bf-3701bac52eb2 admin admin] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:30:09.092271 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef91cf42-7ee6-487b-a5bf-3701bac52eb2 admin admin] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:30:09.100270 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-935fcb64-74e5-4d12-9d32-698979344091 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '42d15fc0515945acacbbffcf2356c7a5', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:30:09.101497 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-935fcb64-74e5-4d12-9d32-698979344091 admin admin] REQUEST_METHOD: `GET` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:30:09.101497 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-935fcb64-74e5-4d12-9d32-698979344091 admin admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:30:09.101709 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-935fcb64-74e5-4d12-9d32-698979344091 admin admin] PATH_INFO: `/v3/roles` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:30:09.102332 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-935fcb64-74e5-4d12-9d32-698979344091 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:30:09.104244 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-935fcb64-74e5-4d12-9d32-698979344091 admin admin] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:30:09.155094 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 820/1630] 10.222.0.47 () {58 vars in 1121 bytes} [Thu Jul 21 08:30:09 2022] GET /identity/v3/roles => generated 1420 bytes in 77 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 08:30:09.156662 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-935fcb64-74e5-4d12-9d32-698979344091 admin admin] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:30:09.169813 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-935fcb64-74e5-4d12-9d32-698979344091 admin admin] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:30:09.190727 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c7d3efb-ea2f-46c1-b017-25ff690baff7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '42d15fc0515945acacbbffcf2356c7a5', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:30:09.194218 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c7d3efb-ea2f-46c1-b017-25ff690baff7 admin admin] REQUEST_METHOD: `PUT` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:30:09.194218 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c7d3efb-ea2f-46c1-b017-25ff690baff7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:30:09.194218 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c7d3efb-ea2f-46c1-b017-25ff690baff7 admin admin] PATH_INFO: `/v3/projects/683e37d66a3b4eec8dd3265f31353d6b/users/e641e0ec780e4538ad00b44ede4e6438/roles/e29870f6e9e2411a9db0d5bf45f32ca5` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:30:09.242905 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-103eccb2-1f14-43d9-9275-5d42f8328ad0 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:30:09.246904 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-103eccb2-1f14-43d9-9275-5d42f8328ad0 service nova] RBAC: Authorization granted {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:30:09.249865 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c7d3efb-ea2f-46c1-b017-25ff690baff7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=683e37d66a3b4eec8dd3265f31353d6b, user_id=e641e0ec780e4538ad00b44ede4e6438, role_id=e29870f6e9e2411a9db0d5bf45f32ca5)` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:30:09.252956 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c7d3efb-ea2f-46c1-b017-25ff690baff7 admin admin] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:30:09.268178 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: [pid: 80245|app: 0|req: 811/1631] 10.222.0.47 () {60 vars in 1365 bytes} [Thu Jul 21 08:30:08 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 411 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 08:30:09.270200 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-103eccb2-1f14-43d9-9275-5d42f8328ad0 service nova] Authenticating user token {{(pid=80245) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:30:09.276769 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 821/1632] 10.222.0.47 () {60 vars in 1482 bytes} [Thu Jul 21 08:30:09 2022] PUT /identity/v3/projects/683e37d66a3b4eec8dd3265f31353d6b/users/e641e0ec780e4538ad00b44ede4e6438/roles/e29870f6e9e2411a9db0d5bf45f32ca5 => generated 0 bytes in 118 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 08:30:09.276769 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c7d3efb-ea2f-46c1-b017-25ff690baff7 admin admin] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:30:09.283500 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-103eccb2-1f14-43d9-9275-5d42f8328ad0 service nova] Validating token access rules against request {{(pid=80245) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:30:09.382152 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG dbcounter [-] [80244] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80244) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:30:09.410892 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c7d3efb-ea2f-46c1-b017-25ff690baff7 admin admin] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:30:09.415387 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32827e22-74c4-4ae7-ae6f-12d053e0d2b3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '42d15fc0515945acacbbffcf2356c7a5', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80245) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:30:09.417427 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32827e22-74c4-4ae7-ae6f-12d053e0d2b3 admin admin] REQUEST_METHOD: `PUT` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:30:09.417771 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32827e22-74c4-4ae7-ae6f-12d053e0d2b3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:30:09.418113 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32827e22-74c4-4ae7-ae6f-12d053e0d2b3 admin admin] PATH_INFO: `/v3/projects/a0ca7cbc167d47a8b5e26af30f6f7fa1/users/21e4296be8d245aa847b55c655d403c5/roles/e29870f6e9e2411a9db0d5bf45f32ca5` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:30:09.418570 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df785cd4-15d3-456d-9f78-18e36b9d64bf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '42d15fc0515945acacbbffcf2356c7a5', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:30:09.420484 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df785cd4-15d3-456d-9f78-18e36b9d64bf admin admin] REQUEST_METHOD: `PUT` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:30:09.420880 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df785cd4-15d3-456d-9f78-18e36b9d64bf admin admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:30:09.421800 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df785cd4-15d3-456d-9f78-18e36b9d64bf admin admin] PATH_INFO: `/v3/projects/85733045af3649ca8b35ffd2eb635bae/users/78013433f60b4d3680e01b7fbaba39b0/roles/e29870f6e9e2411a9db0d5bf45f32ca5` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:30:09.424783 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df785cd4-15d3-456d-9f78-18e36b9d64bf admin admin] RBAC: Authorizing `identity:create_grant(project_id=85733045af3649ca8b35ffd2eb635bae, user_id=78013433f60b4d3680e01b7fbaba39b0, role_id=e29870f6e9e2411a9db0d5bf45f32ca5)` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:30:09.428175 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df785cd4-15d3-456d-9f78-18e36b9d64bf admin admin] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:30:09.444067 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 822/1633] 10.222.0.47 () {60 vars in 1482 bytes} [Thu Jul 21 08:30:09 2022] PUT /identity/v3/projects/85733045af3649ca8b35ffd2eb635bae/users/78013433f60b4d3680e01b7fbaba39b0/roles/e29870f6e9e2411a9db0d5bf45f32ca5 => generated 0 bytes in 169 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 08:30:09.445388 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df785cd4-15d3-456d-9f78-18e36b9d64bf admin admin] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:30:09.463927 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32827e22-74c4-4ae7-ae6f-12d053e0d2b3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a0ca7cbc167d47a8b5e26af30f6f7fa1, user_id=21e4296be8d245aa847b55c655d403c5, role_id=e29870f6e9e2411a9db0d5bf45f32ca5)` {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:30:09.470599 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32827e22-74c4-4ae7-ae6f-12d053e0d2b3 admin admin] RBAC: Authorization granted {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:30:09.505979 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: [pid: 80245|app: 0|req: 812/1634] 10.222.0.47 () {60 vars in 1482 bytes} [Thu Jul 21 08:30:09 2022] PUT /identity/v3/projects/a0ca7cbc167d47a8b5e26af30f6f7fa1/users/21e4296be8d245aa847b55c655d403c5/roles/e29870f6e9e2411a9db0d5bf45f32ca5 => generated 0 bytes in 237 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 08:30:09.510508 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32827e22-74c4-4ae7-ae6f-12d053e0d2b3 admin admin] Authenticating user token {{(pid=80245) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:30:09.571920 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df785cd4-15d3-456d-9f78-18e36b9d64bf admin admin] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:30:09.669590 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32827e22-74c4-4ae7-ae6f-12d053e0d2b3 admin admin] Validating token access rules against request {{(pid=80245) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:30:09.676213 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6dbeb67-7f5f-43a0-83dd-6dd7ec204391 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '42d15fc0515945acacbbffcf2356c7a5', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80245) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:30:09.676428 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6dbeb67-7f5f-43a0-83dd-6dd7ec204391 admin admin] REQUEST_METHOD: `GET` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:30:09.676707 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6dbeb67-7f5f-43a0-83dd-6dd7ec204391 admin admin] SCRIPT_NAME: `/identity` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:30:09.677058 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6dbeb67-7f5f-43a0-83dd-6dd7ec204391 admin admin] PATH_INFO: `/v3/roles` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:30:09.678141 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6dbeb67-7f5f-43a0-83dd-6dd7ec204391 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:30:09.679870 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6dbeb67-7f5f-43a0-83dd-6dd7ec204391 admin admin] RBAC: Authorization granted {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:30:09.692684 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: [pid: 80245|app: 0|req: 813/1635] 10.222.0.47 () {58 vars in 1121 bytes} [Thu Jul 21 08:30:09 2022] GET /identity/v3/roles => generated 1420 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 08:30:09.694081 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5128ece-3d60-414c-93f8-b1267d260f5d None None] REQUEST_METHOD: `POST` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:30:09.694298 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5128ece-3d60-414c-93f8-b1267d260f5d None None] SCRIPT_NAME: `/identity` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:30:09.694577 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5128ece-3d60-414c-93f8-b1267d260f5d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:30:09.702338 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11ceb748-7c2d-4414-942a-c11ff164d194 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '42d15fc0515945acacbbffcf2356c7a5', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:30:09.703712 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11ceb748-7c2d-4414-942a-c11ff164d194 admin admin] REQUEST_METHOD: `PUT` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:30:09.703999 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11ceb748-7c2d-4414-942a-c11ff164d194 admin admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:30:09.704270 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11ceb748-7c2d-4414-942a-c11ff164d194 admin admin] PATH_INFO: `/v3/projects/ca42afd27a3a43fea040e9bad44c8ba9/users/39a796af3051487f8d97c207896e7e97/roles/e29870f6e9e2411a9db0d5bf45f32ca5` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:30:09.728600 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11ceb748-7c2d-4414-942a-c11ff164d194 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ca42afd27a3a43fea040e9bad44c8ba9, user_id=39a796af3051487f8d97c207896e7e97, role_id=e29870f6e9e2411a9db0d5bf45f32ca5)` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:30:09.732084 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11ceb748-7c2d-4414-942a-c11ff164d194 admin admin] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:30:09.733515 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG dbcounter [-] [80245] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80245) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:30:09.738230 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.auth.core [None req-e5128ece-3d60-414c-93f8-b1267d260f5d None None] MFA Rules not processed for user `78013433f60b4d3680e01b7fbaba39b0`. Rule list: `[]` (Enabled: `True`). {{(pid=80245) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 08:30:09.741306 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.fernet_utils [None req-e5128ece-3d60-414c-93f8-b1267d260f5d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80245) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 08:30:09.761933 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 823/1636] 10.222.0.47 () {60 vars in 1482 bytes} [Thu Jul 21 08:30:09 2022] PUT /identity/v3/projects/ca42afd27a3a43fea040e9bad44c8ba9/users/39a796af3051487f8d97c207896e7e97/roles/e29870f6e9e2411a9db0d5bf45f32ca5 => generated 0 bytes in 314 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 08:30:09.763318 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11ceb748-7c2d-4414-942a-c11ff164d194 admin admin] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:30:09.919076 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: [pid: 80245|app: 0|req: 814/1637] 10.222.0.47 () {58 vars in 957 bytes} [Thu Jul 21 08:30:09 2022] POST /identity/v3/auth/tokens => generated 3305 bytes in 225 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 08:30:09.919805 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5128ece-3d60-414c-93f8-b1267d260f5d None None] Authenticating user token {{(pid=80245) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:30:09.935032 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11ceb748-7c2d-4414-942a-c11ff164d194 admin admin] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:30:09.947427 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64eeb41e-82ec-4283-a7c8-d396ea98973a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '42d15fc0515945acacbbffcf2356c7a5', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:30:09.948480 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64eeb41e-82ec-4283-a7c8-d396ea98973a admin admin] REQUEST_METHOD: `GET` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:30:09.948757 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64eeb41e-82ec-4283-a7c8-d396ea98973a admin admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:30:09.949038 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64eeb41e-82ec-4283-a7c8-d396ea98973a admin admin] PATH_INFO: `/v3/roles` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:30:09.951159 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64eeb41e-82ec-4283-a7c8-d396ea98973a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:30:09.958480 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64eeb41e-82ec-4283-a7c8-d396ea98973a admin admin] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:30:09.959562 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5128ece-3d60-414c-93f8-b1267d260f5d None None] Validating token access rules against request {{(pid=80245) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:30:09.972140 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67024bd2-8668-4ae2-9df9-de10fe22a8c8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '42d15fc0515945acacbbffcf2356c7a5', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80245) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:30:09.975000 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67024bd2-8668-4ae2-9df9-de10fe22a8c8 admin admin] REQUEST_METHOD: `PUT` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:30:09.975419 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67024bd2-8668-4ae2-9df9-de10fe22a8c8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:30:09.977890 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 824/1638] 10.222.0.47 () {58 vars in 1121 bytes} [Thu Jul 21 08:30:09 2022] GET /identity/v3/roles => generated 1420 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 08:30:09.978342 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67024bd2-8668-4ae2-9df9-de10fe22a8c8 admin admin] PATH_INFO: `/v3/projects/683e37d66a3b4eec8dd3265f31353d6b/users/e641e0ec780e4538ad00b44ede4e6438/roles/e29870f6e9e2411a9db0d5bf45f32ca5` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:30:09.979264 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f58edfc-7501-4784-a4a7-fae1e44963c9 None None] REQUEST_METHOD: `GET` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:30:09.979584 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f58edfc-7501-4784-a4a7-fae1e44963c9 None None] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:30:09.979904 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f58edfc-7501-4784-a4a7-fae1e44963c9 None None] PATH_INFO: `/` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:30:09.983359 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67024bd2-8668-4ae2-9df9-de10fe22a8c8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=683e37d66a3b4eec8dd3265f31353d6b, user_id=e641e0ec780e4538ad00b44ede4e6438, role_id=e29870f6e9e2411a9db0d5bf45f32ca5)` {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:30:09.986639 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 825/1639] 10.222.0.47 () {54 vars in 912 bytes} [Thu Jul 21 08:30:09 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 08:30:09.987610 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f58edfc-7501-4784-a4a7-fae1e44963c9 None None] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:30:09.990165 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67024bd2-8668-4ae2-9df9-de10fe22a8c8 admin admin] RBAC: Authorization granted {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:30:09.995286 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG dbcounter [-] [80244] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80244) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:30:10.009088 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f58edfc-7501-4784-a4a7-fae1e44963c9 None None] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:30:10.019905 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: [pid: 80245|app: 0|req: 815/1640] 10.222.0.47 () {60 vars in 1482 bytes} [Thu Jul 21 08:30:09 2022] PUT /identity/v3/projects/683e37d66a3b4eec8dd3265f31353d6b/users/e641e0ec780e4538ad00b44ede4e6438/roles/e29870f6e9e2411a9db0d5bf45f32ca5 => generated 0 bytes in 100 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 08:30:10.020636 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67024bd2-8668-4ae2-9df9-de10fe22a8c8 admin admin] Authenticating user token {{(pid=80245) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:30:10.103982 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a0f3621-7e3a-4582-8c44-3c45399a3612 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '42d15fc0515945acacbbffcf2356c7a5', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:30:10.104925 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a0f3621-7e3a-4582-8c44-3c45399a3612 admin admin] REQUEST_METHOD: `GET` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:30:10.105442 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a0f3621-7e3a-4582-8c44-3c45399a3612 admin admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:30:10.105563 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a0f3621-7e3a-4582-8c44-3c45399a3612 admin admin] PATH_INFO: `/v3/roles` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:30:10.106315 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a0f3621-7e3a-4582-8c44-3c45399a3612 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:30:10.117501 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a0f3621-7e3a-4582-8c44-3c45399a3612 admin admin] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:30:10.133108 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 826/1641] 10.222.0.47 () {58 vars in 1121 bytes} [Thu Jul 21 08:30:09 2022] GET /identity/v3/roles => generated 1420 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 08:30:10.134048 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a0f3621-7e3a-4582-8c44-3c45399a3612 admin admin] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:30:10.146245 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67024bd2-8668-4ae2-9df9-de10fe22a8c8 admin admin] Validating token access rules against request {{(pid=80245) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:30:10.149225 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a0f3621-7e3a-4582-8c44-3c45399a3612 admin admin] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:30:10.156295 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9617d366-72fd-4c72-bf23-cbb952af987f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4f40410b7674687ab5362ac982248ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4faef13aa50e4e36b6c265fb1398fa14', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80245) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:30:10.156464 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a28af3e-aeaf-47eb-b7f3-399ca1bcca09 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '42d15fc0515945acacbbffcf2356c7a5', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:30:10.156992 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9617d366-72fd-4c72-bf23-cbb952af987f service cinder] REQUEST_METHOD: `GET` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:30:10.157278 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9617d366-72fd-4c72-bf23-cbb952af987f service cinder] SCRIPT_NAME: `/identity` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:30:10.157475 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a28af3e-aeaf-47eb-b7f3-399ca1bcca09 admin admin] REQUEST_METHOD: `PUT` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:30:10.157590 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9617d366-72fd-4c72-bf23-cbb952af987f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:30:10.157810 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a28af3e-aeaf-47eb-b7f3-399ca1bcca09 admin admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:30:10.158098 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a28af3e-aeaf-47eb-b7f3-399ca1bcca09 admin admin] PATH_INFO: `/v3/projects/a0ca7cbc167d47a8b5e26af30f6f7fa1/users/21e4296be8d245aa847b55c655d403c5/roles/e29870f6e9e2411a9db0d5bf45f32ca5` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:30:10.160437 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a28af3e-aeaf-47eb-b7f3-399ca1bcca09 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a0ca7cbc167d47a8b5e26af30f6f7fa1, user_id=21e4296be8d245aa847b55c655d403c5, role_id=e29870f6e9e2411a9db0d5bf45f32ca5)` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:30:10.163639 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a28af3e-aeaf-47eb-b7f3-399ca1bcca09 admin admin] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:30:10.188722 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 827/1642] 10.222.0.47 () {60 vars in 1482 bytes} [Thu Jul 21 08:30:10 2022] PUT /identity/v3/projects/a0ca7cbc167d47a8b5e26af30f6f7fa1/users/21e4296be8d245aa847b55c655d403c5/roles/e29870f6e9e2411a9db0d5bf45f32ca5 => generated 0 bytes in 46 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 08:30:10.191230 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4aa750f-afbc-45bf-a3e9-f0b22fa9ef25 None None] REQUEST_METHOD: `POST` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:30:10.191558 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4aa750f-afbc-45bf-a3e9-f0b22fa9ef25 None None] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:30:10.191845 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4aa750f-afbc-45bf-a3e9-f0b22fa9ef25 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:30:10.216152 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.auth.core [None req-a4aa750f-afbc-45bf-a3e9-f0b22fa9ef25 None None] MFA Rules not processed for user `ac2b5b99a36442148fe743269054e991`. Rule list: `[]` (Enabled: `True`). {{(pid=80244) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 08:30:10.217409 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.fernet_utils [None req-a4aa750f-afbc-45bf-a3e9-f0b22fa9ef25 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80244) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 08:30:10.302102 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9617d366-72fd-4c72-bf23-cbb952af987f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:30:10.303634 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9617d366-72fd-4c72-bf23-cbb952af987f service cinder] RBAC: Authorization granted {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:30:10.310416 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 828/1643] 10.222.0.47 () {58 vars in 1009 bytes} [Thu Jul 21 08:30:10 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 08:30:10.313221 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eb4dbb3-6d4f-4222-9533-55e28ab2b873 None None] REQUEST_METHOD: `POST` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:30:10.313510 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eb4dbb3-6d4f-4222-9533-55e28ab2b873 None None] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:30:10.313897 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eb4dbb3-6d4f-4222-9533-55e28ab2b873 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:30:10.344113 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: [pid: 80245|app: 0|req: 816/1644] 10.222.0.47 () {60 vars in 1365 bytes} [Thu Jul 21 08:30:10 2022] GET /identity/v3/auth/tokens => generated 3305 bytes in 324 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 08:30:10.344964 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9617d366-72fd-4c72-bf23-cbb952af987f service cinder] Authenticating user token {{(pid=80245) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:30:10.349243 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.auth.core [None req-9eb4dbb3-6d4f-4222-9533-55e28ab2b873 None None] MFA Rules not processed for user `e641e0ec780e4538ad00b44ede4e6438`. Rule list: `[]` (Enabled: `True`). {{(pid=80244) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 08:30:10.350284 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.fernet_utils [None req-9eb4dbb3-6d4f-4222-9533-55e28ab2b873 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80244) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 08:30:10.375570 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9617d366-72fd-4c72-bf23-cbb952af987f service cinder] Validating token access rules against request {{(pid=80245) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:30:10.447806 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 829/1645] 10.222.0.47 () {58 vars in 957 bytes} [Thu Jul 21 08:30:10 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 137 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 08:30:10.450113 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dd12e62-fd6b-4e85-ae83-5c5668bd49e4 None None] REQUEST_METHOD: `POST` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:30:10.450783 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dd12e62-fd6b-4e85-ae83-5c5668bd49e4 None None] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:30:10.451144 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dd12e62-fd6b-4e85-ae83-5c5668bd49e4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:30:10.462150 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG dbcounter [-] [80245] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80245) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:30:10.462845 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG dbcounter [-] [80244] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80244) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:30:10.498871 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.auth.core [None req-4dd12e62-fd6b-4e85-ae83-5c5668bd49e4 None None] MFA Rules not processed for user `21e4296be8d245aa847b55c655d403c5`. Rule list: `[]` (Enabled: `True`). {{(pid=80244) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 08:30:10.501818 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.fernet_utils [None req-4dd12e62-fd6b-4e85-ae83-5c5668bd49e4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80244) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 08:30:10.519209 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c8e5e89-2a3b-4c4a-9f97-d6b14fd683a7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '42d15fc0515945acacbbffcf2356c7a5', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80245) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:30:10.520195 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c8e5e89-2a3b-4c4a-9f97-d6b14fd683a7 admin admin] REQUEST_METHOD: `PUT` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:30:10.520474 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c8e5e89-2a3b-4c4a-9f97-d6b14fd683a7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:30:10.520746 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c8e5e89-2a3b-4c4a-9f97-d6b14fd683a7 admin admin] PATH_INFO: `/v3/projects/ca42afd27a3a43fea040e9bad44c8ba9/users/39a796af3051487f8d97c207896e7e97/roles/e29870f6e9e2411a9db0d5bf45f32ca5` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:30:10.524829 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c8e5e89-2a3b-4c4a-9f97-d6b14fd683a7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ca42afd27a3a43fea040e9bad44c8ba9, user_id=39a796af3051487f8d97c207896e7e97, role_id=e29870f6e9e2411a9db0d5bf45f32ca5)` {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:30:10.530868 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c8e5e89-2a3b-4c4a-9f97-d6b14fd683a7 admin admin] RBAC: Authorization granted {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:30:10.553878 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: [pid: 80245|app: 0|req: 817/1646] 10.222.0.47 () {60 vars in 1482 bytes} [Thu Jul 21 08:30:10 2022] PUT /identity/v3/projects/ca42afd27a3a43fea040e9bad44c8ba9/users/39a796af3051487f8d97c207896e7e97/roles/e29870f6e9e2411a9db0d5bf45f32ca5 => generated 0 bytes in 207 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 08:30:10.553878 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c8e5e89-2a3b-4c4a-9f97-d6b14fd683a7 admin admin] Authenticating user token {{(pid=80245) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:30:10.618801 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 830/1647] 10.222.0.47 () {58 vars in 957 bytes} [Thu Jul 21 08:30:10 2022] POST /identity/v3/auth/tokens => generated 3289 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 08:30:10.620066 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dd12e62-fd6b-4e85-ae83-5c5668bd49e4 None None] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:30:10.682475 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c8e5e89-2a3b-4c4a-9f97-d6b14fd683a7 admin admin] Validating token access rules against request {{(pid=80245) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:30:10.688742 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdbe60b9-bc81-4475-9365-fa3be3f968bb service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4f40410b7674687ab5362ac982248ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4faef13aa50e4e36b6c265fb1398fa14', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80245) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:30:10.689322 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdbe60b9-bc81-4475-9365-fa3be3f968bb service cinder] REQUEST_METHOD: `GET` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:30:10.689767 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdbe60b9-bc81-4475-9365-fa3be3f968bb service cinder] SCRIPT_NAME: `/identity` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:30:10.690055 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdbe60b9-bc81-4475-9365-fa3be3f968bb service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:30:10.726137 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dd12e62-fd6b-4e85-ae83-5c5668bd49e4 None None] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:30:10.731268 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-756c102a-5a82-492c-ba91-0a1905aecbe9 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4f40410b7674687ab5362ac982248ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4faef13aa50e4e36b6c265fb1398fa14', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:30:10.733106 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-756c102a-5a82-492c-ba91-0a1905aecbe9 service cinder] REQUEST_METHOD: `GET` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:30:10.734140 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-756c102a-5a82-492c-ba91-0a1905aecbe9 service cinder] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:30:10.735373 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-756c102a-5a82-492c-ba91-0a1905aecbe9 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:30:10.857373 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdbe60b9-bc81-4475-9365-fa3be3f968bb service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:30:10.869357 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdbe60b9-bc81-4475-9365-fa3be3f968bb service cinder] RBAC: Authorization granted {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:30:10.889048 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: [pid: 80245|app: 0|req: 818/1648] 10.222.0.47 () {60 vars in 1365 bytes} [Thu Jul 21 08:30:10 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 336 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 08:30:10.892227 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80bedf7d-903e-4540-9dd4-304ecc8ddb6f None None] REQUEST_METHOD: `POST` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:30:10.892998 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80bedf7d-903e-4540-9dd4-304ecc8ddb6f None None] SCRIPT_NAME: `/identity` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:30:10.893662 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80bedf7d-903e-4540-9dd4-304ecc8ddb6f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:30:10.900828 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-756c102a-5a82-492c-ba91-0a1905aecbe9 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:30:10.906614 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-756c102a-5a82-492c-ba91-0a1905aecbe9 service cinder] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:30:10.951238 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 831/1649] 10.222.0.47 () {60 vars in 1365 bytes} [Thu Jul 21 08:30:10 2022] GET /identity/v3/auth/tokens => generated 3319 bytes in 327 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 08:30:10.954962 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.auth.core [None req-80bedf7d-903e-4540-9dd4-304ecc8ddb6f None None] MFA Rules not processed for user `39a796af3051487f8d97c207896e7e97`. Rule list: `[]` (Enabled: `True`). {{(pid=80245) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 08:30:10.964213 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.fernet_utils [None req-80bedf7d-903e-4540-9dd4-304ecc8ddb6f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80245) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 08:30:11.134537 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG dbcounter [-] [80245] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80245) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:30:11.186048 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: [pid: 80245|app: 0|req: 819/1650] 10.222.0.47 () {58 vars in 957 bytes} [Thu Jul 21 08:30:10 2022] POST /identity/v3/auth/tokens => generated 3303 bytes in 296 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 08:30:11.192196 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-756c102a-5a82-492c-ba91-0a1905aecbe9 service cinder] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:30:11.205579 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-756c102a-5a82-492c-ba91-0a1905aecbe9 service cinder] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:30:11.212744 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f258242-0dfc-49e9-bcee-2ba860c54437 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4f40410b7674687ab5362ac982248ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4faef13aa50e4e36b6c265fb1398fa14', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:30:11.214490 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f258242-0dfc-49e9-bcee-2ba860c54437 service cinder] REQUEST_METHOD: `GET` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:30:11.214986 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f258242-0dfc-49e9-bcee-2ba860c54437 service cinder] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:30:11.215582 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f258242-0dfc-49e9-bcee-2ba860c54437 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:30:11.278446 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG dbcounter [-] [80244] Writing DB stats keystone:SELECT=100 {{(pid=80244) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:30:11.328766 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f258242-0dfc-49e9-bcee-2ba860c54437 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:30:11.331092 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f258242-0dfc-49e9-bcee-2ba860c54437 service cinder] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:30:11.360580 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 832/1651] 10.222.0.47 () {60 vars in 1365 bytes} [Thu Jul 21 08:30:11 2022] GET /identity/v3/auth/tokens => generated 3289 bytes in 168 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 21 08:30:11.464224 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80bedf7d-903e-4540-9dd4-304ecc8ddb6f None None] Authenticating user token {{(pid=80245) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:30:11.571743 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f258242-0dfc-49e9-bcee-2ba860c54437 service cinder] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:30:11.604548 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f258242-0dfc-49e9-bcee-2ba860c54437 service cinder] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:30:11.624487 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dea64169-7e67-40f6-bdd9-05910f2fadfd service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4f40410b7674687ab5362ac982248ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4faef13aa50e4e36b6c265fb1398fa14', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:30:11.626892 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dea64169-7e67-40f6-bdd9-05910f2fadfd service cinder] REQUEST_METHOD: `GET` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:30:11.627914 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dea64169-7e67-40f6-bdd9-05910f2fadfd service cinder] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:30:11.629920 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dea64169-7e67-40f6-bdd9-05910f2fadfd service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:30:11.669689 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80bedf7d-903e-4540-9dd4-304ecc8ddb6f None None] Validating token access rules against request {{(pid=80245) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:30:11.675026 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dea64169-7e67-40f6-bdd9-05910f2fadfd service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:30:11.683030 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dea64169-7e67-40f6-bdd9-05910f2fadfd service cinder] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:30:11.684321 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8abc671c-9889-4c74-bdc7-13a271791b32 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '42d15fc0515945acacbbffcf2356c7a5', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80245) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:30:11.685124 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8abc671c-9889-4c74-bdc7-13a271791b32 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:30:11.685411 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8abc671c-9889-4c74-bdc7-13a271791b32 admin admin] SCRIPT_NAME: `/identity` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:30:11.685686 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8abc671c-9889-4c74-bdc7-13a271791b32 admin admin] PATH_INFO: `/v3/users/4ec042eca40246e6b32756eafe3ac5bb` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:30:11.686993 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8abc671c-9889-4c74-bdc7-13a271791b32 admin admin] RBAC: Authorizing `identity:delete_user(user_id=4ec042eca40246e6b32756eafe3ac5bb)` {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:30:11.688219 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: WARNING py.warnings [None req-8abc671c-9889-4c74-bdc7-13a271791b32 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jul 21 08:30:11.688219 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: warnings.warn(msg) Jul 21 08:30:11.688219 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: Jul 21 08:30:11.689330 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8abc671c-9889-4c74-bdc7-13a271791b32 admin admin] RBAC: Authorization granted {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:30:11.732970 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 833/1652] 10.222.0.47 () {60 vars in 1365 bytes} [Thu Jul 21 08:30:11 2022] GET /identity/v3/auth/tokens => generated 3303 bytes in 162 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 08:30:11.888346 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.notifications [None req-8abc671c-9889-4c74-bdc7-13a271791b32 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4ec042eca40246e6b32756eafe3ac5bb'} {{(pid=80245) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 08:30:11.923688 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.notifications [None req-8abc671c-9889-4c74-bdc7-13a271791b32 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4ec042eca40246e6b32756eafe3ac5bb'} {{(pid=80245) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 08:30:12.007221 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.notifications [None req-8abc671c-9889-4c74-bdc7-13a271791b32 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4ec042eca40246e6b32756eafe3ac5bb'} {{(pid=80245) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 08:30:12.025162 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dea64169-7e67-40f6-bdd9-05910f2fadfd service cinder] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:30:12.042638 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dea64169-7e67-40f6-bdd9-05910f2fadfd service cinder] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:30:12.048423 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d72bce39-f06f-4a82-a4ad-9af5e4f03d35 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4f40410b7674687ab5362ac982248ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4faef13aa50e4e36b6c265fb1398fa14', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:30:12.049246 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d72bce39-f06f-4a82-a4ad-9af5e4f03d35 service cinder] REQUEST_METHOD: `GET` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:30:12.049710 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d72bce39-f06f-4a82-a4ad-9af5e4f03d35 service cinder] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:30:12.050086 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d72bce39-f06f-4a82-a4ad-9af5e4f03d35 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:30:12.064997 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.notifications [None req-8abc671c-9889-4c74-bdc7-13a271791b32 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4ec042eca40246e6b32756eafe3ac5bb'} {{(pid=80245) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 08:30:12.065892 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.notifications [None req-8abc671c-9889-4c74-bdc7-13a271791b32 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4ec042eca40246e6b32756eafe3ac5bb'} {{(pid=80245) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 08:30:12.107173 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: [pid: 80245|app: 0|req: 820/1653] 10.222.0.47 () {58 vars in 1223 bytes} [Thu Jul 21 08:30:11 2022] DELETE /identity/v3/users/4ec042eca40246e6b32756eafe3ac5bb => generated 0 bytes in 640 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 08:30:12.109287 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6adb885-0ccd-48f2-9b89-5fc12e0cb29c None None] REQUEST_METHOD: `POST` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:30:12.109584 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6adb885-0ccd-48f2-9b89-5fc12e0cb29c None None] SCRIPT_NAME: `/identity` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:30:12.109865 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6adb885-0ccd-48f2-9b89-5fc12e0cb29c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:30:12.194447 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.auth.core [None req-a6adb885-0ccd-48f2-9b89-5fc12e0cb29c None None] MFA Rules not processed for user `2dce40b8d74c41ec94cc36e6ff05d412`. Rule list: `[]` (Enabled: `True`). {{(pid=80245) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 08:30:12.199008 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.fernet_utils [None req-a6adb885-0ccd-48f2-9b89-5fc12e0cb29c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80245) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 08:30:12.243100 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d72bce39-f06f-4a82-a4ad-9af5e4f03d35 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:30:12.244761 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d72bce39-f06f-4a82-a4ad-9af5e4f03d35 service cinder] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:30:12.282184 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 834/1654] 10.222.0.47 () {60 vars in 1365 bytes} [Thu Jul 21 08:30:12 2022] GET /identity/v3/auth/tokens => generated 3222 bytes in 257 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 08:30:12.283670 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d72bce39-f06f-4a82-a4ad-9af5e4f03d35 service cinder] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:30:12.318847 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d72bce39-f06f-4a82-a4ad-9af5e4f03d35 service cinder] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:30:12.371777 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: [pid: 80245|app: 0|req: 821/1655] 10.222.0.47 () {58 vars in 957 bytes} [Thu Jul 21 08:30:12 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 264 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 08:30:12.416188 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6adb885-0ccd-48f2-9b89-5fc12e0cb29c None None] Authenticating user token {{(pid=80245) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:30:12.417777 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.fernet_utils [None req-a6adb885-0ccd-48f2-9b89-5fc12e0cb29c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80245) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 08:30:12.460846 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG dbcounter [-] [80245] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80245) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:30:12.500647 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2e3fcf7-bb3f-47c7-b934-6251122ae887 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4f40410b7674687ab5362ac982248ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4faef13aa50e4e36b6c265fb1398fa14', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:30:12.501324 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2e3fcf7-bb3f-47c7-b934-6251122ae887 service cinder] REQUEST_METHOD: `GET` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:30:12.501612 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2e3fcf7-bb3f-47c7-b934-6251122ae887 service cinder] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:30:12.502093 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2e3fcf7-bb3f-47c7-b934-6251122ae887 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:30:12.504155 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.fernet_utils [None req-c2e3fcf7-bb3f-47c7-b934-6251122ae887 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80244) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 08:30:12.588895 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6adb885-0ccd-48f2-9b89-5fc12e0cb29c None None] Validating token access rules against request {{(pid=80245) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:30:12.599904 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3a956cc-3882-4903-b7ff-6056010eb084 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20ca3c3bd3774691b3374cfbfec6e494', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4faef13aa50e4e36b6c265fb1398fa14', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80245) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:30:12.601127 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3a956cc-3882-4903-b7ff-6056010eb084 service neutron] REQUEST_METHOD: `GET` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:30:12.601127 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3a956cc-3882-4903-b7ff-6056010eb084 service neutron] SCRIPT_NAME: `/identity` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:30:12.601127 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3a956cc-3882-4903-b7ff-6056010eb084 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:30:12.624794 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3a956cc-3882-4903-b7ff-6056010eb084 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:30:12.627319 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3a956cc-3882-4903-b7ff-6056010eb084 service neutron] RBAC: Authorization granted {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:30:12.656369 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: [pid: 80245|app: 0|req: 822/1656] 10.222.0.47 () {60 vars in 1365 bytes} [Thu Jul 21 08:30:12 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 241 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 08:30:12.662898 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2e3fcf7-bb3f-47c7-b934-6251122ae887 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:30:12.665252 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2e3fcf7-bb3f-47c7-b934-6251122ae887 service cinder] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:30:12.713536 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 835/1657] 10.222.0.47 () {60 vars in 1365 bytes} [Thu Jul 21 08:30:12 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 430 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 08:30:13.864897 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12639af8-ec44-4909-a6b9-d2247cd43a60 None None] REQUEST_METHOD: `GET` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:30:13.866085 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12639af8-ec44-4909-a6b9-d2247cd43a60 None None] SCRIPT_NAME: `/identity` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:30:13.866376 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12639af8-ec44-4909-a6b9-d2247cd43a60 None None] PATH_INFO: `/` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:30:13.868393 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: [pid: 80245|app: 0|req: 823/1658] 10.222.0.47 () {54 vars in 962 bytes} [Thu Jul 21 08:30:13 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 08:30:13.892418 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0791927-0cb1-4069-9f67-95e9b42e5749 None None] REQUEST_METHOD: `POST` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:30:13.892951 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0791927-0cb1-4069-9f67-95e9b42e5749 None None] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:30:13.892951 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0791927-0cb1-4069-9f67-95e9b42e5749 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:30:13.934268 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.auth.core [None req-e0791927-0cb1-4069-9f67-95e9b42e5749 None None] MFA Rules not processed for user `ac2b5b99a36442148fe743269054e991`. Rule list: `[]` (Enabled: `True`). {{(pid=80244) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 08:30:13.936035 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.fernet_utils [None req-e0791927-0cb1-4069-9f67-95e9b42e5749 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80244) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 08:30:13.945233 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG dbcounter [-] [80244] Writing DB stats keystone:SELECT=100 {{(pid=80244) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:30:14.005862 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12639af8-ec44-4909-a6b9-d2247cd43a60 None None] Authenticating user token {{(pid=80245) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:30:14.007722 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.fernet_utils [None req-12639af8-ec44-4909-a6b9-d2247cd43a60 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80245) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 08:30:14.052105 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12639af8-ec44-4909-a6b9-d2247cd43a60 None None] Validating token access rules against request {{(pid=80245) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:30:14.064217 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-237af115-be53-4cd6-87cd-189fadd3d79b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '42d15fc0515945acacbbffcf2356c7a5', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80245) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:30:14.065415 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-237af115-be53-4cd6-87cd-189fadd3d79b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:30:14.065642 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 836/1659] 10.222.0.47 () {58 vars in 1059 bytes} [Thu Jul 21 08:30:13 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 08:30:14.070444 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-237af115-be53-4cd6-87cd-189fadd3d79b admin admin] SCRIPT_NAME: `/identity` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:30:14.070744 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-237af115-be53-4cd6-87cd-189fadd3d79b admin admin] PATH_INFO: `/v3/users/8fff6db7bd92410f8d6c680c9b24f305` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:30:14.075624 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-237af115-be53-4cd6-87cd-189fadd3d79b admin admin] RBAC: Authorizing `identity:delete_user(user_id=8fff6db7bd92410f8d6c680c9b24f305)` {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:30:14.077684 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-237af115-be53-4cd6-87cd-189fadd3d79b admin admin] RBAC: Authorization granted {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:30:14.079210 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43dc8c7a-fa1a-4819-9fa0-bcb409ec0113 None None] REQUEST_METHOD: `GET` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:30:14.079740 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43dc8c7a-fa1a-4819-9fa0-bcb409ec0113 None None] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:30:14.080966 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43dc8c7a-fa1a-4819-9fa0-bcb409ec0113 None None] PATH_INFO: `/` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:30:14.083825 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 837/1660] 10.222.0.47 () {54 vars in 912 bytes} [Thu Jul 21 08:30:14 2022] GET /identity => generated 270 bytes in 12 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 08:30:14.092039 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a3291bf-fc1e-467b-9a77-144750ff645e None None] REQUEST_METHOD: `POST` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:30:14.092487 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a3291bf-fc1e-467b-9a77-144750ff645e None None] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:30:14.093753 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a3291bf-fc1e-467b-9a77-144750ff645e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:30:14.157844 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.auth.core [None req-2a3291bf-fc1e-467b-9a77-144750ff645e None None] MFA Rules not processed for user `ac2b5b99a36442148fe743269054e991`. Rule list: `[]` (Enabled: `True`). {{(pid=80244) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 08:30:14.165220 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.fernet_utils [None req-2a3291bf-fc1e-467b-9a77-144750ff645e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80244) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 08:30:14.206387 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.notifications [None req-237af115-be53-4cd6-87cd-189fadd3d79b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8fff6db7bd92410f8d6c680c9b24f305'} {{(pid=80245) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 08:30:14.216764 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 838/1661] 10.222.0.47 () {58 vars in 1009 bytes} [Thu Jul 21 08:30:14 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 08:30:14.222454 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.notifications [None req-237af115-be53-4cd6-87cd-189fadd3d79b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8fff6db7bd92410f8d6c680c9b24f305'} {{(pid=80245) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 08:30:14.235472 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a3291bf-fc1e-467b-9a77-144750ff645e None None] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:30:14.238588 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.fernet_utils [None req-2a3291bf-fc1e-467b-9a77-144750ff645e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80244) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 08:30:14.252451 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.notifications [None req-237af115-be53-4cd6-87cd-189fadd3d79b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8fff6db7bd92410f8d6c680c9b24f305'} {{(pid=80245) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 08:30:14.267228 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.notifications [None req-237af115-be53-4cd6-87cd-189fadd3d79b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8fff6db7bd92410f8d6c680c9b24f305'} {{(pid=80245) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 08:30:14.268339 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.notifications [None req-237af115-be53-4cd6-87cd-189fadd3d79b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8fff6db7bd92410f8d6c680c9b24f305'} {{(pid=80245) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 08:30:14.283901 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: [pid: 80245|app: 0|req: 824/1662] 10.222.0.47 () {58 vars in 1223 bytes} [Thu Jul 21 08:30:14 2022] DELETE /identity/v3/users/8fff6db7bd92410f8d6c680c9b24f305 => generated 0 bytes in 279 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 08:30:14.292846 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95153812-a4dc-4595-9635-ba5f56b52ab6 None None] REQUEST_METHOD: `POST` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:30:14.293137 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95153812-a4dc-4595-9635-ba5f56b52ab6 None None] SCRIPT_NAME: `/identity` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:30:14.293423 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95153812-a4dc-4595-9635-ba5f56b52ab6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:30:14.308657 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a3291bf-fc1e-467b-9a77-144750ff645e None None] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:30:14.360881 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.auth.core [None req-95153812-a4dc-4595-9635-ba5f56b52ab6 None None] MFA Rules not processed for user `2dce40b8d74c41ec94cc36e6ff05d412`. Rule list: `[]` (Enabled: `True`). {{(pid=80245) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 08:30:14.361376 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.fernet_utils [None req-95153812-a4dc-4595-9635-ba5f56b52ab6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80245) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 08:30:14.466007 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74c74209-56f6-4a13-bdf4-9b74868531d5 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4f40410b7674687ab5362ac982248ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4faef13aa50e4e36b6c265fb1398fa14', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:30:14.466756 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74c74209-56f6-4a13-bdf4-9b74868531d5 service cinder] REQUEST_METHOD: `GET` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:30:14.467080 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74c74209-56f6-4a13-bdf4-9b74868531d5 service cinder] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:30:14.467380 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74c74209-56f6-4a13-bdf4-9b74868531d5 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:30:14.478959 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.fernet_utils [None req-74c74209-56f6-4a13-bdf4-9b74868531d5 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80244) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 08:30:14.598041 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: [pid: 80245|app: 0|req: 825/1663] 10.222.0.47 () {58 vars in 957 bytes} [Thu Jul 21 08:30:14 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 306 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 08:30:14.634145 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95153812-a4dc-4595-9635-ba5f56b52ab6 None None] Authenticating user token {{(pid=80245) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:30:14.635932 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.fernet_utils [None req-95153812-a4dc-4595-9635-ba5f56b52ab6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80245) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 08:30:14.739099 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74c74209-56f6-4a13-bdf4-9b74868531d5 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:30:14.743963 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74c74209-56f6-4a13-bdf4-9b74868531d5 service cinder] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:30:14.749805 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG dbcounter [-] [80245] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80245) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:30:14.785259 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95153812-a4dc-4595-9635-ba5f56b52ab6 None None] Validating token access rules against request {{(pid=80245) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:30:14.802369 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 839/1664] 10.222.0.47 () {60 vars in 1365 bytes} [Thu Jul 21 08:30:14 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 569 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 08:30:14.816442 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6708c77f-acb6-4147-b8ce-d1389f091a81 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20ca3c3bd3774691b3374cfbfec6e494', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4faef13aa50e4e36b6c265fb1398fa14', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80245) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:30:14.818103 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6708c77f-acb6-4147-b8ce-d1389f091a81 service neutron] REQUEST_METHOD: `GET` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:30:14.819684 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6708c77f-acb6-4147-b8ce-d1389f091a81 service neutron] SCRIPT_NAME: `/identity` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:30:14.820154 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6708c77f-acb6-4147-b8ce-d1389f091a81 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:30:14.866722 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6708c77f-acb6-4147-b8ce-d1389f091a81 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:30:14.869311 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6708c77f-acb6-4147-b8ce-d1389f091a81 service neutron] RBAC: Authorization granted {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:30:14.886371 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: [pid: 80245|app: 0|req: 826/1665] 10.222.0.47 () {60 vars in 1365 bytes} [Thu Jul 21 08:30:14 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 253 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 08:30:15.177818 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74c74209-56f6-4a13-bdf4-9b74868531d5 service cinder] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:30:15.185146 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.fernet_utils [None req-74c74209-56f6-4a13-bdf4-9b74868531d5 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80244) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 08:30:15.249065 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74c74209-56f6-4a13-bdf4-9b74868531d5 service cinder] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:30:15.272529 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-579d995a-d7ab-49ae-b568-8fbef6fd1ad0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '42d15fc0515945acacbbffcf2356c7a5', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:30:15.274246 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-579d995a-d7ab-49ae-b568-8fbef6fd1ad0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:30:15.276167 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-579d995a-d7ab-49ae-b568-8fbef6fd1ad0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:30:15.276167 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-579d995a-d7ab-49ae-b568-8fbef6fd1ad0 admin admin] PATH_INFO: `/v3/projects/84f16ebf991048df944d6c5d23b48195` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:30:15.283456 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-579d995a-d7ab-49ae-b568-8fbef6fd1ad0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=84f16ebf991048df944d6c5d23b48195)` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:30:15.288670 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: WARNING py.warnings [None req-579d995a-d7ab-49ae-b568-8fbef6fd1ad0 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jul 21 08:30:15.288670 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: warnings.warn(msg) Jul 21 08:30:15.288670 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: Jul 21 08:30:15.289881 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-579d995a-d7ab-49ae-b568-8fbef6fd1ad0 admin admin] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:30:15.462059 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG dbcounter [-] [80244] Writing DB stats keystone:SELECT=98,keystone:DELETE=2 {{(pid=80244) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:30:15.528188 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.notifications [None req-579d995a-d7ab-49ae-b568-8fbef6fd1ad0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '84f16ebf991048df944d6c5d23b48195', 'request_id': 'req-579d995a-d7ab-49ae-b568-8fbef6fd1ad0'} {{(pid=80244) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 08:30:15.557917 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.notifications [None req-579d995a-d7ab-49ae-b568-8fbef6fd1ad0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '84f16ebf991048df944d6c5d23b48195', 'request_id': 'req-579d995a-d7ab-49ae-b568-8fbef6fd1ad0'} {{(pid=80244) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 08:30:15.630457 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.notifications [None req-579d995a-d7ab-49ae-b568-8fbef6fd1ad0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '84f16ebf991048df944d6c5d23b48195', 'request_id': 'req-579d995a-d7ab-49ae-b568-8fbef6fd1ad0'} {{(pid=80244) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 08:30:15.680853 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.notifications [None req-579d995a-d7ab-49ae-b568-8fbef6fd1ad0 admin admin] The token cache is being invalidate because project 84f16ebf991048df944d6c5d23b48195 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80244) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 21 08:30:15.681190 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.notifications [None req-579d995a-d7ab-49ae-b568-8fbef6fd1ad0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80244) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 08:30:15.682022 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.notifications [None req-579d995a-d7ab-49ae-b568-8fbef6fd1ad0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80244) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 08:30:15.688687 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 840/1666] 10.222.0.47 () {58 vars in 1232 bytes} [Thu Jul 21 08:30:15 2022] DELETE /identity/v3/projects/84f16ebf991048df944d6c5d23b48195 => generated 0 bytes in 511 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 08:30:15.694640 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6708c77f-acb6-4147-b8ce-d1389f091a81 service neutron] Authenticating user token {{(pid=80245) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:30:15.714850 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6708c77f-acb6-4147-b8ce-d1389f091a81 service neutron] Validating token access rules against request {{(pid=80245) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:30:15.902804 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9537109e-0e1a-49c5-87eb-5750ad0bb348 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '42d15fc0515945acacbbffcf2356c7a5', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80245) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:30:15.904417 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9537109e-0e1a-49c5-87eb-5750ad0bb348 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:30:15.904845 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9537109e-0e1a-49c5-87eb-5750ad0bb348 admin admin] SCRIPT_NAME: `/identity` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:30:15.905326 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9537109e-0e1a-49c5-87eb-5750ad0bb348 admin admin] PATH_INFO: `/v3/users/e641e0ec780e4538ad00b44ede4e6438` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:30:15.906842 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9537109e-0e1a-49c5-87eb-5750ad0bb348 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e641e0ec780e4538ad00b44ede4e6438)` {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:30:15.909134 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9537109e-0e1a-49c5-87eb-5750ad0bb348 admin admin] RBAC: Authorization granted {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:30:16.041615 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.notifications [None req-9537109e-0e1a-49c5-87eb-5750ad0bb348 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e641e0ec780e4538ad00b44ede4e6438'} {{(pid=80245) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 08:30:16.089093 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.notifications [None req-9537109e-0e1a-49c5-87eb-5750ad0bb348 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e641e0ec780e4538ad00b44ede4e6438'} {{(pid=80245) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 08:30:16.122580 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.notifications [None req-9537109e-0e1a-49c5-87eb-5750ad0bb348 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e641e0ec780e4538ad00b44ede4e6438'} {{(pid=80245) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 08:30:16.141936 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.notifications [None req-9537109e-0e1a-49c5-87eb-5750ad0bb348 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e641e0ec780e4538ad00b44ede4e6438'} {{(pid=80245) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 08:30:16.144621 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.notifications [None req-9537109e-0e1a-49c5-87eb-5750ad0bb348 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e641e0ec780e4538ad00b44ede4e6438'} {{(pid=80245) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 08:30:16.161284 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: [pid: 80245|app: 0|req: 827/1667] 10.222.0.47 () {58 vars in 1223 bytes} [Thu Jul 21 08:30:15 2022] DELETE /identity/v3/users/e641e0ec780e4538ad00b44ede4e6438 => generated 0 bytes in 469 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 08:30:16.569748 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-579d995a-d7ab-49ae-b568-8fbef6fd1ad0 admin admin] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:30:16.572595 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.fernet_utils [None req-579d995a-d7ab-49ae-b568-8fbef6fd1ad0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80244) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 08:30:16.608728 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-579d995a-d7ab-49ae-b568-8fbef6fd1ad0 admin admin] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:30:16.809173 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55cb1fe9-72d3-4720-9ef0-45b25cafeec1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '42d15fc0515945acacbbffcf2356c7a5', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:30:16.810085 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55cb1fe9-72d3-4720-9ef0-45b25cafeec1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:30:16.810384 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55cb1fe9-72d3-4720-9ef0-45b25cafeec1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:30:16.810650 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55cb1fe9-72d3-4720-9ef0-45b25cafeec1 admin admin] PATH_INFO: `/v3/users/976a5d6aedea4670827416763642c61d` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:30:16.816525 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55cb1fe9-72d3-4720-9ef0-45b25cafeec1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=976a5d6aedea4670827416763642c61d)` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:30:16.818531 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55cb1fe9-72d3-4720-9ef0-45b25cafeec1 admin admin] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:30:16.976331 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.notifications [None req-55cb1fe9-72d3-4720-9ef0-45b25cafeec1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '976a5d6aedea4670827416763642c61d'} {{(pid=80244) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 08:30:16.993002 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.notifications [None req-55cb1fe9-72d3-4720-9ef0-45b25cafeec1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '976a5d6aedea4670827416763642c61d'} {{(pid=80244) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 08:30:17.020515 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.notifications [None req-55cb1fe9-72d3-4720-9ef0-45b25cafeec1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '976a5d6aedea4670827416763642c61d'} {{(pid=80244) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 08:30:17.022541 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.notifications [None req-55cb1fe9-72d3-4720-9ef0-45b25cafeec1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '976a5d6aedea4670827416763642c61d'} {{(pid=80244) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 08:30:17.037458 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.notifications [None req-55cb1fe9-72d3-4720-9ef0-45b25cafeec1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '976a5d6aedea4670827416763642c61d'} {{(pid=80244) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 08:30:17.051358 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 841/1668] 10.222.0.47 () {58 vars in 1223 bytes} [Thu Jul 21 08:30:16 2022] DELETE /identity/v3/users/976a5d6aedea4670827416763642c61d => generated 0 bytes in 482 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 08:30:17.056227 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-685a78be-8d3e-46d9-a266-811815102d8a None None] REQUEST_METHOD: `POST` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:30:17.057309 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-685a78be-8d3e-46d9-a266-811815102d8a None None] SCRIPT_NAME: `/identity` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:30:17.057733 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-685a78be-8d3e-46d9-a266-811815102d8a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:30:17.115813 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4457aab4-19d6-4b06-9de0-11b5f0febb23 None None] REQUEST_METHOD: `POST` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:30:17.116167 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4457aab4-19d6-4b06-9de0-11b5f0febb23 None None] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:30:17.116739 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4457aab4-19d6-4b06-9de0-11b5f0febb23 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:30:17.138898 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.auth.core [None req-685a78be-8d3e-46d9-a266-811815102d8a None None] MFA Rules not processed for user `2dce40b8d74c41ec94cc36e6ff05d412`. Rule list: `[]` (Enabled: `True`). {{(pid=80245) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 08:30:17.141414 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.fernet_utils [None req-685a78be-8d3e-46d9-a266-811815102d8a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80245) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 08:30:17.230728 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.auth.core [None req-4457aab4-19d6-4b06-9de0-11b5f0febb23 None None] MFA Rules not processed for user `ad56309db7044cb2a86651d4149cd090`. Rule list: `[]` (Enabled: `True`). {{(pid=80244) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 08:30:17.246140 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.fernet_utils [None req-4457aab4-19d6-4b06-9de0-11b5f0febb23 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80244) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 08:30:17.341411 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG dbcounter [-] [80244] Writing DB stats keystone:SELECT=83,keystone:DELETE=15,keystone:INSERT=2 {{(pid=80244) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:30:17.376196 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG dbcounter [-] [80245] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80245) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:30:17.450313 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: [pid: 80245|app: 0|req: 828/1669] 10.222.0.47 () {58 vars in 957 bytes} [Thu Jul 21 08:30:17 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 395 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 08:30:17.468612 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-685a78be-8d3e-46d9-a266-811815102d8a None None] Authenticating user token {{(pid=80245) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:30:17.470348 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.fernet_utils [None req-685a78be-8d3e-46d9-a266-811815102d8a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80245) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 08:30:17.471257 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 842/1670] 10.222.0.47 () {58 vars in 957 bytes} [Thu Jul 21 08:30:17 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 360 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 08:30:17.489052 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4457aab4-19d6-4b06-9de0-11b5f0febb23 None None] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:30:17.490487 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.fernet_utils [None req-4457aab4-19d6-4b06-9de0-11b5f0febb23 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80244) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 08:30:17.520367 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4457aab4-19d6-4b06-9de0-11b5f0febb23 None None] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:30:17.531460 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a61588ba-c5b0-4ad9-90e2-b60f1f98d1a3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '42d15fc0515945acacbbffcf2356c7a5', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:30:17.532278 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a61588ba-c5b0-4ad9-90e2-b60f1f98d1a3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:30:17.532561 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a61588ba-c5b0-4ad9-90e2-b60f1f98d1a3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:30:17.532832 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a61588ba-c5b0-4ad9-90e2-b60f1f98d1a3 admin admin] PATH_INFO: `/v3/projects/683e37d66a3b4eec8dd3265f31353d6b` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:30:17.541561 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a61588ba-c5b0-4ad9-90e2-b60f1f98d1a3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=683e37d66a3b4eec8dd3265f31353d6b)` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:30:17.543548 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a61588ba-c5b0-4ad9-90e2-b60f1f98d1a3 admin admin] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:30:17.605606 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-685a78be-8d3e-46d9-a266-811815102d8a None None] Validating token access rules against request {{(pid=80245) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:30:17.613039 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c0e33d9-2d3c-4d6c-aa9c-d9dad7a412e8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20ca3c3bd3774691b3374cfbfec6e494', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4faef13aa50e4e36b6c265fb1398fa14', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80245) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:30:17.619761 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c0e33d9-2d3c-4d6c-aa9c-d9dad7a412e8 service neutron] REQUEST_METHOD: `GET` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:30:17.630618 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c0e33d9-2d3c-4d6c-aa9c-d9dad7a412e8 service neutron] SCRIPT_NAME: `/identity` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:30:17.633726 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c0e33d9-2d3c-4d6c-aa9c-d9dad7a412e8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:30:17.656688 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c0e33d9-2d3c-4d6c-aa9c-d9dad7a412e8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:30:17.658458 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c0e33d9-2d3c-4d6c-aa9c-d9dad7a412e8 service neutron] RBAC: Authorization granted {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:30:17.682336 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: [pid: 80245|app: 0|req: 829/1671] 10.222.0.47 () {60 vars in 1365 bytes} [Thu Jul 21 08:30:17 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 214 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 21 08:30:17.686365 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c0e33d9-2d3c-4d6c-aa9c-d9dad7a412e8 service neutron] Authenticating user token {{(pid=80245) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:30:17.693047 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.fernet_utils [None req-2c0e33d9-2d3c-4d6c-aa9c-d9dad7a412e8 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80245) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 08:30:17.776706 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.notifications [None req-a61588ba-c5b0-4ad9-90e2-b60f1f98d1a3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '683e37d66a3b4eec8dd3265f31353d6b', 'request_id': 'req-a61588ba-c5b0-4ad9-90e2-b60f1f98d1a3'} {{(pid=80244) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 08:30:17.799261 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.notifications [None req-a61588ba-c5b0-4ad9-90e2-b60f1f98d1a3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '683e37d66a3b4eec8dd3265f31353d6b', 'request_id': 'req-a61588ba-c5b0-4ad9-90e2-b60f1f98d1a3'} {{(pid=80244) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 08:30:17.818465 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.notifications [None req-a61588ba-c5b0-4ad9-90e2-b60f1f98d1a3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '683e37d66a3b4eec8dd3265f31353d6b', 'request_id': 'req-a61588ba-c5b0-4ad9-90e2-b60f1f98d1a3'} {{(pid=80244) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 08:30:17.864071 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.notifications [None req-a61588ba-c5b0-4ad9-90e2-b60f1f98d1a3 admin admin] The token cache is being invalidate because project 683e37d66a3b4eec8dd3265f31353d6b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80244) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 21 08:30:17.865832 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.notifications [None req-a61588ba-c5b0-4ad9-90e2-b60f1f98d1a3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80244) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 08:30:17.868296 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.notifications [None req-a61588ba-c5b0-4ad9-90e2-b60f1f98d1a3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80244) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 08:30:17.874850 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 843/1672] 10.222.0.47 () {58 vars in 1232 bytes} [Thu Jul 21 08:30:17 2022] DELETE /identity/v3/projects/683e37d66a3b4eec8dd3265f31353d6b => generated 0 bytes in 386 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 08:30:17.876188 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a61588ba-c5b0-4ad9-90e2-b60f1f98d1a3 admin admin] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:30:17.880858 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.fernet_utils [None req-a61588ba-c5b0-4ad9-90e2-b60f1f98d1a3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80244) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 08:30:18.027136 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c0e33d9-2d3c-4d6c-aa9c-d9dad7a412e8 service neutron] Validating token access rules against request {{(pid=80245) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:30:18.071819 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a61588ba-c5b0-4ad9-90e2-b60f1f98d1a3 admin admin] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:30:18.091100 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b39bb852-d7bb-4462-a8f5-2d98e1150fd3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '42d15fc0515945acacbbffcf2356c7a5', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:30:18.100664 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b39bb852-d7bb-4462-a8f5-2d98e1150fd3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:30:18.100886 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b39bb852-d7bb-4462-a8f5-2d98e1150fd3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:30:18.101120 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b39bb852-d7bb-4462-a8f5-2d98e1150fd3 admin admin] PATH_INFO: `/v3/projects/b4b35ea61009452684dbd4b39032bed1` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:30:18.104606 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b39bb852-d7bb-4462-a8f5-2d98e1150fd3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b4b35ea61009452684dbd4b39032bed1)` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:30:18.106565 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b39bb852-d7bb-4462-a8f5-2d98e1150fd3 admin admin] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:30:18.179546 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3adfc6b0-2e9c-4715-b752-fb7ffe80bd85 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2b5b99a36442148fe743269054e991', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4faef13aa50e4e36b6c265fb1398fa14', 'project_domain_id': 'default', 'roles': ['member', 'service', 'ResellerAdmin', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80245) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:30:18.182715 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3adfc6b0-2e9c-4715-b752-fb7ffe80bd85 service nova] REQUEST_METHOD: `GET` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:30:18.183008 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3adfc6b0-2e9c-4715-b752-fb7ffe80bd85 service nova] SCRIPT_NAME: `/identity` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:30:18.183293 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3adfc6b0-2e9c-4715-b752-fb7ffe80bd85 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:30:18.192102 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.fernet_utils [None req-3adfc6b0-2e9c-4715-b752-fb7ffe80bd85 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80245) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 08:30:18.275914 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.notifications [None req-b39bb852-d7bb-4462-a8f5-2d98e1150fd3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b4b35ea61009452684dbd4b39032bed1', 'request_id': 'req-b39bb852-d7bb-4462-a8f5-2d98e1150fd3'} {{(pid=80244) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 08:30:18.287065 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG dbcounter [-] [80244] Writing DB stats keystone:SELECT=86,keystone:DELETE=12,keystone:INSERT=2 {{(pid=80244) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:30:18.294996 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.notifications [None req-b39bb852-d7bb-4462-a8f5-2d98e1150fd3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b4b35ea61009452684dbd4b39032bed1', 'request_id': 'req-b39bb852-d7bb-4462-a8f5-2d98e1150fd3'} {{(pid=80244) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 08:30:18.308108 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG dbcounter [-] [80245] Writing DB stats keystone:SELECT=100 {{(pid=80245) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:30:18.312547 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.notifications [None req-b39bb852-d7bb-4462-a8f5-2d98e1150fd3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b4b35ea61009452684dbd4b39032bed1', 'request_id': 'req-b39bb852-d7bb-4462-a8f5-2d98e1150fd3'} {{(pid=80244) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 08:30:18.342497 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.notifications [None req-b39bb852-d7bb-4462-a8f5-2d98e1150fd3 admin admin] The token cache is being invalidate because project b4b35ea61009452684dbd4b39032bed1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80244) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 21 08:30:18.342887 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.notifications [None req-b39bb852-d7bb-4462-a8f5-2d98e1150fd3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80244) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 08:30:18.344214 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.notifications [None req-b39bb852-d7bb-4462-a8f5-2d98e1150fd3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80244) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 08:30:18.346837 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 844/1673] 10.222.0.47 () {58 vars in 1232 bytes} [Thu Jul 21 08:30:17 2022] DELETE /identity/v3/projects/b4b35ea61009452684dbd4b39032bed1 => generated 0 bytes in 471 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 08:30:18.348212 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9e567fd-bcbe-46e6-aa9e-2249e0ca7a42 None None] REQUEST_METHOD: `POST` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:30:18.348505 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9e567fd-bcbe-46e6-aa9e-2249e0ca7a42 None None] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:30:18.348788 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9e567fd-bcbe-46e6-aa9e-2249e0ca7a42 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:30:18.392425 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3adfc6b0-2e9c-4715-b752-fb7ffe80bd85 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:30:18.394191 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3adfc6b0-2e9c-4715-b752-fb7ffe80bd85 service nova] RBAC: Authorization granted {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:30:18.408339 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.auth.core [None req-c9e567fd-bcbe-46e6-aa9e-2249e0ca7a42 None None] MFA Rules not processed for user `2dce40b8d74c41ec94cc36e6ff05d412`. Rule list: `[]` (Enabled: `True`). {{(pid=80244) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 08:30:18.412426 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: [pid: 80245|app: 0|req: 830/1674] 10.222.0.47 () {60 vars in 1365 bytes} [Thu Jul 21 08:30:17 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 729 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 08:30:18.413304 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3adfc6b0-2e9c-4715-b752-fb7ffe80bd85 service nova] Authenticating user token {{(pid=80245) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:30:18.415238 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.fernet_utils [None req-c9e567fd-bcbe-46e6-aa9e-2249e0ca7a42 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80244) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 08:30:18.441169 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3adfc6b0-2e9c-4715-b752-fb7ffe80bd85 service nova] Validating token access rules against request {{(pid=80245) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:30:18.576754 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0a73528-9f42-44cc-86fc-1dfcbdad2d9e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '42d15fc0515945acacbbffcf2356c7a5', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80245) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:30:18.577564 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0a73528-9f42-44cc-86fc-1dfcbdad2d9e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:30:18.577847 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0a73528-9f42-44cc-86fc-1dfcbdad2d9e admin admin] SCRIPT_NAME: `/identity` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:30:18.578123 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0a73528-9f42-44cc-86fc-1dfcbdad2d9e admin admin] PATH_INFO: `/v3/users/78013433f60b4d3680e01b7fbaba39b0` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:30:18.584666 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0a73528-9f42-44cc-86fc-1dfcbdad2d9e admin admin] RBAC: Authorizing `identity:delete_user(user_id=78013433f60b4d3680e01b7fbaba39b0)` {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:30:18.589508 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0a73528-9f42-44cc-86fc-1dfcbdad2d9e admin admin] RBAC: Authorization granted {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:30:18.615808 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 845/1675] 10.222.0.47 () {58 vars in 957 bytes} [Thu Jul 21 08:30:18 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 268 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 08:30:18.616851 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9e567fd-bcbe-46e6-aa9e-2249e0ca7a42 None None] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:30:18.618283 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.fernet_utils [None req-c9e567fd-bcbe-46e6-aa9e-2249e0ca7a42 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80244) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 08:30:18.644236 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9e567fd-bcbe-46e6-aa9e-2249e0ca7a42 None None] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:30:18.655818 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf89db5c-7158-40b7-9204-cb98a630c19b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '42d15fc0515945acacbbffcf2356c7a5', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:30:18.656840 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf89db5c-7158-40b7-9204-cb98a630c19b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:30:18.657149 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf89db5c-7158-40b7-9204-cb98a630c19b admin admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:30:18.657437 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf89db5c-7158-40b7-9204-cb98a630c19b admin admin] PATH_INFO: `/v3/projects/a9d8ff00b3ae4c18971814d983d93e05` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:30:18.658933 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf89db5c-7158-40b7-9204-cb98a630c19b admin admin] RBAC: Authorizing `identity:delete_project(project_id=a9d8ff00b3ae4c18971814d983d93e05)` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:30:18.661939 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf89db5c-7158-40b7-9204-cb98a630c19b admin admin] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:30:18.699003 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.notifications [None req-d0a73528-9f42-44cc-86fc-1dfcbdad2d9e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '78013433f60b4d3680e01b7fbaba39b0'} {{(pid=80245) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 08:30:18.720013 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.notifications [None req-d0a73528-9f42-44cc-86fc-1dfcbdad2d9e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '78013433f60b4d3680e01b7fbaba39b0'} {{(pid=80245) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 08:30:18.750575 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.notifications [None req-d0a73528-9f42-44cc-86fc-1dfcbdad2d9e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '78013433f60b4d3680e01b7fbaba39b0'} {{(pid=80245) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 08:30:18.761560 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.notifications [None req-bf89db5c-7158-40b7-9204-cb98a630c19b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a9d8ff00b3ae4c18971814d983d93e05', 'request_id': 'req-bf89db5c-7158-40b7-9204-cb98a630c19b'} {{(pid=80244) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 08:30:18.787219 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.notifications [None req-d0a73528-9f42-44cc-86fc-1dfcbdad2d9e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '78013433f60b4d3680e01b7fbaba39b0'} {{(pid=80245) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 08:30:18.791400 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.notifications [None req-d0a73528-9f42-44cc-86fc-1dfcbdad2d9e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '78013433f60b4d3680e01b7fbaba39b0'} {{(pid=80245) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 08:30:18.793900 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.notifications [None req-bf89db5c-7158-40b7-9204-cb98a630c19b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a9d8ff00b3ae4c18971814d983d93e05', 'request_id': 'req-bf89db5c-7158-40b7-9204-cb98a630c19b'} {{(pid=80244) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 08:30:18.823844 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.notifications [None req-bf89db5c-7158-40b7-9204-cb98a630c19b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a9d8ff00b3ae4c18971814d983d93e05', 'request_id': 'req-bf89db5c-7158-40b7-9204-cb98a630c19b'} {{(pid=80244) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 08:30:18.826845 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: [pid: 80245|app: 0|req: 831/1676] 10.222.0.47 () {58 vars in 1223 bytes} [Thu Jul 21 08:30:18 2022] DELETE /identity/v3/users/78013433f60b4d3680e01b7fbaba39b0 => generated 0 bytes in 414 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 08:30:18.842916 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9a306d1-37e9-4b2e-a7d3-dd33250c2a8e None None] REQUEST_METHOD: `POST` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:30:18.843235 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9a306d1-37e9-4b2e-a7d3-dd33250c2a8e None None] SCRIPT_NAME: `/identity` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:30:18.843521 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9a306d1-37e9-4b2e-a7d3-dd33250c2a8e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:30:18.870756 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.notifications [None req-bf89db5c-7158-40b7-9204-cb98a630c19b admin admin] The token cache is being invalidate because project a9d8ff00b3ae4c18971814d983d93e05 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80244) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 21 08:30:18.877749 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.notifications [None req-bf89db5c-7158-40b7-9204-cb98a630c19b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80244) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 08:30:18.882200 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.notifications [None req-bf89db5c-7158-40b7-9204-cb98a630c19b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80244) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 08:30:18.892249 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 846/1677] 10.222.0.47 () {58 vars in 1232 bytes} [Thu Jul 21 08:30:18 2022] DELETE /identity/v3/projects/a9d8ff00b3ae4c18971814d983d93e05 => generated 0 bytes in 276 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 08:30:18.903277 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8399f80-1362-431c-9551-ea661de581e3 None None] REQUEST_METHOD: `POST` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:30:18.903602 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8399f80-1362-431c-9551-ea661de581e3 None None] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:30:18.903889 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8399f80-1362-431c-9551-ea661de581e3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:30:18.925628 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.auth.core [None req-d9a306d1-37e9-4b2e-a7d3-dd33250c2a8e None None] MFA Rules not processed for user `2dce40b8d74c41ec94cc36e6ff05d412`. Rule list: `[]` (Enabled: `True`). {{(pid=80245) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 08:30:18.933647 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.fernet_utils [None req-d9a306d1-37e9-4b2e-a7d3-dd33250c2a8e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80245) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 08:30:18.977766 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.auth.core [None req-e8399f80-1362-431c-9551-ea661de581e3 None None] MFA Rules not processed for user `2dce40b8d74c41ec94cc36e6ff05d412`. Rule list: `[]` (Enabled: `True`). {{(pid=80244) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 08:30:18.982906 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.fernet_utils [None req-e8399f80-1362-431c-9551-ea661de581e3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80244) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 08:30:19.087043 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG dbcounter [-] [80245] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80245) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:30:19.137332 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: [pid: 80245|app: 0|req: 832/1678] 10.222.0.47 () {58 vars in 957 bytes} [Thu Jul 21 08:30:18 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 310 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 08:30:19.142391 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9a306d1-37e9-4b2e-a7d3-dd33250c2a8e None None] Authenticating user token {{(pid=80245) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:30:19.172647 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9a306d1-37e9-4b2e-a7d3-dd33250c2a8e None None] Validating token access rules against request {{(pid=80245) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:30:19.177187 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 847/1679] 10.222.0.47 () {58 vars in 957 bytes} [Thu Jul 21 08:30:18 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 276 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 08:30:19.182366 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-998538a5-f3f9-488d-abdb-f7d0269fd7ee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '42d15fc0515945acacbbffcf2356c7a5', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80245) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:30:19.183749 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-998538a5-f3f9-488d-abdb-f7d0269fd7ee admin admin] REQUEST_METHOD: `GET` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:30:19.185521 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-998538a5-f3f9-488d-abdb-f7d0269fd7ee admin admin] SCRIPT_NAME: `/identity` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:30:19.186255 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-998538a5-f3f9-488d-abdb-f7d0269fd7ee admin admin] PATH_INFO: `/v3/domains` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:30:19.187849 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d400ba4-3bb8-49aa-a7fb-c4f1373536bb None None] REQUEST_METHOD: `POST` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:30:19.187977 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d400ba4-3bb8-49aa-a7fb-c4f1373536bb None None] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:30:19.188226 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d400ba4-3bb8-49aa-a7fb-c4f1373536bb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:30:19.189302 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-998538a5-f3f9-488d-abdb-f7d0269fd7ee admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80245) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 08:30:19.190322 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-998538a5-f3f9-488d-abdb-f7d0269fd7ee admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:30:19.194672 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-998538a5-f3f9-488d-abdb-f7d0269fd7ee admin admin] RBAC: Authorization granted {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:30:19.216416 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG dbcounter [-] [80244] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=80244) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:30:19.226530 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: [pid: 80245|app: 0|req: 833/1680] 10.222.0.47 () {58 vars in 1152 bytes} [Thu Jul 21 08:30:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 85 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 08:30:19.231513 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.auth.core [None req-3d400ba4-3bb8-49aa-a7fb-c4f1373536bb None None] MFA Rules not processed for user `2dce40b8d74c41ec94cc36e6ff05d412`. Rule list: `[]` (Enabled: `True`). {{(pid=80244) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 08:30:19.232478 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-998538a5-f3f9-488d-abdb-f7d0269fd7ee admin admin] Authenticating user token {{(pid=80245) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:30:19.237918 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.fernet_utils [None req-3d400ba4-3bb8-49aa-a7fb-c4f1373536bb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80244) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 08:30:19.255363 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 848/1681] 10.222.0.47 () {58 vars in 957 bytes} [Thu Jul 21 08:30:19 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 67 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 21 08:30:19.257888 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d400ba4-3bb8-49aa-a7fb-c4f1373536bb None None] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:30:19.261624 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-998538a5-f3f9-488d-abdb-f7d0269fd7ee admin admin] Validating token access rules against request {{(pid=80245) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:30:19.271970 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d400ba4-3bb8-49aa-a7fb-c4f1373536bb None None] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:30:19.278859 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a9b12c8-79e9-4f4b-aa61-c0d46a034d50 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '42d15fc0515945acacbbffcf2356c7a5', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:30:19.279870 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a9b12c8-79e9-4f4b-aa61-c0d46a034d50 admin admin] REQUEST_METHOD: `GET` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:30:19.280416 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a9b12c8-79e9-4f4b-aa61-c0d46a034d50 admin admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:30:19.280901 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a9b12c8-79e9-4f4b-aa61-c0d46a034d50 admin admin] PATH_INFO: `/v3/domains` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:30:19.283215 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a9b12c8-79e9-4f4b-aa61-c0d46a034d50 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80244) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 08:30:19.283642 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a9b12c8-79e9-4f4b-aa61-c0d46a034d50 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:30:19.284580 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9256846-51de-48c3-a950-4ff1b93f02a0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '42d15fc0515945acacbbffcf2356c7a5', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80245) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:30:19.285274 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9256846-51de-48c3-a950-4ff1b93f02a0 admin admin] REQUEST_METHOD: `POST` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:30:19.285618 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9256846-51de-48c3-a950-4ff1b93f02a0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:30:19.285837 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9256846-51de-48c3-a950-4ff1b93f02a0 admin admin] PATH_INFO: `/v3/projects` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:30:19.286087 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a9b12c8-79e9-4f4b-aa61-c0d46a034d50 admin admin] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:30:19.286541 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9256846-51de-48c3-a950-4ff1b93f02a0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:30:19.288514 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9256846-51de-48c3-a950-4ff1b93f02a0 admin admin] RBAC: Authorization granted {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:30:19.303378 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 849/1682] 10.222.0.47 () {58 vars in 1152 bytes} [Thu Jul 21 08:30:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 21 08:30:19.308338 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a9b12c8-79e9-4f4b-aa61-c0d46a034d50 admin admin] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:30:19.346006 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a9b12c8-79e9-4f4b-aa61-c0d46a034d50 admin admin] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:30:19.349474 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: [pid: 80245|app: 0|req: 834/1683] 10.222.0.47 () {60 vars in 1152 bytes} [Thu Jul 21 08:30:19 2022] POST /identity/v3/projects => generated 359 bytes in 116 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 08:30:19.350544 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9256846-51de-48c3-a950-4ff1b93f02a0 admin admin] Authenticating user token {{(pid=80245) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:30:19.492986 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9256846-51de-48c3-a950-4ff1b93f02a0 admin admin] Validating token access rules against request {{(pid=80245) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:30:19.505620 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1864bc67-127c-4aff-aed5-a9b2471aa3ac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '42d15fc0515945acacbbffcf2356c7a5', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80245) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:30:19.506327 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1864bc67-127c-4aff-aed5-a9b2471aa3ac admin admin] REQUEST_METHOD: `POST` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:30:19.506617 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1864bc67-127c-4aff-aed5-a9b2471aa3ac admin admin] SCRIPT_NAME: `/identity` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:30:19.506896 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1864bc67-127c-4aff-aed5-a9b2471aa3ac admin admin] PATH_INFO: `/v3/users` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:30:19.507647 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1864bc67-127c-4aff-aed5-a9b2471aa3ac admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:30:19.509636 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1864bc67-127c-4aff-aed5-a9b2471aa3ac admin admin] RBAC: Authorization granted {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:30:19.524965 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56e38167-9166-4f0b-97a6-e0773945c035 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '42d15fc0515945acacbbffcf2356c7a5', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:30:19.529918 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56e38167-9166-4f0b-97a6-e0773945c035 admin admin] REQUEST_METHOD: `POST` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:30:19.531705 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56e38167-9166-4f0b-97a6-e0773945c035 admin admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:30:19.532402 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56e38167-9166-4f0b-97a6-e0773945c035 admin admin] PATH_INFO: `/v3/projects` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:30:19.533890 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56e38167-9166-4f0b-97a6-e0773945c035 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:30:19.537895 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56e38167-9166-4f0b-97a6-e0773945c035 admin admin] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:30:19.548097 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: [pid: 80245|app: 0|req: 835/1684] 10.222.0.47 () {60 vars in 1143 bytes} [Thu Jul 21 08:30:19 2022] POST /identity/v3/users => generated 280 bytes in 195 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 21 08:30:19.549375 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1864bc67-127c-4aff-aed5-a9b2471aa3ac admin admin] Authenticating user token {{(pid=80245) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:30:19.580692 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 850/1685] 10.222.0.47 () {60 vars in 1152 bytes} [Thu Jul 21 08:30:19 2022] POST /identity/v3/projects => generated 365 bytes in 270 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 08:30:19.581738 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56e38167-9166-4f0b-97a6-e0773945c035 admin admin] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:30:19.587274 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1864bc67-127c-4aff-aed5-a9b2471aa3ac admin admin] Validating token access rules against request {{(pid=80245) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:30:19.708262 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f7480e3-de94-4c76-b4e8-17c17b27450e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '42d15fc0515945acacbbffcf2356c7a5', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80245) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:30:19.709340 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f7480e3-de94-4c76-b4e8-17c17b27450e admin admin] REQUEST_METHOD: `GET` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:30:19.710515 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f7480e3-de94-4c76-b4e8-17c17b27450e admin admin] SCRIPT_NAME: `/identity` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:30:19.711002 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f7480e3-de94-4c76-b4e8-17c17b27450e admin admin] PATH_INFO: `/v3/roles` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:30:19.712242 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f7480e3-de94-4c76-b4e8-17c17b27450e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:30:19.715287 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f7480e3-de94-4c76-b4e8-17c17b27450e admin admin] RBAC: Authorization granted {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:30:19.753147 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: [pid: 80245|app: 0|req: 836/1686] 10.222.0.47 () {58 vars in 1121 bytes} [Thu Jul 21 08:30:19 2022] GET /identity/v3/roles => generated 1420 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 08:30:19.761644 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f7480e3-de94-4c76-b4e8-17c17b27450e admin admin] Authenticating user token {{(pid=80245) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:30:19.783753 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56e38167-9166-4f0b-97a6-e0773945c035 admin admin] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:30:19.787355 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f7480e3-de94-4c76-b4e8-17c17b27450e admin admin] Validating token access rules against request {{(pid=80245) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:30:19.798434 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afc13a4f-cde9-4fb8-a0dd-41fb305bf73c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '42d15fc0515945acacbbffcf2356c7a5', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80245) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:30:19.799156 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afc13a4f-cde9-4fb8-a0dd-41fb305bf73c admin admin] REQUEST_METHOD: `GET` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:30:19.799494 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afc13a4f-cde9-4fb8-a0dd-41fb305bf73c admin admin] SCRIPT_NAME: `/identity` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:30:19.799781 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afc13a4f-cde9-4fb8-a0dd-41fb305bf73c admin admin] PATH_INFO: `/v3/roles` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:30:19.800568 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afc13a4f-cde9-4fb8-a0dd-41fb305bf73c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:30:19.802240 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afc13a4f-cde9-4fb8-a0dd-41fb305bf73c admin admin] RBAC: Authorization granted {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:30:19.810532 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f699a2d7-59ee-4089-bcc3-e1834da8f3f3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '42d15fc0515945acacbbffcf2356c7a5', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:30:19.811364 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f699a2d7-59ee-4089-bcc3-e1834da8f3f3 admin admin] REQUEST_METHOD: `POST` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:30:19.811548 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f699a2d7-59ee-4089-bcc3-e1834da8f3f3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:30:19.811824 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f699a2d7-59ee-4089-bcc3-e1834da8f3f3 admin admin] PATH_INFO: `/v3/users` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:30:19.817814 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f699a2d7-59ee-4089-bcc3-e1834da8f3f3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:30:19.825204 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: [pid: 80245|app: 0|req: 837/1687] 10.222.0.47 () {58 vars in 1121 bytes} [Thu Jul 21 08:30:19 2022] GET /identity/v3/roles => generated 1420 bytes in 64 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 08:30:19.829416 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f699a2d7-59ee-4089-bcc3-e1834da8f3f3 admin admin] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:30:19.830093 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afc13a4f-cde9-4fb8-a0dd-41fb305bf73c admin admin] Authenticating user token {{(pid=80245) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:30:19.864562 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afc13a4f-cde9-4fb8-a0dd-41fb305bf73c admin admin] Validating token access rules against request {{(pid=80245) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:30:19.879146 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f756b5c7-dd86-4595-9292-f8215494f5bb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '42d15fc0515945acacbbffcf2356c7a5', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80245) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:30:19.882148 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f756b5c7-dd86-4595-9292-f8215494f5bb admin admin] REQUEST_METHOD: `PUT` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:30:19.882762 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f756b5c7-dd86-4595-9292-f8215494f5bb admin admin] SCRIPT_NAME: `/identity` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:30:19.884019 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f756b5c7-dd86-4595-9292-f8215494f5bb admin admin] PATH_INFO: `/v3/projects/098fa33a679c4d10872207508b660352/users/5f0e08cdb72e4810aec7b7b45eb205b3/roles/3a074738221e4601b5205715ba87d240` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:30:19.890642 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 851/1688] 10.222.0.47 () {60 vars in 1143 bytes} [Thu Jul 21 08:30:19 2022] POST /identity/v3/users => generated 283 bytes in 308 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 21 08:30:19.891916 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f699a2d7-59ee-4089-bcc3-e1834da8f3f3 admin admin] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:30:19.917024 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f699a2d7-59ee-4089-bcc3-e1834da8f3f3 admin admin] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:30:19.933702 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58f0ee28-5421-452d-b217-c74ce65403cf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '42d15fc0515945acacbbffcf2356c7a5', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:30:19.934798 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58f0ee28-5421-452d-b217-c74ce65403cf admin admin] REQUEST_METHOD: `GET` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:30:19.935289 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58f0ee28-5421-452d-b217-c74ce65403cf admin admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:30:19.935814 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58f0ee28-5421-452d-b217-c74ce65403cf admin admin] PATH_INFO: `/v3/roles` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:30:19.936803 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58f0ee28-5421-452d-b217-c74ce65403cf admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:30:19.938754 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58f0ee28-5421-452d-b217-c74ce65403cf admin admin] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:30:19.939961 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f756b5c7-dd86-4595-9292-f8215494f5bb admin admin] RBAC: Authorizing `identity:create_grant(project_id=098fa33a679c4d10872207508b660352, user_id=5f0e08cdb72e4810aec7b7b45eb205b3, role_id=3a074738221e4601b5205715ba87d240)` {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:30:19.944036 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f756b5c7-dd86-4595-9292-f8215494f5bb admin admin] RBAC: Authorization granted {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:30:19.969436 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 852/1689] 10.222.0.47 () {58 vars in 1121 bytes} [Thu Jul 21 08:30:19 2022] GET /identity/v3/roles => generated 1420 bytes in 78 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 08:30:19.980882 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: [pid: 80245|app: 0|req: 838/1690] 10.222.0.47 () {60 vars in 1482 bytes} [Thu Jul 21 08:30:19 2022] PUT /identity/v3/projects/098fa33a679c4d10872207508b660352/users/5f0e08cdb72e4810aec7b7b45eb205b3/roles/3a074738221e4601b5205715ba87d240 => generated 0 bytes in 151 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 08:30:19.984635 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f756b5c7-dd86-4595-9292-f8215494f5bb admin admin] Authenticating user token {{(pid=80245) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:30:19.993059 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a989303-d993-4565-8e25-04b12231be18 None None] REQUEST_METHOD: `POST` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:30:19.994228 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a989303-d993-4565-8e25-04b12231be18 None None] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:30:19.994716 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a989303-d993-4565-8e25-04b12231be18 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:30:20.012930 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG dbcounter [-] [80245] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80245) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:30:20.082040 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.auth.core [None req-0a989303-d993-4565-8e25-04b12231be18 None None] MFA Rules not processed for user `5f0e08cdb72e4810aec7b7b45eb205b3`. Rule list: `[]` (Enabled: `True`). {{(pid=80244) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 08:30:20.083317 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.fernet_utils [None req-0a989303-d993-4565-8e25-04b12231be18 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80244) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 08:30:20.154105 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f756b5c7-dd86-4595-9292-f8215494f5bb admin admin] Validating token access rules against request {{(pid=80245) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:30:20.177270 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f84b4f4-343c-4452-abf4-4d54fde1c8d4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '42d15fc0515945acacbbffcf2356c7a5', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80245) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:30:20.177985 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f84b4f4-343c-4452-abf4-4d54fde1c8d4 admin admin] REQUEST_METHOD: `GET` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:30:20.178275 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f84b4f4-343c-4452-abf4-4d54fde1c8d4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:30:20.178611 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f84b4f4-343c-4452-abf4-4d54fde1c8d4 admin admin] PATH_INFO: `/v3/roles` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:30:20.179469 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f84b4f4-343c-4452-abf4-4d54fde1c8d4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:30:20.181860 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f84b4f4-343c-4452-abf4-4d54fde1c8d4 admin admin] RBAC: Authorization granted {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:30:20.206180 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG dbcounter [-] [80244] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80244) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:30:20.211593 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: [pid: 80245|app: 0|req: 839/1691] 10.222.0.47 () {58 vars in 1121 bytes} [Thu Jul 21 08:30:19 2022] GET /identity/v3/roles => generated 1420 bytes in 230 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 08:30:20.216513 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f84b4f4-343c-4452-abf4-4d54fde1c8d4 admin admin] Authenticating user token {{(pid=80245) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:30:20.241879 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f84b4f4-343c-4452-abf4-4d54fde1c8d4 admin admin] Validating token access rules against request {{(pid=80245) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:30:20.260485 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4341d4ad-fb87-4b95-b4cd-3aa17576a2b9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '42d15fc0515945acacbbffcf2356c7a5', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80245) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:30:20.261415 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4341d4ad-fb87-4b95-b4cd-3aa17576a2b9 admin admin] REQUEST_METHOD: `PUT` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:30:20.261695 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4341d4ad-fb87-4b95-b4cd-3aa17576a2b9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:30:20.261969 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4341d4ad-fb87-4b95-b4cd-3aa17576a2b9 admin admin] PATH_INFO: `/v3/projects/dd675e3ea3e34a219be692f3a9c6e2ee/users/75d3a8a0833b4b228499ad834d978e2a/roles/3a074738221e4601b5205715ba87d240` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:30:20.269642 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 853/1692] 10.222.0.47 () {58 vars in 957 bytes} [Thu Jul 21 08:30:19 2022] POST /identity/v3/auth/tokens => generated 3226 bytes in 277 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 08:30:20.306630 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a989303-d993-4565-8e25-04b12231be18 None None] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:30:20.310262 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4341d4ad-fb87-4b95-b4cd-3aa17576a2b9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=dd675e3ea3e34a219be692f3a9c6e2ee, user_id=75d3a8a0833b4b228499ad834d978e2a, role_id=3a074738221e4601b5205715ba87d240)` {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:30:20.310707 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.fernet_utils [None req-0a989303-d993-4565-8e25-04b12231be18 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80244) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 08:30:20.315129 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4341d4ad-fb87-4b95-b4cd-3aa17576a2b9 admin admin] RBAC: Authorization granted {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:30:20.357063 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: [pid: 80245|app: 0|req: 840/1693] 10.222.0.47 () {60 vars in 1482 bytes} [Thu Jul 21 08:30:20 2022] PUT /identity/v3/projects/dd675e3ea3e34a219be692f3a9c6e2ee/users/75d3a8a0833b4b228499ad834d978e2a/roles/3a074738221e4601b5205715ba87d240 => generated 0 bytes in 141 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 21 08:30:20.371232 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e81d09e-e044-4b3f-833b-84f3a96a5db3 None None] REQUEST_METHOD: `POST` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:30:20.371460 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e81d09e-e044-4b3f-833b-84f3a96a5db3 None None] SCRIPT_NAME: `/identity` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:30:20.371744 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e81d09e-e044-4b3f-833b-84f3a96a5db3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:30:20.427527 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.auth.core [None req-2e81d09e-e044-4b3f-833b-84f3a96a5db3 None None] MFA Rules not processed for user `75d3a8a0833b4b228499ad834d978e2a`. Rule list: `[]` (Enabled: `True`). {{(pid=80245) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 08:30:20.430245 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.fernet_utils [None req-2e81d09e-e044-4b3f-833b-84f3a96a5db3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80245) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 08:30:20.447510 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a989303-d993-4565-8e25-04b12231be18 None None] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:30:20.560152 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eab6e76e-7923-401b-bfd0-29cb20f26e9d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4f40410b7674687ab5362ac982248ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4faef13aa50e4e36b6c265fb1398fa14', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:30:20.560855 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eab6e76e-7923-401b-bfd0-29cb20f26e9d service cinder] REQUEST_METHOD: `GET` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:30:20.561931 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eab6e76e-7923-401b-bfd0-29cb20f26e9d service cinder] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:30:20.561931 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eab6e76e-7923-401b-bfd0-29cb20f26e9d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:30:20.656594 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: [pid: 80245|app: 0|req: 841/1694] 10.222.0.47 () {58 vars in 957 bytes} [Thu Jul 21 08:30:20 2022] POST /identity/v3/auth/tokens => generated 3232 bytes in 286 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 21 08:30:20.660956 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e81d09e-e044-4b3f-833b-84f3a96a5db3 None None] Authenticating user token {{(pid=80245) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:30:20.711630 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eab6e76e-7923-401b-bfd0-29cb20f26e9d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:30:20.713282 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eab6e76e-7923-401b-bfd0-29cb20f26e9d service cinder] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:30:20.743603 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 854/1695] 10.222.0.47 () {60 vars in 1365 bytes} [Thu Jul 21 08:30:20 2022] GET /identity/v3/auth/tokens => generated 3226 bytes in 432 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 08:30:20.744751 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eab6e76e-7923-401b-bfd0-29cb20f26e9d service cinder] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:30:20.858915 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e81d09e-e044-4b3f-833b-84f3a96a5db3 None None] Validating token access rules against request {{(pid=80245) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:30:20.870148 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd96640e-c056-4b98-b14f-b330f31a406e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '42d15fc0515945acacbbffcf2356c7a5', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80245) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:30:20.872658 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd96640e-c056-4b98-b14f-b330f31a406e admin admin] REQUEST_METHOD: `POST` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:30:20.873438 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd96640e-c056-4b98-b14f-b330f31a406e admin admin] SCRIPT_NAME: `/identity` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:30:20.875506 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd96640e-c056-4b98-b14f-b330f31a406e admin admin] PATH_INFO: `/v3/projects` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:30:20.878443 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG dbcounter [-] [80244] Writing DB stats keystone:SELECT=100 {{(pid=80244) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:30:20.886561 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd96640e-c056-4b98-b14f-b330f31a406e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:30:20.893717 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd96640e-c056-4b98-b14f-b330f31a406e admin admin] RBAC: Authorization granted {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:30:20.903732 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eab6e76e-7923-401b-bfd0-29cb20f26e9d service cinder] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:30:20.926198 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-146dcfe3-1c14-4792-934c-951eb41424a1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '42d15fc0515945acacbbffcf2356c7a5', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:30:20.927169 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-146dcfe3-1c14-4792-934c-951eb41424a1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:30:20.927412 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-146dcfe3-1c14-4792-934c-951eb41424a1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:30:20.927692 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-146dcfe3-1c14-4792-934c-951eb41424a1 admin admin] PATH_INFO: `/v3/users/5f0e08cdb72e4810aec7b7b45eb205b3` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:30:20.928963 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-146dcfe3-1c14-4792-934c-951eb41424a1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=5f0e08cdb72e4810aec7b7b45eb205b3)` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:30:20.931275 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-146dcfe3-1c14-4792-934c-951eb41424a1 admin admin] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:30:20.946403 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG dbcounter [-] [80245] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80245) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:30:20.993832 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: [pid: 80245|app: 0|req: 842/1696] 10.222.0.47 () {60 vars in 1152 bytes} [Thu Jul 21 08:30:20 2022] POST /identity/v3/projects => generated 365 bytes in 330 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 08:30:20.995208 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd96640e-c056-4b98-b14f-b330f31a406e admin admin] Authenticating user token {{(pid=80245) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:30:21.055791 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.notifications [None req-146dcfe3-1c14-4792-934c-951eb41424a1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5f0e08cdb72e4810aec7b7b45eb205b3'} {{(pid=80244) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 08:30:21.071297 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.notifications [None req-146dcfe3-1c14-4792-934c-951eb41424a1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5f0e08cdb72e4810aec7b7b45eb205b3'} {{(pid=80244) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 08:30:21.104244 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.notifications [None req-146dcfe3-1c14-4792-934c-951eb41424a1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5f0e08cdb72e4810aec7b7b45eb205b3'} {{(pid=80244) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 08:30:21.110242 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.notifications [None req-146dcfe3-1c14-4792-934c-951eb41424a1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5f0e08cdb72e4810aec7b7b45eb205b3'} {{(pid=80244) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 08:30:21.131971 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.notifications [None req-146dcfe3-1c14-4792-934c-951eb41424a1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5f0e08cdb72e4810aec7b7b45eb205b3'} {{(pid=80244) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 08:30:21.157817 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 855/1697] 10.222.0.47 () {58 vars in 1223 bytes} [Thu Jul 21 08:30:20 2022] DELETE /identity/v3/users/5f0e08cdb72e4810aec7b7b45eb205b3 => generated 0 bytes in 414 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 21 08:30:21.162838 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-538148b1-7f4c-4d6d-ad07-c7968a1a548f None None] REQUEST_METHOD: `POST` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:30:21.162970 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-538148b1-7f4c-4d6d-ad07-c7968a1a548f None None] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:30:21.163241 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-538148b1-7f4c-4d6d-ad07-c7968a1a548f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:30:21.210623 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.auth.core [None req-538148b1-7f4c-4d6d-ad07-c7968a1a548f None None] MFA Rules not processed for user `2dce40b8d74c41ec94cc36e6ff05d412`. Rule list: `[]` (Enabled: `True`). {{(pid=80244) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 08:30:21.211954 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.fernet_utils [None req-538148b1-7f4c-4d6d-ad07-c7968a1a548f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80244) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 08:30:21.223163 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd96640e-c056-4b98-b14f-b330f31a406e admin admin] Validating token access rules against request {{(pid=80245) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:30:21.388871 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 856/1698] 10.222.0.47 () {58 vars in 957 bytes} [Thu Jul 21 08:30:21 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 227 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 21 08:30:21.391929 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b2f2004-8e95-42f3-b95c-184b0d07f686 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '42d15fc0515945acacbbffcf2356c7a5', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80245) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:30:21.392702 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b2f2004-8e95-42f3-b95c-184b0d07f686 admin admin] REQUEST_METHOD: `POST` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:30:21.392993 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b2f2004-8e95-42f3-b95c-184b0d07f686 admin admin] SCRIPT_NAME: `/identity` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:30:21.393312 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b2f2004-8e95-42f3-b95c-184b0d07f686 admin admin] PATH_INFO: `/v3/users` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:30:21.394078 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b2f2004-8e95-42f3-b95c-184b0d07f686 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:30:21.397320 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b2f2004-8e95-42f3-b95c-184b0d07f686 admin admin] RBAC: Authorization granted {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:30:21.457100 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: [pid: 80245|app: 0|req: 843/1699] 10.222.0.47 () {60 vars in 1143 bytes} [Thu Jul 21 08:30:20 2022] POST /identity/v3/users => generated 289 bytes in 463 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 08:30:21.459217 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-538148b1-7f4c-4d6d-ad07-c7968a1a548f None None] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:30:21.462707 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.fernet_utils [None req-538148b1-7f4c-4d6d-ad07-c7968a1a548f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80244) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 08:30:21.480593 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-538148b1-7f4c-4d6d-ad07-c7968a1a548f None None] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:30:21.498636 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5543df0e-c8a5-4ac1-880b-cfbe8e598cc5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '42d15fc0515945acacbbffcf2356c7a5', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:30:21.500053 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5543df0e-c8a5-4ac1-880b-cfbe8e598cc5 admin admin] REQUEST_METHOD: `GET` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:30:21.500432 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5543df0e-c8a5-4ac1-880b-cfbe8e598cc5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:30:21.500803 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5543df0e-c8a5-4ac1-880b-cfbe8e598cc5 admin admin] PATH_INFO: `/v3/roles` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:30:21.501711 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5543df0e-c8a5-4ac1-880b-cfbe8e598cc5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:30:21.504012 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5543df0e-c8a5-4ac1-880b-cfbe8e598cc5 admin admin] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:30:21.507399 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b2f2004-8e95-42f3-b95c-184b0d07f686 admin admin] Authenticating user token {{(pid=80245) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:30:21.510388 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.fernet_utils [None req-7b2f2004-8e95-42f3-b95c-184b0d07f686 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80245) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 08:30:21.521065 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 857/1700] 10.222.0.47 () {58 vars in 1121 bytes} [Thu Jul 21 08:30:21 2022] GET /identity/v3/roles => generated 1420 bytes in 62 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 08:30:21.526268 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5543df0e-c8a5-4ac1-880b-cfbe8e598cc5 admin admin] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:30:21.562889 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5543df0e-c8a5-4ac1-880b-cfbe8e598cc5 admin admin] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:30:21.576029 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e7be69c-ac04-42cd-b218-cebc460bb571 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '42d15fc0515945acacbbffcf2356c7a5', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:30:21.578314 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e7be69c-ac04-42cd-b218-cebc460bb571 admin admin] REQUEST_METHOD: `PUT` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:30:21.578837 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e7be69c-ac04-42cd-b218-cebc460bb571 admin admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:30:21.579231 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e7be69c-ac04-42cd-b218-cebc460bb571 admin admin] PATH_INFO: `/v3/projects/d2b04a0c9dcd4f08a6db8d8847819ac0/users/a0b0f566f8fd43fa91147e5c9eb5eb0d/roles/e29870f6e9e2411a9db0d5bf45f32ca5` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:30:21.599080 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e7be69c-ac04-42cd-b218-cebc460bb571 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d2b04a0c9dcd4f08a6db8d8847819ac0, user_id=a0b0f566f8fd43fa91147e5c9eb5eb0d, role_id=e29870f6e9e2411a9db0d5bf45f32ca5)` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:30:21.611108 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e7be69c-ac04-42cd-b218-cebc460bb571 admin admin] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:30:21.640772 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 858/1701] 10.222.0.47 () {60 vars in 1482 bytes} [Thu Jul 21 08:30:21 2022] PUT /identity/v3/projects/d2b04a0c9dcd4f08a6db8d8847819ac0/users/a0b0f566f8fd43fa91147e5c9eb5eb0d/roles/e29870f6e9e2411a9db0d5bf45f32ca5 => generated 0 bytes in 115 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 21 08:30:21.644941 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e7be69c-ac04-42cd-b218-cebc460bb571 admin admin] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:30:21.696322 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b2f2004-8e95-42f3-b95c-184b0d07f686 admin admin] Validating token access rules against request {{(pid=80245) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:30:21.731210 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG dbcounter [-] [80244] Writing DB stats keystone:SELECT=88,keystone:DELETE=10,keystone:INSERT=2 {{(pid=80244) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:30:21.822373 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e7be69c-ac04-42cd-b218-cebc460bb571 admin admin] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:30:21.848223 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2da1c59b-88c2-42a3-8bd6-55944cbe980a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '42d15fc0515945acacbbffcf2356c7a5', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:30:21.849043 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2da1c59b-88c2-42a3-8bd6-55944cbe980a admin admin] REQUEST_METHOD: `GET` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:30:21.849389 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2da1c59b-88c2-42a3-8bd6-55944cbe980a admin admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:30:21.849558 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2da1c59b-88c2-42a3-8bd6-55944cbe980a admin admin] PATH_INFO: `/v3/roles` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:30:21.850331 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2da1c59b-88c2-42a3-8bd6-55944cbe980a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:30:21.857848 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2da1c59b-88c2-42a3-8bd6-55944cbe980a admin admin] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:30:21.894221 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 859/1702] 10.222.0.47 () {58 vars in 1121 bytes} [Thu Jul 21 08:30:21 2022] GET /identity/v3/roles => generated 1420 bytes in 250 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 08:30:21.895787 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2da1c59b-88c2-42a3-8bd6-55944cbe980a admin admin] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:30:21.899262 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG dbcounter [-] [80245] Writing DB stats keystone:INSERT=4,keystone:SELECT=96 {{(pid=80245) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:30:21.912324 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa339035-73bb-4688-b140-469f5ec7304e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20ca3c3bd3774691b3374cfbfec6e494', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4faef13aa50e4e36b6c265fb1398fa14', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80245) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:30:21.930023 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa339035-73bb-4688-b140-469f5ec7304e service neutron] REQUEST_METHOD: `GET` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:30:21.930023 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa339035-73bb-4688-b140-469f5ec7304e service neutron] SCRIPT_NAME: `/identity` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:30:21.930023 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa339035-73bb-4688-b140-469f5ec7304e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:30:21.952941 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2da1c59b-88c2-42a3-8bd6-55944cbe980a admin admin] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:30:21.976942 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa339035-73bb-4688-b140-469f5ec7304e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:30:21.978073 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba643eff-0892-4cf1-9257-0c7ec81adf9a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '42d15fc0515945acacbbffcf2356c7a5', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:30:21.978995 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba643eff-0892-4cf1-9257-0c7ec81adf9a admin admin] REQUEST_METHOD: `PUT` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:30:21.979278 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba643eff-0892-4cf1-9257-0c7ec81adf9a admin admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:30:21.979607 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba643eff-0892-4cf1-9257-0c7ec81adf9a admin admin] PATH_INFO: `/v3/projects/d2b04a0c9dcd4f08a6db8d8847819ac0/users/a0b0f566f8fd43fa91147e5c9eb5eb0d/roles/e29870f6e9e2411a9db0d5bf45f32ca5` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:30:21.983987 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba643eff-0892-4cf1-9257-0c7ec81adf9a admin admin] RBAC: Authorizing `identity:create_grant(project_id=d2b04a0c9dcd4f08a6db8d8847819ac0, user_id=a0b0f566f8fd43fa91147e5c9eb5eb0d, role_id=e29870f6e9e2411a9db0d5bf45f32ca5)` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:30:21.986816 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba643eff-0892-4cf1-9257-0c7ec81adf9a admin admin] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:30:21.988112 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa339035-73bb-4688-b140-469f5ec7304e service neutron] RBAC: Authorization granted {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:30:22.015143 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 860/1703] 10.222.0.47 () {60 vars in 1482 bytes} [Thu Jul 21 08:30:21 2022] PUT /identity/v3/projects/d2b04a0c9dcd4f08a6db8d8847819ac0/users/a0b0f566f8fd43fa91147e5c9eb5eb0d/roles/e29870f6e9e2411a9db0d5bf45f32ca5 => generated 0 bytes in 119 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 08:30:22.016734 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71942777-32ba-42b2-829d-b794c1cbf8e3 None None] REQUEST_METHOD: `POST` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:30:22.017032 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71942777-32ba-42b2-829d-b794c1cbf8e3 None None] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:30:22.017316 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71942777-32ba-42b2-829d-b794c1cbf8e3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:30:22.037855 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: [pid: 80245|app: 0|req: 844/1704] 10.222.0.47 () {60 vars in 1365 bytes} [Thu Jul 21 08:30:21 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 531 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 21 08:30:22.042501 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac86ab03-5dd8-4846-b0c1-4de2ad5748e6 None None] REQUEST_METHOD: `POST` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:30:22.042794 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac86ab03-5dd8-4846-b0c1-4de2ad5748e6 None None] SCRIPT_NAME: `/identity` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:30:22.043077 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac86ab03-5dd8-4846-b0c1-4de2ad5748e6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:30:22.096215 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.auth.core [None req-71942777-32ba-42b2-829d-b794c1cbf8e3 None None] MFA Rules not processed for user `ad56309db7044cb2a86651d4149cd090`. Rule list: `[]` (Enabled: `True`). {{(pid=80244) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 08:30:22.098290 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.fernet_utils [None req-71942777-32ba-42b2-829d-b794c1cbf8e3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80244) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 08:30:22.153481 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.auth.core [None req-ac86ab03-5dd8-4846-b0c1-4de2ad5748e6 None None] MFA Rules not processed for user `a0b0f566f8fd43fa91147e5c9eb5eb0d`. Rule list: `[]` (Enabled: `True`). {{(pid=80245) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 08:30:22.153934 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.fernet_utils [None req-ac86ab03-5dd8-4846-b0c1-4de2ad5748e6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80245) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 08:30:22.247024 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 861/1705] 10.222.0.47 () {58 vars in 957 bytes} [Thu Jul 21 08:30:22 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 230 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 08:30:22.358527 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71942777-32ba-42b2-829d-b794c1cbf8e3 None None] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:30:22.360207 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.fernet_utils [None req-71942777-32ba-42b2-829d-b794c1cbf8e3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80244) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 08:30:22.528223 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: [pid: 80245|app: 0|req: 845/1706] 10.222.0.47 () {58 vars in 957 bytes} [Thu Jul 21 08:30:22 2022] POST /identity/v3/auth/tokens => generated 3299 bytes in 489 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 08:30:22.543429 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac86ab03-5dd8-4846-b0c1-4de2ad5748e6 None None] Authenticating user token {{(pid=80245) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:30:22.549636 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.fernet_utils [None req-ac86ab03-5dd8-4846-b0c1-4de2ad5748e6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80245) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 08:30:22.573894 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71942777-32ba-42b2-829d-b794c1cbf8e3 None None] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:30:22.583183 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15ce55ae-a2b3-49de-a0f3-b8d16976c9de admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '42d15fc0515945acacbbffcf2356c7a5', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:30:22.584582 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15ce55ae-a2b3-49de-a0f3-b8d16976c9de admin admin] REQUEST_METHOD: `DELETE` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:30:22.584582 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15ce55ae-a2b3-49de-a0f3-b8d16976c9de admin admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:30:22.584783 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15ce55ae-a2b3-49de-a0f3-b8d16976c9de admin admin] PATH_INFO: `/v3/projects/85733045af3649ca8b35ffd2eb635bae` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:30:22.591448 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15ce55ae-a2b3-49de-a0f3-b8d16976c9de admin admin] RBAC: Authorizing `identity:delete_project(project_id=85733045af3649ca8b35ffd2eb635bae)` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:30:22.593777 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15ce55ae-a2b3-49de-a0f3-b8d16976c9de admin admin] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:30:22.706257 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac86ab03-5dd8-4846-b0c1-4de2ad5748e6 None None] Validating token access rules against request {{(pid=80245) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:30:22.720893 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ba57c91-206a-4821-a290-ba0545d41508 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4f40410b7674687ab5362ac982248ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4faef13aa50e4e36b6c265fb1398fa14', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80245) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:30:22.721414 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ba57c91-206a-4821-a290-ba0545d41508 service cinder] REQUEST_METHOD: `GET` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:30:22.721699 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ba57c91-206a-4821-a290-ba0545d41508 service cinder] SCRIPT_NAME: `/identity` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:30:22.721979 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ba57c91-206a-4821-a290-ba0545d41508 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:30:22.760136 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ba57c91-206a-4821-a290-ba0545d41508 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:30:22.761621 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ba57c91-206a-4821-a290-ba0545d41508 service cinder] RBAC: Authorization granted {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:30:22.764289 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.notifications [None req-15ce55ae-a2b3-49de-a0f3-b8d16976c9de admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '85733045af3649ca8b35ffd2eb635bae', 'request_id': 'req-15ce55ae-a2b3-49de-a0f3-b8d16976c9de'} {{(pid=80244) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 08:30:22.778234 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.notifications [None req-15ce55ae-a2b3-49de-a0f3-b8d16976c9de admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '85733045af3649ca8b35ffd2eb635bae', 'request_id': 'req-15ce55ae-a2b3-49de-a0f3-b8d16976c9de'} {{(pid=80244) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 08:30:22.798249 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: [pid: 80245|app: 0|req: 846/1707] 10.222.0.47 () {60 vars in 1365 bytes} [Thu Jul 21 08:30:22 2022] GET /identity/v3/auth/tokens => generated 3299 bytes in 255 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 21 08:30:22.802326 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.notifications [None req-15ce55ae-a2b3-49de-a0f3-b8d16976c9de admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '85733045af3649ca8b35ffd2eb635bae', 'request_id': 'req-15ce55ae-a2b3-49de-a0f3-b8d16976c9de'} {{(pid=80244) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 08:30:22.808679 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG dbcounter [-] [80244] Writing DB stats keystone:SELECT=92,keystone:INSERT=2,keystone:DELETE=6 {{(pid=80244) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:30:22.827849 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.notifications [None req-15ce55ae-a2b3-49de-a0f3-b8d16976c9de admin admin] The token cache is being invalidate because project 85733045af3649ca8b35ffd2eb635bae was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80244) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 21 08:30:22.828264 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.notifications [None req-15ce55ae-a2b3-49de-a0f3-b8d16976c9de admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80244) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 08:30:22.831729 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.notifications [None req-15ce55ae-a2b3-49de-a0f3-b8d16976c9de admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80244) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 08:30:22.842152 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 862/1708] 10.222.0.47 () {58 vars in 1232 bytes} [Thu Jul 21 08:30:22 2022] DELETE /identity/v3/projects/85733045af3649ca8b35ffd2eb635bae => generated 0 bytes in 481 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 21 08:30:22.845412 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26daa0af-8bba-4b39-a12d-f886c39da54d None None] REQUEST_METHOD: `POST` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:30:22.845708 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26daa0af-8bba-4b39-a12d-f886c39da54d None None] SCRIPT_NAME: `/identity` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:30:22.845989 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26daa0af-8bba-4b39-a12d-f886c39da54d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:30:22.891046 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.auth.core [None req-26daa0af-8bba-4b39-a12d-f886c39da54d None None] MFA Rules not processed for user `2dce40b8d74c41ec94cc36e6ff05d412`. Rule list: `[]` (Enabled: `True`). {{(pid=80245) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 08:30:22.892359 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.fernet_utils [None req-26daa0af-8bba-4b39-a12d-f886c39da54d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80245) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 08:30:22.971889 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15ce55ae-a2b3-49de-a0f3-b8d16976c9de admin admin] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:30:22.973400 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.fernet_utils [None req-15ce55ae-a2b3-49de-a0f3-b8d16976c9de admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80244) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 08:30:23.100389 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: [pid: 80245|app: 0|req: 847/1709] 10.222.0.47 () {58 vars in 957 bytes} [Thu Jul 21 08:30:22 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 257 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 08:30:23.105119 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31c217f1-99b7-450b-8c98-2dc05cc644a4 None None] REQUEST_METHOD: `POST` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:30:23.105279 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31c217f1-99b7-450b-8c98-2dc05cc644a4 None None] SCRIPT_NAME: `/identity` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:30:23.105521 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31c217f1-99b7-450b-8c98-2dc05cc644a4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:30:23.147292 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15ce55ae-a2b3-49de-a0f3-b8d16976c9de admin admin] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:30:23.165614 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-757cada3-103e-4118-8d6c-4608c311c95a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4f40410b7674687ab5362ac982248ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4faef13aa50e4e36b6c265fb1398fa14', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:30:23.166219 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-757cada3-103e-4118-8d6c-4608c311c95a service cinder] REQUEST_METHOD: `GET` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:30:23.166509 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-757cada3-103e-4118-8d6c-4608c311c95a service cinder] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:30:23.166792 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-757cada3-103e-4118-8d6c-4608c311c95a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:30:23.173005 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.fernet_utils [None req-757cada3-103e-4118-8d6c-4608c311c95a service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80244) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 08:30:23.175164 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.auth.core [None req-31c217f1-99b7-450b-8c98-2dc05cc644a4 None None] MFA Rules not processed for user `2dce40b8d74c41ec94cc36e6ff05d412`. Rule list: `[]` (Enabled: `True`). {{(pid=80245) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 08:30:23.176610 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.fernet_utils [None req-31c217f1-99b7-450b-8c98-2dc05cc644a4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80245) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 08:30:23.203945 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: [pid: 80245|app: 0|req: 848/1710] 10.222.0.47 () {58 vars in 957 bytes} [Thu Jul 21 08:30:23 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 08:30:23.209573 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31c217f1-99b7-450b-8c98-2dc05cc644a4 None None] Authenticating user token {{(pid=80245) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:30:23.240685 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31c217f1-99b7-450b-8c98-2dc05cc644a4 None None] Validating token access rules against request {{(pid=80245) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:30:23.262698 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d1dd7b1-8505-4d17-8649-abb6a218b21d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '42d15fc0515945acacbbffcf2356c7a5', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80245) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:30:23.265400 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d1dd7b1-8505-4d17-8649-abb6a218b21d admin admin] REQUEST_METHOD: `GET` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:30:23.265785 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d1dd7b1-8505-4d17-8649-abb6a218b21d admin admin] SCRIPT_NAME: `/identity` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:30:23.266002 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d1dd7b1-8505-4d17-8649-abb6a218b21d admin admin] PATH_INFO: `/v3/domains` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:30:23.266778 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d1dd7b1-8505-4d17-8649-abb6a218b21d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80245) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 08:30:23.267094 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d1dd7b1-8505-4d17-8649-abb6a218b21d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:30:23.268786 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d1dd7b1-8505-4d17-8649-abb6a218b21d admin admin] RBAC: Authorization granted {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:30:23.302290 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG dbcounter [-] [80245] Writing DB stats keystone:SELECT=100 {{(pid=80245) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:30:23.332438 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: [pid: 80245|app: 0|req: 849/1711] 10.222.0.47 () {58 vars in 1152 bytes} [Thu Jul 21 08:30:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 125 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 08:30:23.338077 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d1dd7b1-8505-4d17-8649-abb6a218b21d admin admin] Authenticating user token {{(pid=80245) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:30:23.356859 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d1dd7b1-8505-4d17-8649-abb6a218b21d admin admin] Validating token access rules against request {{(pid=80245) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:30:23.370963 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-757cada3-103e-4118-8d6c-4608c311c95a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:30:23.371270 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37f55f0a-df16-4c3f-aa08-0f75064b452e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '42d15fc0515945acacbbffcf2356c7a5', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80245) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:30:23.372589 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37f55f0a-df16-4c3f-aa08-0f75064b452e admin admin] REQUEST_METHOD: `POST` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:30:23.372733 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-757cada3-103e-4118-8d6c-4608c311c95a service cinder] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:30:23.373402 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37f55f0a-df16-4c3f-aa08-0f75064b452e admin admin] SCRIPT_NAME: `/identity` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:30:23.374485 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37f55f0a-df16-4c3f-aa08-0f75064b452e admin admin] PATH_INFO: `/v3/projects` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:30:23.376422 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37f55f0a-df16-4c3f-aa08-0f75064b452e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:30:23.384569 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37f55f0a-df16-4c3f-aa08-0f75064b452e admin admin] RBAC: Authorization granted {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:30:23.396328 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 863/1712] 10.222.0.47 () {60 vars in 1365 bytes} [Thu Jul 21 08:30:22 2022] GET /identity/v3/auth/tokens => generated 3232 bytes in 425 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 21 08:30:23.442173 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: [pid: 80245|app: 0|req: 850/1713] 10.222.0.47 () {60 vars in 1152 bytes} [Thu Jul 21 08:30:23 2022] POST /identity/v3/projects => generated 367 bytes in 105 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 08:30:23.446792 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-757cada3-103e-4118-8d6c-4608c311c95a service cinder] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:30:23.476677 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-757cada3-103e-4118-8d6c-4608c311c95a service cinder] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:30:23.675233 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a2b5bc5-ae27-434b-8722-6f9af9dfff6b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '42d15fc0515945acacbbffcf2356c7a5', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:30:23.680154 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a2b5bc5-ae27-434b-8722-6f9af9dfff6b admin admin] REQUEST_METHOD: `POST` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:30:23.680626 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a2b5bc5-ae27-434b-8722-6f9af9dfff6b admin admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:30:23.681162 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a2b5bc5-ae27-434b-8722-6f9af9dfff6b admin admin] PATH_INFO: `/v3/users` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:30:23.683022 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a2b5bc5-ae27-434b-8722-6f9af9dfff6b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:30:23.688760 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a2b5bc5-ae27-434b-8722-6f9af9dfff6b admin admin] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:30:23.740318 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37f55f0a-df16-4c3f-aa08-0f75064b452e admin admin] Authenticating user token {{(pid=80245) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:30:23.752536 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37f55f0a-df16-4c3f-aa08-0f75064b452e admin admin] Validating token access rules against request {{(pid=80245) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:30:23.764808 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 864/1714] 10.222.0.47 () {60 vars in 1143 bytes} [Thu Jul 21 08:30:23 2022] POST /identity/v3/users => generated 284 bytes in 318 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 08:30:23.775832 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f222c73e-69dd-4684-9914-ea73bd9cfdb9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '42d15fc0515945acacbbffcf2356c7a5', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80245) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:30:23.776606 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f222c73e-69dd-4684-9914-ea73bd9cfdb9 admin admin] REQUEST_METHOD: `GET` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:30:23.776861 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f222c73e-69dd-4684-9914-ea73bd9cfdb9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:30:23.777144 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f222c73e-69dd-4684-9914-ea73bd9cfdb9 admin admin] PATH_INFO: `/v3/roles` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:30:23.777904 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f222c73e-69dd-4684-9914-ea73bd9cfdb9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:30:23.779807 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f222c73e-69dd-4684-9914-ea73bd9cfdb9 admin admin] RBAC: Authorization granted {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:30:23.809398 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: [pid: 80245|app: 0|req: 851/1715] 10.222.0.47 () {58 vars in 1121 bytes} [Thu Jul 21 08:30:23 2022] GET /identity/v3/roles => generated 1420 bytes in 70 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 21 08:30:23.814471 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a2b5bc5-ae27-434b-8722-6f9af9dfff6b admin admin] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:30:23.838891 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a2b5bc5-ae27-434b-8722-6f9af9dfff6b admin admin] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:30:23.845184 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5989ed23-6bb5-4f69-9ad0-6e41997e6bbe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '42d15fc0515945acacbbffcf2356c7a5', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:30:23.845889 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5989ed23-6bb5-4f69-9ad0-6e41997e6bbe admin admin] REQUEST_METHOD: `GET` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:30:23.846180 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5989ed23-6bb5-4f69-9ad0-6e41997e6bbe admin admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:30:23.846461 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5989ed23-6bb5-4f69-9ad0-6e41997e6bbe admin admin] PATH_INFO: `/v3/roles` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:30:23.850623 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5989ed23-6bb5-4f69-9ad0-6e41997e6bbe admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:30:23.854144 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5989ed23-6bb5-4f69-9ad0-6e41997e6bbe admin admin] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:30:23.891216 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 865/1716] 10.222.0.47 () {58 vars in 1121 bytes} [Thu Jul 21 08:30:23 2022] GET /identity/v3/roles => generated 1420 bytes in 77 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 08:30:23.908832 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f222c73e-69dd-4684-9914-ea73bd9cfdb9 admin admin] Authenticating user token {{(pid=80245) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:30:23.918335 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f222c73e-69dd-4684-9914-ea73bd9cfdb9 admin admin] Validating token access rules against request {{(pid=80245) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:30:23.934686 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-166dc432-42ac-4467-92e0-b18fe3386013 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '42d15fc0515945acacbbffcf2356c7a5', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80245) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:30:23.935966 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-166dc432-42ac-4467-92e0-b18fe3386013 admin admin] REQUEST_METHOD: `PUT` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:30:23.935966 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-166dc432-42ac-4467-92e0-b18fe3386013 admin admin] SCRIPT_NAME: `/identity` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:30:23.936797 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-166dc432-42ac-4467-92e0-b18fe3386013 admin admin] PATH_INFO: `/v3/projects/16c2903b2f4c473eb05b9a29920ae497/users/2d412a765e97490f9c83b00b1363209b/roles/3a074738221e4601b5205715ba87d240` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:30:23.991372 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-166dc432-42ac-4467-92e0-b18fe3386013 admin admin] RBAC: Authorizing `identity:create_grant(project_id=16c2903b2f4c473eb05b9a29920ae497, user_id=2d412a765e97490f9c83b00b1363209b, role_id=3a074738221e4601b5205715ba87d240)` {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:30:23.995018 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-166dc432-42ac-4467-92e0-b18fe3386013 admin admin] RBAC: Authorization granted {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:30:24.021752 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: [pid: 80245|app: 0|req: 852/1717] 10.222.0.47 () {60 vars in 1482 bytes} [Thu Jul 21 08:30:23 2022] PUT /identity/v3/projects/16c2903b2f4c473eb05b9a29920ae497/users/2d412a765e97490f9c83b00b1363209b/roles/3a074738221e4601b5205715ba87d240 => generated 0 bytes in 123 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 08:30:24.035676 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae86cac1-d3b6-4478-8cc5-693dd52dddb4 None None] REQUEST_METHOD: `POST` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:30:24.036196 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae86cac1-d3b6-4478-8cc5-693dd52dddb4 None None] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:30:24.036426 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae86cac1-d3b6-4478-8cc5-693dd52dddb4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:30:24.091908 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.auth.core [None req-ae86cac1-d3b6-4478-8cc5-693dd52dddb4 None None] MFA Rules not processed for user `2d412a765e97490f9c83b00b1363209b`. Rule list: `[]` (Enabled: `True`). {{(pid=80244) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 08:30:24.103388 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.fernet_utils [None req-ae86cac1-d3b6-4478-8cc5-693dd52dddb4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80244) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 08:30:24.144380 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG dbcounter [-] [80244] Writing DB stats keystone:DELETE=1,keystone:SELECT=96,keystone:INSERT=3 {{(pid=80244) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:30:24.310379 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 866/1718] 10.222.0.47 () {58 vars in 957 bytes} [Thu Jul 21 08:30:24 2022] POST /identity/v3/auth/tokens => generated 3234 bytes in 276 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 08:30:24.315693 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-166dc432-42ac-4467-92e0-b18fe3386013 admin admin] Authenticating user token {{(pid=80245) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:30:24.506760 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-166dc432-42ac-4467-92e0-b18fe3386013 admin admin] Validating token access rules against request {{(pid=80245) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:30:24.521157 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37971f3b-1e54-48e1-bf95-c359b01d34b1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '42d15fc0515945acacbbffcf2356c7a5', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80245) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:30:24.521942 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37971f3b-1e54-48e1-bf95-c359b01d34b1 admin admin] REQUEST_METHOD: `POST` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:30:24.522112 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37971f3b-1e54-48e1-bf95-c359b01d34b1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:30:24.522549 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37971f3b-1e54-48e1-bf95-c359b01d34b1 admin admin] PATH_INFO: `/v3/projects` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:30:24.523421 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37971f3b-1e54-48e1-bf95-c359b01d34b1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:30:24.527316 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37971f3b-1e54-48e1-bf95-c359b01d34b1 admin admin] RBAC: Authorization granted {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:30:24.608675 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: [pid: 80245|app: 0|req: 853/1719] 10.222.0.47 () {60 vars in 1152 bytes} [Thu Jul 21 08:30:24 2022] POST /identity/v3/projects => generated 369 bytes in 293 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 08:30:24.609078 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae86cac1-d3b6-4478-8cc5-693dd52dddb4 None None] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:30:24.640233 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae86cac1-d3b6-4478-8cc5-693dd52dddb4 None None] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:30:24.941091 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51214d14-6170-41ae-9e99-4ffdfead85af admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '42d15fc0515945acacbbffcf2356c7a5', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:30:24.941802 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51214d14-6170-41ae-9e99-4ffdfead85af admin admin] REQUEST_METHOD: `POST` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:30:24.942087 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51214d14-6170-41ae-9e99-4ffdfead85af admin admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:30:24.942436 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51214d14-6170-41ae-9e99-4ffdfead85af admin admin] PATH_INFO: `/v3/users` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:30:24.951376 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51214d14-6170-41ae-9e99-4ffdfead85af admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:30:24.954455 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51214d14-6170-41ae-9e99-4ffdfead85af admin admin] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:30:25.020735 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 867/1720] 10.222.0.47 () {60 vars in 1143 bytes} [Thu Jul 21 08:30:24 2022] POST /identity/v3/users => generated 291 bytes in 411 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 21 08:30:25.033138 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37971f3b-1e54-48e1-bf95-c359b01d34b1 admin admin] Authenticating user token {{(pid=80245) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:30:25.086193 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37971f3b-1e54-48e1-bf95-c359b01d34b1 admin admin] Validating token access rules against request {{(pid=80245) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:30:25.101293 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f12003cf-1f12-400d-bbcc-4a7491c0f8b8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '42d15fc0515945acacbbffcf2356c7a5', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80245) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:30:25.101987 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f12003cf-1f12-400d-bbcc-4a7491c0f8b8 admin admin] REQUEST_METHOD: `GET` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:30:25.102271 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f12003cf-1f12-400d-bbcc-4a7491c0f8b8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:30:25.102545 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f12003cf-1f12-400d-bbcc-4a7491c0f8b8 admin admin] PATH_INFO: `/v3/roles` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:30:25.103309 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f12003cf-1f12-400d-bbcc-4a7491c0f8b8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:30:25.105225 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f12003cf-1f12-400d-bbcc-4a7491c0f8b8 admin admin] RBAC: Authorization granted {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:30:25.122662 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: [pid: 80245|app: 0|req: 854/1721] 10.222.0.47 () {58 vars in 1121 bytes} [Thu Jul 21 08:30:25 2022] GET /identity/v3/roles => generated 1420 bytes in 90 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 08:30:25.127122 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51214d14-6170-41ae-9e99-4ffdfead85af admin admin] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:30:25.153196 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51214d14-6170-41ae-9e99-4ffdfead85af admin admin] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:30:25.171696 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2c15e58-82c3-4194-9ada-8ff954c96392 None None] REQUEST_METHOD: `GET` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:30:25.174519 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2c15e58-82c3-4194-9ada-8ff954c96392 None None] SCRIPT_NAME: `/identity` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:30:25.175119 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2c15e58-82c3-4194-9ada-8ff954c96392 None None] PATH_INFO: `/` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:30:25.177255 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: [pid: 80245|app: 0|req: 855/1722] 10.222.0.47 () {54 vars in 912 bytes} [Thu Jul 21 08:30:25 2022] GET /identity => generated 270 bytes in 12 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 08:30:25.180024 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c726eed9-e8e7-449e-ab91-d846a4df21fe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '42d15fc0515945acacbbffcf2356c7a5', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:30:25.181576 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c726eed9-e8e7-449e-ab91-d846a4df21fe admin admin] REQUEST_METHOD: `PUT` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:30:25.182980 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c726eed9-e8e7-449e-ab91-d846a4df21fe admin admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:30:25.183452 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c726eed9-e8e7-449e-ab91-d846a4df21fe admin admin] PATH_INFO: `/v3/projects/ebfbe7fe67e94ce0be4508e814147208/users/04c2ae4751b1424eb9ae873706a5150b/roles/e29870f6e9e2411a9db0d5bf45f32ca5` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:30:25.189721 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8d58d08-fddd-4702-a040-eced689cb896 None None] REQUEST_METHOD: `POST` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:30:25.190732 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8d58d08-fddd-4702-a040-eced689cb896 None None] SCRIPT_NAME: `/identity` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:30:25.191216 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8d58d08-fddd-4702-a040-eced689cb896 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:30:25.239908 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c726eed9-e8e7-449e-ab91-d846a4df21fe admin admin] RBAC: Authorizing `identity:create_grant(project_id=ebfbe7fe67e94ce0be4508e814147208, user_id=04c2ae4751b1424eb9ae873706a5150b, role_id=e29870f6e9e2411a9db0d5bf45f32ca5)` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:30:25.242617 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c726eed9-e8e7-449e-ab91-d846a4df21fe admin admin] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:30:25.272513 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 868/1723] 10.222.0.47 () {60 vars in 1482 bytes} [Thu Jul 21 08:30:25 2022] PUT /identity/v3/projects/ebfbe7fe67e94ce0be4508e814147208/users/04c2ae4751b1424eb9ae873706a5150b/roles/e29870f6e9e2411a9db0d5bf45f32ca5 => generated 0 bytes in 143 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 08:30:25.273666 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c726eed9-e8e7-449e-ab91-d846a4df21fe admin admin] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:30:25.280497 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.auth.core [None req-d8d58d08-fddd-4702-a040-eced689cb896 None None] MFA Rules not processed for user `20ca3c3bd3774691b3374cfbfec6e494`. Rule list: `[]` (Enabled: `True`). {{(pid=80245) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 08:30:25.284844 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.fernet_utils [None req-d8d58d08-fddd-4702-a040-eced689cb896 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80245) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 08:30:25.429014 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: [pid: 80245|app: 0|req: 856/1724] 10.222.0.47 () {58 vars in 1009 bytes} [Thu Jul 21 08:30:25 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 244 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 08:30:25.430534 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8d58d08-fddd-4702-a040-eced689cb896 None None] Authenticating user token {{(pid=80245) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:30:25.432879 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.fernet_utils [None req-d8d58d08-fddd-4702-a040-eced689cb896 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80245) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 08:30:25.451495 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG dbcounter [-] [80245] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80245) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:30:25.454354 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c726eed9-e8e7-449e-ab91-d846a4df21fe admin admin] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:30:25.462717 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1317b48-336c-4a7b-a479-78cc331f6e94 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '42d15fc0515945acacbbffcf2356c7a5', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:30:25.463685 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1317b48-336c-4a7b-a479-78cc331f6e94 admin admin] REQUEST_METHOD: `GET` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:30:25.464567 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1317b48-336c-4a7b-a479-78cc331f6e94 admin admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:30:25.464953 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1317b48-336c-4a7b-a479-78cc331f6e94 admin admin] PATH_INFO: `/v3/roles` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:30:25.466644 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1317b48-336c-4a7b-a479-78cc331f6e94 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:30:25.471069 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1317b48-336c-4a7b-a479-78cc331f6e94 admin admin] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:30:25.501235 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 869/1725] 10.222.0.47 () {58 vars in 1121 bytes} [Thu Jul 21 08:30:25 2022] GET /identity/v3/roles => generated 1420 bytes in 228 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 08:30:25.502892 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40c04b71-c8ca-45e8-a71d-d318b5fd103b None None] REQUEST_METHOD: `POST` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:30:25.503363 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40c04b71-c8ca-45e8-a71d-d318b5fd103b None None] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:30:25.504175 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40c04b71-c8ca-45e8-a71d-d318b5fd103b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:30:25.542553 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.auth.core [None req-40c04b71-c8ca-45e8-a71d-d318b5fd103b None None] MFA Rules not processed for user `2465817c3ba340f4870debf937929499`. Rule list: `[]` (Enabled: `True`). {{(pid=80244) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 08:30:25.544780 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.fernet_utils [None req-40c04b71-c8ca-45e8-a71d-d318b5fd103b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80244) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 08:30:25.555151 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG dbcounter [-] [80244] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80244) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:30:25.604300 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8d58d08-fddd-4702-a040-eced689cb896 None None] Validating token access rules against request {{(pid=80245) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:30:25.620959 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-079c90d3-73fc-415a-a063-29deee9320ca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '42d15fc0515945acacbbffcf2356c7a5', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80245) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:30:25.622191 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-079c90d3-73fc-415a-a063-29deee9320ca admin admin] REQUEST_METHOD: `DELETE` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:30:25.622699 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-079c90d3-73fc-415a-a063-29deee9320ca admin admin] SCRIPT_NAME: `/identity` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:30:25.624525 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-079c90d3-73fc-415a-a063-29deee9320ca admin admin] PATH_INFO: `/v3/projects/098fa33a679c4d10872207508b660352` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:30:25.627689 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-079c90d3-73fc-415a-a063-29deee9320ca admin admin] RBAC: Authorizing `identity:delete_project(project_id=098fa33a679c4d10872207508b660352)` {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:30:25.633216 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-079c90d3-73fc-415a-a063-29deee9320ca admin admin] RBAC: Authorization granted {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:30:25.661826 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 870/1726] 10.222.0.47 () {58 vars in 957 bytes} [Thu Jul 21 08:30:25 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 160 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 08:30:25.674867 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40c04b71-c8ca-45e8-a71d-d318b5fd103b None None] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:30:25.677038 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.fernet_utils [None req-40c04b71-c8ca-45e8-a71d-d318b5fd103b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80244) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 08:30:25.691300 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40c04b71-c8ca-45e8-a71d-d318b5fd103b None None] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:30:25.696882 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c31c30a-3962-43f0-84c5-558948830591 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20ca3c3bd3774691b3374cfbfec6e494', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4faef13aa50e4e36b6c265fb1398fa14', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:30:25.698394 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c31c30a-3962-43f0-84c5-558948830591 service neutron] REQUEST_METHOD: `GET` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:30:25.698857 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c31c30a-3962-43f0-84c5-558948830591 service neutron] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:30:25.699292 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c31c30a-3962-43f0-84c5-558948830591 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:30:25.708721 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c31c30a-3962-43f0-84c5-558948830591 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:30:25.711421 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c31c30a-3962-43f0-84c5-558948830591 service neutron] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:30:25.726139 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 871/1727] 10.222.0.47 () {60 vars in 1365 bytes} [Thu Jul 21 08:30:25 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 52 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 08:30:25.728168 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c31c30a-3962-43f0-84c5-558948830591 service neutron] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:30:25.753328 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c31c30a-3962-43f0-84c5-558948830591 service neutron] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:30:25.787553 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.notifications [None req-079c90d3-73fc-415a-a063-29deee9320ca admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '098fa33a679c4d10872207508b660352', 'request_id': 'req-079c90d3-73fc-415a-a063-29deee9320ca'} {{(pid=80245) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 08:30:25.813708 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.notifications [None req-079c90d3-73fc-415a-a063-29deee9320ca admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '098fa33a679c4d10872207508b660352', 'request_id': 'req-079c90d3-73fc-415a-a063-29deee9320ca'} {{(pid=80245) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 08:30:25.837051 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.notifications [None req-079c90d3-73fc-415a-a063-29deee9320ca admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '098fa33a679c4d10872207508b660352', 'request_id': 'req-079c90d3-73fc-415a-a063-29deee9320ca'} {{(pid=80245) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 08:30:25.869213 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fa84506-409a-40bc-941b-2c89a31c67b3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '42d15fc0515945acacbbffcf2356c7a5', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:30:25.871941 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fa84506-409a-40bc-941b-2c89a31c67b3 admin admin] REQUEST_METHOD: `PUT` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:30:25.872444 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fa84506-409a-40bc-941b-2c89a31c67b3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:30:25.872831 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fa84506-409a-40bc-941b-2c89a31c67b3 admin admin] PATH_INFO: `/v3/projects/ebfbe7fe67e94ce0be4508e814147208/users/04c2ae4751b1424eb9ae873706a5150b/roles/e29870f6e9e2411a9db0d5bf45f32ca5` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:30:25.883041 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fa84506-409a-40bc-941b-2c89a31c67b3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ebfbe7fe67e94ce0be4508e814147208, user_id=04c2ae4751b1424eb9ae873706a5150b, role_id=e29870f6e9e2411a9db0d5bf45f32ca5)` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:30:25.886325 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fa84506-409a-40bc-941b-2c89a31c67b3 admin admin] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:30:25.886697 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.notifications [None req-079c90d3-73fc-415a-a063-29deee9320ca admin admin] The token cache is being invalidate because project 098fa33a679c4d10872207508b660352 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80245) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 21 08:30:25.887017 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.notifications [None req-079c90d3-73fc-415a-a063-29deee9320ca admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80245) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 08:30:25.896861 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.notifications [None req-079c90d3-73fc-415a-a063-29deee9320ca admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80245) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 08:30:25.905248 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: [pid: 80245|app: 0|req: 857/1728] 10.222.0.47 () {58 vars in 1232 bytes} [Thu Jul 21 08:30:25 2022] DELETE /identity/v3/projects/098fa33a679c4d10872207508b660352 => generated 0 bytes in 475 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 08:30:25.912503 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-673a7bfc-fd2b-43d1-be89-6b74ac6baefb None None] REQUEST_METHOD: `POST` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:30:25.913113 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-673a7bfc-fd2b-43d1-be89-6b74ac6baefb None None] SCRIPT_NAME: `/identity` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:30:25.914635 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-673a7bfc-fd2b-43d1-be89-6b74ac6baefb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:30:25.939113 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 872/1729] 10.222.0.47 () {60 vars in 1482 bytes} [Thu Jul 21 08:30:25 2022] PUT /identity/v3/projects/ebfbe7fe67e94ce0be4508e814147208/users/04c2ae4751b1424eb9ae873706a5150b/roles/e29870f6e9e2411a9db0d5bf45f32ca5 => generated 0 bytes in 211 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 08:30:25.944461 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1999e7f2-d2ed-470b-b97e-017ec77df749 None None] REQUEST_METHOD: `POST` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:30:25.944779 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1999e7f2-d2ed-470b-b97e-017ec77df749 None None] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:30:25.949470 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1999e7f2-d2ed-470b-b97e-017ec77df749 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:30:26.019655 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.auth.core [None req-673a7bfc-fd2b-43d1-be89-6b74ac6baefb None None] MFA Rules not processed for user `2465817c3ba340f4870debf937929499`. Rule list: `[]` (Enabled: `True`). {{(pid=80245) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 08:30:26.024740 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.fernet_utils [None req-673a7bfc-fd2b-43d1-be89-6b74ac6baefb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80245) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 08:30:26.044465 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.auth.core [None req-1999e7f2-d2ed-470b-b97e-017ec77df749 None None] MFA Rules not processed for user `2dce40b8d74c41ec94cc36e6ff05d412`. Rule list: `[]` (Enabled: `True`). {{(pid=80244) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 08:30:26.045765 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.fernet_utils [None req-1999e7f2-d2ed-470b-b97e-017ec77df749 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80244) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 08:30:26.257117 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: [pid: 80245|app: 0|req: 858/1730] 10.222.0.47 () {58 vars in 957 bytes} [Thu Jul 21 08:30:25 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 351 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 08:30:26.261888 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baa612e1-63aa-49fc-8d67-615b66a72f3f None None] REQUEST_METHOD: `POST` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:30:26.262210 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baa612e1-63aa-49fc-8d67-615b66a72f3f None None] SCRIPT_NAME: `/identity` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:30:26.263977 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baa612e1-63aa-49fc-8d67-615b66a72f3f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:30:26.285875 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 873/1731] 10.222.0.47 () {58 vars in 957 bytes} [Thu Jul 21 08:30:25 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 347 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 08:30:26.286817 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1999e7f2-d2ed-470b-b97e-017ec77df749 None None] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:30:26.296438 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.fernet_utils [None req-1999e7f2-d2ed-470b-b97e-017ec77df749 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80244) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 08:30:26.360163 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.auth.core [None req-baa612e1-63aa-49fc-8d67-615b66a72f3f None None] MFA Rules not processed for user `04c2ae4751b1424eb9ae873706a5150b`. Rule list: `[]` (Enabled: `True`). {{(pid=80245) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 08:30:26.364265 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.fernet_utils [None req-baa612e1-63aa-49fc-8d67-615b66a72f3f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80245) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 08:30:26.442791 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG dbcounter [-] [80244] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80244) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:30:26.493544 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG dbcounter [-] [80245] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80245) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:30:26.580450 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1999e7f2-d2ed-470b-b97e-017ec77df749 None None] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:30:26.585883 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: [pid: 80245|app: 0|req: 859/1732] 10.222.0.47 () {58 vars in 957 bytes} [Thu Jul 21 08:30:26 2022] POST /identity/v3/auth/tokens => generated 3303 bytes in 325 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 08:30:26.585883 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b96a91f-fd31-41bf-8995-b601389f50bb None None] REQUEST_METHOD: `POST` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:30:26.586974 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b96a91f-fd31-41bf-8995-b601389f50bb None None] SCRIPT_NAME: `/identity` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:30:26.586974 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b96a91f-fd31-41bf-8995-b601389f50bb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:30:26.595715 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-826f3b9d-8ebd-4f9d-9b7c-803d6f152909 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2b5b99a36442148fe743269054e991', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4faef13aa50e4e36b6c265fb1398fa14', 'project_domain_id': 'default', 'roles': ['member', 'service', 'ResellerAdmin', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:30:26.596732 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-826f3b9d-8ebd-4f9d-9b7c-803d6f152909 service nova] REQUEST_METHOD: `GET` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:30:26.597157 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-826f3b9d-8ebd-4f9d-9b7c-803d6f152909 service nova] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:30:26.597585 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-826f3b9d-8ebd-4f9d-9b7c-803d6f152909 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:30:26.603386 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.fernet_utils [None req-826f3b9d-8ebd-4f9d-9b7c-803d6f152909 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80244) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 08:30:26.628993 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-826f3b9d-8ebd-4f9d-9b7c-803d6f152909 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:30:26.630948 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-826f3b9d-8ebd-4f9d-9b7c-803d6f152909 service nova] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:30:26.634005 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.auth.core [None req-1b96a91f-fd31-41bf-8995-b601389f50bb None None] MFA Rules not processed for user `2dce40b8d74c41ec94cc36e6ff05d412`. Rule list: `[]` (Enabled: `True`). {{(pid=80245) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 08:30:26.635301 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.fernet_utils [None req-1b96a91f-fd31-41bf-8995-b601389f50bb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80245) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 08:30:26.652885 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: [pid: 80245|app: 0|req: 860/1733] 10.222.0.47 () {58 vars in 957 bytes} [Thu Jul 21 08:30:26 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 67 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 08:30:26.652885 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2b90757-0f23-4968-8fe6-2ad6758be923 None None] REQUEST_METHOD: `POST` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:30:26.652885 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2b90757-0f23-4968-8fe6-2ad6758be923 None None] SCRIPT_NAME: `/identity` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:30:26.652885 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2b90757-0f23-4968-8fe6-2ad6758be923 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:30:26.660064 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 874/1734] 10.222.0.47 () {60 vars in 1365 bytes} [Thu Jul 21 08:30:26 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 373 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 08:30:26.660972 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-826f3b9d-8ebd-4f9d-9b7c-803d6f152909 service nova] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:30:26.668411 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.fernet_utils [None req-826f3b9d-8ebd-4f9d-9b7c-803d6f152909 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80244) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 08:30:26.691916 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.auth.core [None req-e2b90757-0f23-4968-8fe6-2ad6758be923 None None] MFA Rules not processed for user `3528104f3e914d62a10e80e13d3c97de`. Rule list: `[]` (Enabled: `True`). {{(pid=80245) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 08:30:26.693995 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.fernet_utils [None req-e2b90757-0f23-4968-8fe6-2ad6758be923 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80245) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 08:30:26.840583 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-826f3b9d-8ebd-4f9d-9b7c-803d6f152909 service nova] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:30:26.859424 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f081685a-5835-4ee8-9cd3-434d1dbc9614 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4f40410b7674687ab5362ac982248ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4faef13aa50e4e36b6c265fb1398fa14', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:30:26.860079 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f081685a-5835-4ee8-9cd3-434d1dbc9614 service cinder] REQUEST_METHOD: `GET` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:30:26.860398 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f081685a-5835-4ee8-9cd3-434d1dbc9614 service cinder] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:30:26.860590 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f081685a-5835-4ee8-9cd3-434d1dbc9614 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:30:26.886828 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f081685a-5835-4ee8-9cd3-434d1dbc9614 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:30:26.890821 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f081685a-5835-4ee8-9cd3-434d1dbc9614 service cinder] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:30:26.910165 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 875/1735] 10.222.0.47 () {60 vars in 1365 bytes} [Thu Jul 21 08:30:26 2022] GET /identity/v3/auth/tokens => generated 3303 bytes in 249 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 08:30:26.911045 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f081685a-5835-4ee8-9cd3-434d1dbc9614 service cinder] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:30:26.922535 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f081685a-5835-4ee8-9cd3-434d1dbc9614 service cinder] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:30:26.925153 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: [pid: 80245|app: 0|req: 861/1736] 10.222.0.47 () {58 vars in 957 bytes} [Thu Jul 21 08:30:26 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 274 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 08:30:26.935907 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65ecabf2-2c58-460c-ac3e-1208c5ec1575 None None] REQUEST_METHOD: `POST` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:30:26.937182 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65ecabf2-2c58-460c-ac3e-1208c5ec1575 None None] SCRIPT_NAME: `/identity` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:30:26.937675 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65ecabf2-2c58-460c-ac3e-1208c5ec1575 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:30:26.942198 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9eb0857a-2fc5-408b-b623-415fbd9d3035 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '42d15fc0515945acacbbffcf2356c7a5', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:30:26.942872 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eb0857a-2fc5-408b-b623-415fbd9d3035 admin admin] REQUEST_METHOD: `GET` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:30:26.943145 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eb0857a-2fc5-408b-b623-415fbd9d3035 admin admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:30:26.943448 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eb0857a-2fc5-408b-b623-415fbd9d3035 admin admin] PATH_INFO: `/v3/domains` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:30:26.944184 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9eb0857a-2fc5-408b-b623-415fbd9d3035 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80244) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 08:30:26.944487 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9eb0857a-2fc5-408b-b623-415fbd9d3035 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:30:26.946099 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9eb0857a-2fc5-408b-b623-415fbd9d3035 admin admin] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:30:26.982046 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.auth.core [None req-65ecabf2-2c58-460c-ac3e-1208c5ec1575 None None] MFA Rules not processed for user `3528104f3e914d62a10e80e13d3c97de`. Rule list: `[]` (Enabled: `True`). {{(pid=80245) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 08:30:26.989058 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.fernet_utils [None req-65ecabf2-2c58-460c-ac3e-1208c5ec1575 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80245) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 08:30:26.994047 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 876/1737] 10.222.0.47 () {58 vars in 1152 bytes} [Thu Jul 21 08:30:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 83 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 08:30:27.002589 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9eb0857a-2fc5-408b-b623-415fbd9d3035 admin admin] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:30:27.021936 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: [pid: 80245|app: 0|req: 862/1738] 10.222.0.47 () {58 vars in 957 bytes} [Thu Jul 21 08:30:26 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 08:30:27.023179 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1899f3f-08a7-4d07-b405-39d739717d58 None None] REQUEST_METHOD: `GET` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:30:27.025169 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1899f3f-08a7-4d07-b405-39d739717d58 None None] SCRIPT_NAME: `/identity` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:30:27.025169 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1899f3f-08a7-4d07-b405-39d739717d58 None None] PATH_INFO: `/` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:30:27.025776 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: [pid: 80245|app: 0|req: 863/1739] 10.222.0.47 () {54 vars in 961 bytes} [Thu Jul 21 08:30:27 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 08:30:27.026053 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9eb0857a-2fc5-408b-b623-415fbd9d3035 admin admin] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:30:27.033013 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-754d6383-b530-49ff-8cb7-7754c17080af None None] REQUEST_METHOD: `POST` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:30:27.033313 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-754d6383-b530-49ff-8cb7-7754c17080af None None] SCRIPT_NAME: `/identity` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:30:27.033576 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-754d6383-b530-49ff-8cb7-7754c17080af None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:30:27.056820 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc1eea4d-4dd3-4fc9-b029-da6566ffaf2d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '42d15fc0515945acacbbffcf2356c7a5', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:30:27.058076 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc1eea4d-4dd3-4fc9-b029-da6566ffaf2d admin admin] REQUEST_METHOD: `POST` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:30:27.058361 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc1eea4d-4dd3-4fc9-b029-da6566ffaf2d admin admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:30:27.058641 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc1eea4d-4dd3-4fc9-b029-da6566ffaf2d admin admin] PATH_INFO: `/v3/projects` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:30:27.059407 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc1eea4d-4dd3-4fc9-b029-da6566ffaf2d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:30:27.062041 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc1eea4d-4dd3-4fc9-b029-da6566ffaf2d admin admin] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:30:27.069792 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.auth.core [None req-754d6383-b530-49ff-8cb7-7754c17080af None None] MFA Rules not processed for user `3379b32543474d0a8e897484e85c7d04`. Rule list: `[]` (Enabled: `True`). {{(pid=80245) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 08:30:27.071044 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.fernet_utils [None req-754d6383-b530-49ff-8cb7-7754c17080af None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80245) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 08:30:27.147615 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 877/1740] 10.222.0.47 () {60 vars in 1152 bytes} [Thu Jul 21 08:30:26 2022] POST /identity/v3/projects => generated 385 bytes in 147 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 21 08:30:27.152004 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc1eea4d-4dd3-4fc9-b029-da6566ffaf2d admin admin] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:30:27.246181 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: [pid: 80245|app: 0|req: 864/1741] 10.222.0.47 () {58 vars in 1058 bytes} [Thu Jul 21 08:30:27 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 214 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 21 08:30:27.247546 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-754d6383-b530-49ff-8cb7-7754c17080af None None] Authenticating user token {{(pid=80245) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:30:27.260577 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.fernet_utils [None req-754d6383-b530-49ff-8cb7-7754c17080af None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80245) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 08:30:27.293240 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-754d6383-b530-49ff-8cb7-7754c17080af None None] Validating token access rules against request {{(pid=80245) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:30:27.392479 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc1eea4d-4dd3-4fc9-b029-da6566ffaf2d admin admin] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:30:27.405834 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50efc69a-dedb-4adc-82ba-f10baf706553 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '42d15fc0515945acacbbffcf2356c7a5', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:30:27.406548 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50efc69a-dedb-4adc-82ba-f10baf706553 admin admin] REQUEST_METHOD: `POST` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:30:27.406839 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50efc69a-dedb-4adc-82ba-f10baf706553 admin admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:30:27.407253 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50efc69a-dedb-4adc-82ba-f10baf706553 admin admin] PATH_INFO: `/v3/users` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:30:27.410456 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50efc69a-dedb-4adc-82ba-f10baf706553 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:30:27.415359 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50efc69a-dedb-4adc-82ba-f10baf706553 admin admin] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:30:27.436742 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG dbcounter [-] [80245] Writing DB stats keystone:SELECT=100 {{(pid=80245) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:30:27.466354 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 878/1742] 10.222.0.47 () {60 vars in 1143 bytes} [Thu Jul 21 08:30:27 2022] POST /identity/v3/users => generated 293 bytes in 314 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 08:30:27.467765 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50efc69a-dedb-4adc-82ba-f10baf706553 admin admin] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:30:27.510801 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG dbcounter [-] [80244] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80244) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:30:27.512728 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-256fd31d-a404-4264-b25d-c030a67ac637 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2b5b99a36442148fe743269054e991', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4faef13aa50e4e36b6c265fb1398fa14', 'project_domain_id': 'default', 'roles': ['member', 'service', 'ResellerAdmin', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80245) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:30:27.513641 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-256fd31d-a404-4264-b25d-c030a67ac637 service nova] REQUEST_METHOD: `GET` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:30:27.513932 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-256fd31d-a404-4264-b25d-c030a67ac637 service nova] SCRIPT_NAME: `/identity` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:30:27.514930 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-256fd31d-a404-4264-b25d-c030a67ac637 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:30:27.671174 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50efc69a-dedb-4adc-82ba-f10baf706553 admin admin] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:30:27.673964 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-256fd31d-a404-4264-b25d-c030a67ac637 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:30:27.675773 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-256fd31d-a404-4264-b25d-c030a67ac637 service nova] RBAC: Authorization granted {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:30:27.685304 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7051bf1-3716-4ad3-95e7-b9fc5c26da2f service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3379b32543474d0a8e897484e85c7d04', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4faef13aa50e4e36b6c265fb1398fa14', 'project_domain_id': 'default', 'roles': ['service', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:30:27.686849 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7051bf1-3716-4ad3-95e7-b9fc5c26da2f service placement] REQUEST_METHOD: `GET` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:30:27.688175 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7051bf1-3716-4ad3-95e7-b9fc5c26da2f service placement] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:30:27.688663 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7051bf1-3716-4ad3-95e7-b9fc5c26da2f service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:30:27.691853 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.fernet_utils [None req-c7051bf1-3716-4ad3-95e7-b9fc5c26da2f service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80244) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 08:30:27.704777 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: [pid: 80245|app: 0|req: 865/1743] 10.222.0.47 () {60 vars in 1365 bytes} [Thu Jul 21 08:30:27 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 458 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 08:30:27.705881 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-256fd31d-a404-4264-b25d-c030a67ac637 service nova] Authenticating user token {{(pid=80245) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:30:27.719625 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-256fd31d-a404-4264-b25d-c030a67ac637 service nova] Validating token access rules against request {{(pid=80245) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:30:27.719774 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7051bf1-3716-4ad3-95e7-b9fc5c26da2f service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:30:27.724501 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be44f1e7-1ede-494a-8b9c-0d5b9c7a8e01 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '42d15fc0515945acacbbffcf2356c7a5', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80245) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:30:27.725193 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be44f1e7-1ede-494a-8b9c-0d5b9c7a8e01 admin admin] REQUEST_METHOD: `GET` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:30:27.725529 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be44f1e7-1ede-494a-8b9c-0d5b9c7a8e01 admin admin] SCRIPT_NAME: `/identity` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:30:27.725637 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7051bf1-3716-4ad3-95e7-b9fc5c26da2f service placement] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:30:27.725807 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be44f1e7-1ede-494a-8b9c-0d5b9c7a8e01 admin admin] PATH_INFO: `/v3/roles` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:30:27.726578 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be44f1e7-1ede-494a-8b9c-0d5b9c7a8e01 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:30:27.728407 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be44f1e7-1ede-494a-8b9c-0d5b9c7a8e01 admin admin] RBAC: Authorization granted {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:30:27.753701 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 879/1744] 10.222.0.47 () {60 vars in 1365 bytes} [Thu Jul 21 08:30:27 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 287 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 08:30:27.753885 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: [pid: 80245|app: 0|req: 866/1745] 10.222.0.47 () {58 vars in 1121 bytes} [Thu Jul 21 08:30:27 2022] GET /identity/v3/roles => generated 1420 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 08:30:27.755047 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7051bf1-3716-4ad3-95e7-b9fc5c26da2f service placement] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:30:27.755723 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2611200-636a-4c17-92fa-b406d51c098f None None] REQUEST_METHOD: `GET` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:30:27.756014 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2611200-636a-4c17-92fa-b406d51c098f None None] SCRIPT_NAME: `/identity` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:30:27.756294 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2611200-636a-4c17-92fa-b406d51c098f None None] PATH_INFO: `/` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:30:27.760475 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.fernet_utils [None req-c7051bf1-3716-4ad3-95e7-b9fc5c26da2f service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80244) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 08:30:27.767788 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: [pid: 80245|app: 0|req: 867/1746] 10.222.0.47 () {54 vars in 912 bytes} [Thu Jul 21 08:30:27 2022] GET /identity => generated 270 bytes in 13 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 08:30:27.768736 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2611200-636a-4c17-92fa-b406d51c098f None None] Authenticating user token {{(pid=80245) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:30:27.796262 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7051bf1-3716-4ad3-95e7-b9fc5c26da2f service placement] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:30:27.808659 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa5d3230-ca89-450f-8d1d-28c92061d8c2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '42d15fc0515945acacbbffcf2356c7a5', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:30:27.809560 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa5d3230-ca89-450f-8d1d-28c92061d8c2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:30:27.809855 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa5d3230-ca89-450f-8d1d-28c92061d8c2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:30:27.810134 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa5d3230-ca89-450f-8d1d-28c92061d8c2 admin admin] PATH_INFO: `/v3/users/75d3a8a0833b4b228499ad834d978e2a` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:30:27.820963 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa5d3230-ca89-450f-8d1d-28c92061d8c2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=75d3a8a0833b4b228499ad834d978e2a)` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:30:27.826525 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2611200-636a-4c17-92fa-b406d51c098f None None] Validating token access rules against request {{(pid=80245) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:30:27.827491 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa5d3230-ca89-450f-8d1d-28c92061d8c2 admin admin] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:30:27.856378 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a1da6a5-fd64-440c-8dae-2027551b3f6e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '42d15fc0515945acacbbffcf2356c7a5', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80245) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:30:27.857080 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a1da6a5-fd64-440c-8dae-2027551b3f6e admin admin] REQUEST_METHOD: `GET` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:30:27.857365 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a1da6a5-fd64-440c-8dae-2027551b3f6e admin admin] SCRIPT_NAME: `/identity` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:30:27.857643 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a1da6a5-fd64-440c-8dae-2027551b3f6e admin admin] PATH_INFO: `/v3/roles` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:30:27.858909 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a1da6a5-fd64-440c-8dae-2027551b3f6e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:30:27.860590 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a1da6a5-fd64-440c-8dae-2027551b3f6e admin admin] RBAC: Authorization granted {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:30:27.904454 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: [pid: 80245|app: 0|req: 868/1747] 10.222.0.47 () {58 vars in 1121 bytes} [Thu Jul 21 08:30:27 2022] GET /identity/v3/roles => generated 1420 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 08:30:27.909354 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a5d78cd-c485-4b26-880e-02f089a48e9e None None] REQUEST_METHOD: `POST` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:30:27.909648 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a5d78cd-c485-4b26-880e-02f089a48e9e None None] SCRIPT_NAME: `/identity` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:30:27.909927 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a5d78cd-c485-4b26-880e-02f089a48e9e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:30:27.975756 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.auth.core [None req-4a5d78cd-c485-4b26-880e-02f089a48e9e None None] MFA Rules not processed for user `20ca3c3bd3774691b3374cfbfec6e494`. Rule list: `[]` (Enabled: `True`). {{(pid=80245) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 08:30:27.978387 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.notifications [None req-aa5d3230-ca89-450f-8d1d-28c92061d8c2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '75d3a8a0833b4b228499ad834d978e2a'} {{(pid=80244) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 08:30:27.978848 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.fernet_utils [None req-4a5d78cd-c485-4b26-880e-02f089a48e9e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80245) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 08:30:27.995265 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.notifications [None req-aa5d3230-ca89-450f-8d1d-28c92061d8c2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '75d3a8a0833b4b228499ad834d978e2a'} {{(pid=80244) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 08:30:28.039432 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.notifications [None req-aa5d3230-ca89-450f-8d1d-28c92061d8c2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '75d3a8a0833b4b228499ad834d978e2a'} {{(pid=80244) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 08:30:28.040246 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.notifications [None req-aa5d3230-ca89-450f-8d1d-28c92061d8c2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '75d3a8a0833b4b228499ad834d978e2a'} {{(pid=80244) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 08:30:28.056396 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.notifications [None req-aa5d3230-ca89-450f-8d1d-28c92061d8c2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '75d3a8a0833b4b228499ad834d978e2a'} {{(pid=80244) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 08:30:28.100170 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: [pid: 80245|app: 0|req: 869/1748] 10.222.0.47 () {58 vars in 1009 bytes} [Thu Jul 21 08:30:27 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 195 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 08:30:28.101148 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a5d78cd-c485-4b26-880e-02f089a48e9e None None] Authenticating user token {{(pid=80245) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:30:28.104979 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 880/1749] 10.222.0.47 () {58 vars in 1223 bytes} [Thu Jul 21 08:30:27 2022] DELETE /identity/v3/users/75d3a8a0833b4b228499ad834d978e2a => generated 0 bytes in 351 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 08:30:28.107961 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.fernet_utils [None req-4a5d78cd-c485-4b26-880e-02f089a48e9e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80245) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 08:30:28.111852 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea4a8d85-5184-401d-a8dd-151c7272fea1 None None] REQUEST_METHOD: `POST` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:30:28.126696 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea4a8d85-5184-401d-a8dd-151c7272fea1 None None] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:30:28.127121 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea4a8d85-5184-401d-a8dd-151c7272fea1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:30:28.150438 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a5d78cd-c485-4b26-880e-02f089a48e9e None None] Validating token access rules against request {{(pid=80245) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:30:28.183941 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.auth.core [None req-ea4a8d85-5184-401d-a8dd-151c7272fea1 None None] MFA Rules not processed for user `2dce40b8d74c41ec94cc36e6ff05d412`. Rule list: `[]` (Enabled: `True`). {{(pid=80244) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 08:30:28.185522 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.fernet_utils [None req-ea4a8d85-5184-401d-a8dd-151c7272fea1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80244) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 08:30:28.425740 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6222e8db-3fe0-4fbd-8faa-1c52704d307f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '42d15fc0515945acacbbffcf2356c7a5', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80245) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:30:28.426641 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6222e8db-3fe0-4fbd-8faa-1c52704d307f admin admin] REQUEST_METHOD: `PUT` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:30:28.427731 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6222e8db-3fe0-4fbd-8faa-1c52704d307f admin admin] SCRIPT_NAME: `/identity` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:30:28.428031 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6222e8db-3fe0-4fbd-8faa-1c52704d307f admin admin] PATH_INFO: `/v3/projects/f1bc6cc5583649de8d9149eaa4317ae3/users/7c9ed461f0e5488f84eef4ede64db72f/roles/3a074738221e4601b5205715ba87d240` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:30:28.479277 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG dbcounter [-] [80245] Writing DB stats keystone:SELECT=100 {{(pid=80245) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:30:28.486667 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6222e8db-3fe0-4fbd-8faa-1c52704d307f admin admin] RBAC: Authorizing `identity:create_grant(project_id=f1bc6cc5583649de8d9149eaa4317ae3, user_id=7c9ed461f0e5488f84eef4ede64db72f, role_id=3a074738221e4601b5205715ba87d240)` {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:30:28.489426 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6222e8db-3fe0-4fbd-8faa-1c52704d307f admin admin] RBAC: Authorization granted {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:30:28.505983 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 881/1750] 10.222.0.47 () {58 vars in 957 bytes} [Thu Jul 21 08:30:28 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 395 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 08:30:28.512735 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea4a8d85-5184-401d-a8dd-151c7272fea1 None None] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:30:28.517031 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.fernet_utils [None req-ea4a8d85-5184-401d-a8dd-151c7272fea1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80244) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 08:30:28.547860 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: [pid: 80245|app: 0|req: 870/1751] 10.222.0.47 () {60 vars in 1482 bytes} [Thu Jul 21 08:30:28 2022] PUT /identity/v3/projects/f1bc6cc5583649de8d9149eaa4317ae3/users/7c9ed461f0e5488f84eef4ede64db72f/roles/3a074738221e4601b5205715ba87d240 => generated 0 bytes in 437 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 08:30:28.547860 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6222e8db-3fe0-4fbd-8faa-1c52704d307f admin admin] Authenticating user token {{(pid=80245) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:30:28.547860 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.fernet_utils [None req-6222e8db-3fe0-4fbd-8faa-1c52704d307f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80245) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 08:30:28.590248 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG dbcounter [-] [80244] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80244) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:30:28.744608 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea4a8d85-5184-401d-a8dd-151c7272fea1 None None] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:30:28.791525 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6222e8db-3fe0-4fbd-8faa-1c52704d307f admin admin] Validating token access rules against request {{(pid=80245) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:30:28.814491 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8b82b1c-1c5b-4c36-94eb-127c96713b79 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20ca3c3bd3774691b3374cfbfec6e494', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4faef13aa50e4e36b6c265fb1398fa14', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80245) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:30:28.815440 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8b82b1c-1c5b-4c36-94eb-127c96713b79 service neutron] REQUEST_METHOD: `GET` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:30:28.816286 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8b82b1c-1c5b-4c36-94eb-127c96713b79 service neutron] SCRIPT_NAME: `/identity` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:30:28.816801 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8b82b1c-1c5b-4c36-94eb-127c96713b79 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:30:28.886073 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06811756-fca2-4e96-ab63-692b1c160342 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20ca3c3bd3774691b3374cfbfec6e494', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4faef13aa50e4e36b6c265fb1398fa14', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:30:28.886829 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06811756-fca2-4e96-ab63-692b1c160342 service neutron] REQUEST_METHOD: `GET` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:30:28.887127 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06811756-fca2-4e96-ab63-692b1c160342 service neutron] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:30:28.887443 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06811756-fca2-4e96-ab63-692b1c160342 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:30:28.913443 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06811756-fca2-4e96-ab63-692b1c160342 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:30:28.916282 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06811756-fca2-4e96-ab63-692b1c160342 service neutron] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:30:28.971996 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 882/1752] 10.222.0.47 () {60 vars in 1365 bytes} [Thu Jul 21 08:30:28 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 460 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 08:30:28.976091 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c323ebde-71f9-42c0-afdf-5fc6bd7503fe None None] REQUEST_METHOD: `POST` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:30:28.979049 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c323ebde-71f9-42c0-afdf-5fc6bd7503fe None None] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:30:28.979526 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c323ebde-71f9-42c0-afdf-5fc6bd7503fe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:30:29.061509 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.auth.core [None req-c323ebde-71f9-42c0-afdf-5fc6bd7503fe None None] MFA Rules not processed for user `7c9ed461f0e5488f84eef4ede64db72f`. Rule list: `[]` (Enabled: `True`). {{(pid=80244) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 08:30:29.062828 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.fernet_utils [None req-c323ebde-71f9-42c0-afdf-5fc6bd7503fe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80244) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 08:30:29.137060 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8b82b1c-1c5b-4c36-94eb-127c96713b79 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:30:29.147954 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8b82b1c-1c5b-4c36-94eb-127c96713b79 service neutron] RBAC: Authorization granted {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:30:29.213302 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: [pid: 80245|app: 0|req: 871/1753] 10.222.0.47 () {60 vars in 1365 bytes} [Thu Jul 21 08:30:28 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 675 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 08:30:29.251886 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 883/1754] 10.222.0.47 () {58 vars in 957 bytes} [Thu Jul 21 08:30:28 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 279 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 08:30:29.255701 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8b82b1c-1c5b-4c36-94eb-127c96713b79 service neutron] Authenticating user token {{(pid=80245) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:30:29.272714 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8b82b1c-1c5b-4c36-94eb-127c96713b79 service neutron] Validating token access rules against request {{(pid=80245) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:30:29.288694 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f6fdde3-6760-47f6-96bc-719821b55e0d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '42d15fc0515945acacbbffcf2356c7a5', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80245) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:30:29.289484 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f6fdde3-6760-47f6-96bc-719821b55e0d admin admin] REQUEST_METHOD: `POST` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:30:29.291660 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f6fdde3-6760-47f6-96bc-719821b55e0d admin admin] SCRIPT_NAME: `/identity` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:30:29.292119 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f6fdde3-6760-47f6-96bc-719821b55e0d admin admin] PATH_INFO: `/v3/projects` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:30:29.292991 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f6fdde3-6760-47f6-96bc-719821b55e0d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:30:29.297309 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f6fdde3-6760-47f6-96bc-719821b55e0d admin admin] RBAC: Authorization granted {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:30:29.384084 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: [pid: 80245|app: 0|req: 872/1755] 10.222.0.47 () {60 vars in 1152 bytes} [Thu Jul 21 08:30:29 2022] POST /identity/v3/projects => generated 381 bytes in 129 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 08:30:29.388571 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c323ebde-71f9-42c0-afdf-5fc6bd7503fe None None] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:30:29.410712 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c323ebde-71f9-42c0-afdf-5fc6bd7503fe None None] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:30:29.666874 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72aa7027-d76d-4b09-a2cd-20a6c7fa2ba0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '42d15fc0515945acacbbffcf2356c7a5', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:30:29.667848 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72aa7027-d76d-4b09-a2cd-20a6c7fa2ba0 admin admin] REQUEST_METHOD: `POST` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:30:29.668320 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72aa7027-d76d-4b09-a2cd-20a6c7fa2ba0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:30:29.668782 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72aa7027-d76d-4b09-a2cd-20a6c7fa2ba0 admin admin] PATH_INFO: `/v3/users` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:30:29.669660 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72aa7027-d76d-4b09-a2cd-20a6c7fa2ba0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:30:29.672158 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72aa7027-d76d-4b09-a2cd-20a6c7fa2ba0 admin admin] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:30:29.722724 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG dbcounter [-] [80244] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80244) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:30:29.754687 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 884/1756] 10.222.0.47 () {60 vars in 1143 bytes} [Thu Jul 21 08:30:29 2022] POST /identity/v3/users => generated 291 bytes in 367 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 08:30:29.758468 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f6fdde3-6760-47f6-96bc-719821b55e0d admin admin] Authenticating user token {{(pid=80245) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:30:29.772727 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f6fdde3-6760-47f6-96bc-719821b55e0d admin admin] Validating token access rules against request {{(pid=80245) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:30:29.799986 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f3f89cb-7583-4ef5-99c4-af7a4b8c2269 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '42d15fc0515945acacbbffcf2356c7a5', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80245) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:30:29.800924 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f3f89cb-7583-4ef5-99c4-af7a4b8c2269 admin admin] REQUEST_METHOD: `GET` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:30:29.801443 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f3f89cb-7583-4ef5-99c4-af7a4b8c2269 admin admin] SCRIPT_NAME: `/identity` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:30:29.803112 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f3f89cb-7583-4ef5-99c4-af7a4b8c2269 admin admin] PATH_INFO: `/v3/roles` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:30:29.804895 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f3f89cb-7583-4ef5-99c4-af7a4b8c2269 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:30:29.806779 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f3f89cb-7583-4ef5-99c4-af7a4b8c2269 admin admin] RBAC: Authorization granted {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:30:29.831942 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: [pid: 80245|app: 0|req: 873/1757] 10.222.0.47 () {58 vars in 1121 bytes} [Thu Jul 21 08:30:29 2022] GET /identity/v3/roles => generated 1420 bytes in 74 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 08:30:29.835982 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72aa7027-d76d-4b09-a2cd-20a6c7fa2ba0 admin admin] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:30:29.847415 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72aa7027-d76d-4b09-a2cd-20a6c7fa2ba0 admin admin] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:30:29.865259 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-913282ee-a387-4f42-960a-87517023ea63 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '42d15fc0515945acacbbffcf2356c7a5', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:30:29.867362 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d66ae181-a6d3-48d9-a093-af1c05aab38c None None] REQUEST_METHOD: `POST` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:30:29.867960 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d66ae181-a6d3-48d9-a093-af1c05aab38c None None] SCRIPT_NAME: `/identity` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:30:29.868465 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d66ae181-a6d3-48d9-a093-af1c05aab38c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:30:29.871933 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-913282ee-a387-4f42-960a-87517023ea63 admin admin] REQUEST_METHOD: `GET` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:30:29.873035 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-913282ee-a387-4f42-960a-87517023ea63 admin admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:30:29.874044 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-913282ee-a387-4f42-960a-87517023ea63 admin admin] PATH_INFO: `/v3/roles` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:30:29.875818 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-913282ee-a387-4f42-960a-87517023ea63 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:30:29.884583 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-913282ee-a387-4f42-960a-87517023ea63 admin admin] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:30:29.916453 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 885/1758] 10.222.0.47 () {58 vars in 1121 bytes} [Thu Jul 21 08:30:29 2022] GET /identity/v3/roles => generated 1420 bytes in 81 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 08:30:29.917689 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-913282ee-a387-4f42-960a-87517023ea63 admin admin] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:30:29.917829 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.auth.core [None req-d66ae181-a6d3-48d9-a093-af1c05aab38c None None] MFA Rules not processed for user `3528104f3e914d62a10e80e13d3c97de`. Rule list: `[]` (Enabled: `True`). {{(pid=80245) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 08:30:29.919264 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.fernet_utils [None req-d66ae181-a6d3-48d9-a093-af1c05aab38c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80245) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 08:30:29.957770 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-913282ee-a387-4f42-960a-87517023ea63 admin admin] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:30:29.972594 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a91f6d7-532f-47b5-896d-cc50327bb28a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '42d15fc0515945acacbbffcf2356c7a5', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:30:29.977515 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a91f6d7-532f-47b5-896d-cc50327bb28a admin admin] REQUEST_METHOD: `PUT` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:30:29.978055 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a91f6d7-532f-47b5-896d-cc50327bb28a admin admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:30:29.978510 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a91f6d7-532f-47b5-896d-cc50327bb28a admin admin] PATH_INFO: `/v3/projects/ea6fdf0407e4492a9080992904579562/users/4426ac2210594b1b910cd76aa2183df2/roles/3a074738221e4601b5205715ba87d240` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:30:30.004927 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a91f6d7-532f-47b5-896d-cc50327bb28a admin admin] RBAC: Authorizing `identity:create_grant(project_id=ea6fdf0407e4492a9080992904579562, user_id=4426ac2210594b1b910cd76aa2183df2, role_id=3a074738221e4601b5205715ba87d240)` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:30:30.015032 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a91f6d7-532f-47b5-896d-cc50327bb28a admin admin] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:30:30.051728 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 886/1759] 10.222.0.47 () {60 vars in 1482 bytes} [Thu Jul 21 08:30:29 2022] PUT /identity/v3/projects/ea6fdf0407e4492a9080992904579562/users/4426ac2210594b1b910cd76aa2183df2/roles/3a074738221e4601b5205715ba87d240 => generated 0 bytes in 135 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 08:30:30.068559 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1e8e4f8-fe04-4736-a6a4-4283484be0ce None None] REQUEST_METHOD: `POST` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:30:30.069031 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1e8e4f8-fe04-4736-a6a4-4283484be0ce None None] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:30:30.069633 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1e8e4f8-fe04-4736-a6a4-4283484be0ce None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:30:30.102547 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: [pid: 80245|app: 0|req: 874/1760] 10.222.0.47 () {58 vars in 957 bytes} [Thu Jul 21 08:30:29 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 236 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 08:30:30.133655 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.auth.core [None req-e1e8e4f8-fe04-4736-a6a4-4283484be0ce None None] MFA Rules not processed for user `4426ac2210594b1b910cd76aa2183df2`. Rule list: `[]` (Enabled: `True`). {{(pid=80244) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 08:30:30.134915 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.fernet_utils [None req-e1e8e4f8-fe04-4736-a6a4-4283484be0ce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80244) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 08:30:30.392545 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 887/1761] 10.222.0.47 () {58 vars in 957 bytes} [Thu Jul 21 08:30:30 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 325 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 08:30:30.399737 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d66ae181-a6d3-48d9-a093-af1c05aab38c None None] Authenticating user token {{(pid=80245) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:30:30.420015 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d66ae181-a6d3-48d9-a093-af1c05aab38c None None] Validating token access rules against request {{(pid=80245) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:30:30.462372 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG dbcounter [-] [80245] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80245) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:30:30.645588 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cf885fd-afd1-4ad5-913b-ccc47ca524a8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '42d15fc0515945acacbbffcf2356c7a5', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80245) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:30:30.647710 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cf885fd-afd1-4ad5-913b-ccc47ca524a8 admin admin] REQUEST_METHOD: `POST` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:30:30.649016 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cf885fd-afd1-4ad5-913b-ccc47ca524a8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:30:30.650026 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cf885fd-afd1-4ad5-913b-ccc47ca524a8 admin admin] PATH_INFO: `/v3/projects` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:30:30.654048 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cf885fd-afd1-4ad5-913b-ccc47ca524a8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:30:30.660839 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cf885fd-afd1-4ad5-913b-ccc47ca524a8 admin admin] RBAC: Authorization granted {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:30:30.775606 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: [pid: 80245|app: 0|req: 875/1762] 10.222.0.47 () {60 vars in 1152 bytes} [Thu Jul 21 08:30:30 2022] POST /identity/v3/projects => generated 383 bytes in 375 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 08:30:30.776858 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1e8e4f8-fe04-4736-a6a4-4283484be0ce None None] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:30:30.797447 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1e8e4f8-fe04-4736-a6a4-4283484be0ce None None] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:30:30.850859 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cf885fd-afd1-4ad5-913b-ccc47ca524a8 admin admin] Authenticating user token {{(pid=80245) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:30:30.853134 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.fernet_utils [None req-1cf885fd-afd1-4ad5-913b-ccc47ca524a8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80245) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 08:30:31.060416 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6020f8fa-5b72-4493-9b4e-bcfe10855b45 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '42d15fc0515945acacbbffcf2356c7a5', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:30:31.061575 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6020f8fa-5b72-4493-9b4e-bcfe10855b45 admin admin] REQUEST_METHOD: `POST` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:30:31.061861 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6020f8fa-5b72-4493-9b4e-bcfe10855b45 admin admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:30:31.062135 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6020f8fa-5b72-4493-9b4e-bcfe10855b45 admin admin] PATH_INFO: `/v3/users` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:30:31.067736 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6020f8fa-5b72-4493-9b4e-bcfe10855b45 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:30:31.077331 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6020f8fa-5b72-4493-9b4e-bcfe10855b45 admin admin] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:30:31.095592 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cf885fd-afd1-4ad5-913b-ccc47ca524a8 admin admin] Validating token access rules against request {{(pid=80245) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:30:31.111940 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1137cca7-1720-483b-a6fc-730f2bf96565 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '42d15fc0515945acacbbffcf2356c7a5', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80245) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:30:31.112842 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1137cca7-1720-483b-a6fc-730f2bf96565 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:30:31.113119 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1137cca7-1720-483b-a6fc-730f2bf96565 admin admin] SCRIPT_NAME: `/identity` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:30:31.113422 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1137cca7-1720-483b-a6fc-730f2bf96565 admin admin] PATH_INFO: `/v3/projects/dd675e3ea3e34a219be692f3a9c6e2ee` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:30:31.114991 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1137cca7-1720-483b-a6fc-730f2bf96565 admin admin] RBAC: Authorizing `identity:delete_project(project_id=dd675e3ea3e34a219be692f3a9c6e2ee)` {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:30:31.117532 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1137cca7-1720-483b-a6fc-730f2bf96565 admin admin] RBAC: Authorization granted {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:30:31.147398 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 888/1763] 10.222.0.47 () {60 vars in 1143 bytes} [Thu Jul 21 08:30:30 2022] POST /identity/v3/users => generated 298 bytes in 371 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 08:30:31.148885 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6020f8fa-5b72-4493-9b4e-bcfe10855b45 admin admin] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:30:31.169171 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6020f8fa-5b72-4493-9b4e-bcfe10855b45 admin admin] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:30:31.177124 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11e54b10-7602-45ca-a9b1-9075036b102d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '42d15fc0515945acacbbffcf2356c7a5', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:30:31.180631 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11e54b10-7602-45ca-a9b1-9075036b102d admin admin] REQUEST_METHOD: `GET` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:30:31.180769 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11e54b10-7602-45ca-a9b1-9075036b102d admin admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:30:31.181064 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11e54b10-7602-45ca-a9b1-9075036b102d admin admin] PATH_INFO: `/v3/roles` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:30:31.181828 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11e54b10-7602-45ca-a9b1-9075036b102d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:30:31.188575 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11e54b10-7602-45ca-a9b1-9075036b102d admin admin] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:30:31.222672 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 889/1764] 10.222.0.47 () {58 vars in 1121 bytes} [Thu Jul 21 08:30:31 2022] GET /identity/v3/roles => generated 1420 bytes in 75 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 08:30:31.223531 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11e54b10-7602-45ca-a9b1-9075036b102d admin admin] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:30:31.250765 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11e54b10-7602-45ca-a9b1-9075036b102d admin admin] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:30:31.263699 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-879c37b6-e4f8-464b-a321-f034abb25c78 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '42d15fc0515945acacbbffcf2356c7a5', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:30:31.267382 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-879c37b6-e4f8-464b-a321-f034abb25c78 admin admin] REQUEST_METHOD: `PUT` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:30:31.267889 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-879c37b6-e4f8-464b-a321-f034abb25c78 admin admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:30:31.270107 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-879c37b6-e4f8-464b-a321-f034abb25c78 admin admin] PATH_INFO: `/v3/projects/b0c086d149ca495783890b4200ccfdc2/users/f554b9336b00415c82aa32414c516d78/roles/e29870f6e9e2411a9db0d5bf45f32ca5` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:30:31.329751 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-879c37b6-e4f8-464b-a321-f034abb25c78 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b0c086d149ca495783890b4200ccfdc2, user_id=f554b9336b00415c82aa32414c516d78, role_id=e29870f6e9e2411a9db0d5bf45f32ca5)` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:30:31.335293 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-879c37b6-e4f8-464b-a321-f034abb25c78 admin admin] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:30:31.347372 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.notifications [None req-1137cca7-1720-483b-a6fc-730f2bf96565 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'dd675e3ea3e34a219be692f3a9c6e2ee', 'request_id': 'req-1137cca7-1720-483b-a6fc-730f2bf96565'} {{(pid=80245) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 08:30:31.365324 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG dbcounter [-] [80244] Writing DB stats keystone:INSERT=7,keystone:SELECT=93 {{(pid=80244) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:30:31.379580 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 890/1765] 10.222.0.47 () {60 vars in 1482 bytes} [Thu Jul 21 08:30:31 2022] PUT /identity/v3/projects/b0c086d149ca495783890b4200ccfdc2/users/f554b9336b00415c82aa32414c516d78/roles/e29870f6e9e2411a9db0d5bf45f32ca5 => generated 0 bytes in 156 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 08:30:31.388423 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-879c37b6-e4f8-464b-a321-f034abb25c78 admin admin] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:30:31.388766 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.notifications [None req-1137cca7-1720-483b-a6fc-730f2bf96565 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'dd675e3ea3e34a219be692f3a9c6e2ee', 'request_id': 'req-1137cca7-1720-483b-a6fc-730f2bf96565'} {{(pid=80245) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 08:30:31.426736 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.notifications [None req-1137cca7-1720-483b-a6fc-730f2bf96565 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'dd675e3ea3e34a219be692f3a9c6e2ee', 'request_id': 'req-1137cca7-1720-483b-a6fc-730f2bf96565'} {{(pid=80245) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 08:30:31.480751 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.notifications [None req-1137cca7-1720-483b-a6fc-730f2bf96565 admin admin] The token cache is being invalidate because project dd675e3ea3e34a219be692f3a9c6e2ee was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80245) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 21 08:30:31.487801 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.notifications [None req-1137cca7-1720-483b-a6fc-730f2bf96565 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80245) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 08:30:31.490739 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.notifications [None req-1137cca7-1720-483b-a6fc-730f2bf96565 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80245) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 08:30:31.505211 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: [pid: 80245|app: 0|req: 876/1766] 10.222.0.47 () {58 vars in 1232 bytes} [Thu Jul 21 08:30:30 2022] DELETE /identity/v3/projects/dd675e3ea3e34a219be692f3a9c6e2ee => generated 0 bytes in 650 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 08:30:31.506313 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1137cca7-1720-483b-a6fc-730f2bf96565 admin admin] Authenticating user token {{(pid=80245) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:30:31.516327 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.fernet_utils [None req-1137cca7-1720-483b-a6fc-730f2bf96565 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80245) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 08:30:31.619677 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG dbcounter [-] [80245] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=80245) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:30:31.698838 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-879c37b6-e4f8-464b-a321-f034abb25c78 admin admin] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:30:31.715987 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1137cca7-1720-483b-a6fc-730f2bf96565 admin admin] Validating token access rules against request {{(pid=80245) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:30:31.719358 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40331418-8b8c-4249-bf68-b4aa4702e2f9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '42d15fc0515945acacbbffcf2356c7a5', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:30:31.720040 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40331418-8b8c-4249-bf68-b4aa4702e2f9 admin admin] REQUEST_METHOD: `GET` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:30:31.720475 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40331418-8b8c-4249-bf68-b4aa4702e2f9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:30:31.720894 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40331418-8b8c-4249-bf68-b4aa4702e2f9 admin admin] PATH_INFO: `/v3/roles` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:30:31.721804 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40331418-8b8c-4249-bf68-b4aa4702e2f9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:30:31.726278 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40331418-8b8c-4249-bf68-b4aa4702e2f9 admin admin] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:30:31.755374 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfcaa629-385d-4fe0-bdf5-b83ac401c217 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '42d15fc0515945acacbbffcf2356c7a5', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80245) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:30:31.756747 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfcaa629-385d-4fe0-bdf5-b83ac401c217 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:30:31.756747 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfcaa629-385d-4fe0-bdf5-b83ac401c217 admin admin] SCRIPT_NAME: `/identity` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:30:31.756936 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfcaa629-385d-4fe0-bdf5-b83ac401c217 admin admin] PATH_INFO: `/v3/users/a0b0f566f8fd43fa91147e5c9eb5eb0d` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:30:31.758095 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfcaa629-385d-4fe0-bdf5-b83ac401c217 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a0b0f566f8fd43fa91147e5c9eb5eb0d)` {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:30:31.760560 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfcaa629-385d-4fe0-bdf5-b83ac401c217 admin admin] RBAC: Authorization granted {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:30:31.772179 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 891/1767] 10.222.0.47 () {58 vars in 1121 bytes} [Thu Jul 21 08:30:31 2022] GET /identity/v3/roles => generated 1420 bytes in 384 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 08:30:31.778733 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40331418-8b8c-4249-bf68-b4aa4702e2f9 admin admin] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:30:31.786298 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.fernet_utils [None req-40331418-8b8c-4249-bf68-b4aa4702e2f9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80244) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 08:30:31.819795 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40331418-8b8c-4249-bf68-b4aa4702e2f9 admin admin] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:30:31.832536 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d441590e-3075-4335-8ac5-25e0bdb60440 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '42d15fc0515945acacbbffcf2356c7a5', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:30:31.833790 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d441590e-3075-4335-8ac5-25e0bdb60440 admin admin] REQUEST_METHOD: `PUT` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:30:31.834281 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d441590e-3075-4335-8ac5-25e0bdb60440 admin admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:30:31.840851 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d441590e-3075-4335-8ac5-25e0bdb60440 admin admin] PATH_INFO: `/v3/projects/b0c086d149ca495783890b4200ccfdc2/users/f554b9336b00415c82aa32414c516d78/roles/e29870f6e9e2411a9db0d5bf45f32ca5` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:30:31.850915 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d441590e-3075-4335-8ac5-25e0bdb60440 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b0c086d149ca495783890b4200ccfdc2, user_id=f554b9336b00415c82aa32414c516d78, role_id=e29870f6e9e2411a9db0d5bf45f32ca5)` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:30:31.852764 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.notifications [None req-cfcaa629-385d-4fe0-bdf5-b83ac401c217 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a0b0f566f8fd43fa91147e5c9eb5eb0d'} {{(pid=80245) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 08:30:31.861637 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d441590e-3075-4335-8ac5-25e0bdb60440 admin admin] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:30:31.875585 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.notifications [None req-cfcaa629-385d-4fe0-bdf5-b83ac401c217 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a0b0f566f8fd43fa91147e5c9eb5eb0d'} {{(pid=80245) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 08:30:31.910586 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 892/1768] 10.222.0.47 () {60 vars in 1482 bytes} [Thu Jul 21 08:30:31 2022] PUT /identity/v3/projects/b0c086d149ca495783890b4200ccfdc2/users/f554b9336b00415c82aa32414c516d78/roles/e29870f6e9e2411a9db0d5bf45f32ca5 => generated 0 bytes in 132 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 08:30:31.913579 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.notifications [None req-cfcaa629-385d-4fe0-bdf5-b83ac401c217 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a0b0f566f8fd43fa91147e5c9eb5eb0d'} {{(pid=80245) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 08:30:31.922643 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cae1c3ea-ca17-4bf1-8fdd-59fe4f696ee2 None None] REQUEST_METHOD: `POST` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:30:31.923168 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cae1c3ea-ca17-4bf1-8fdd-59fe4f696ee2 None None] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:30:31.923659 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cae1c3ea-ca17-4bf1-8fdd-59fe4f696ee2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:30:31.930440 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.notifications [None req-cfcaa629-385d-4fe0-bdf5-b83ac401c217 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a0b0f566f8fd43fa91147e5c9eb5eb0d'} {{(pid=80245) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 08:30:31.933863 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.notifications [None req-cfcaa629-385d-4fe0-bdf5-b83ac401c217 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a0b0f566f8fd43fa91147e5c9eb5eb0d'} {{(pid=80245) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 08:30:31.971436 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.auth.core [None req-cae1c3ea-ca17-4bf1-8fdd-59fe4f696ee2 None None] MFA Rules not processed for user `f554b9336b00415c82aa32414c516d78`. Rule list: `[]` (Enabled: `True`). {{(pid=80244) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 08:30:31.973907 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.fernet_utils [None req-cae1c3ea-ca17-4bf1-8fdd-59fe4f696ee2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80244) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 08:30:31.981862 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: [pid: 80245|app: 0|req: 877/1769] 10.222.0.47 () {58 vars in 1223 bytes} [Thu Jul 21 08:30:31 2022] DELETE /identity/v3/users/a0b0f566f8fd43fa91147e5c9eb5eb0d => generated 0 bytes in 476 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 08:30:32.091408 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 893/1770] 10.222.0.47 () {58 vars in 957 bytes} [Thu Jul 21 08:30:31 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 08:30:32.260459 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfcaa629-385d-4fe0-bdf5-b83ac401c217 admin admin] Authenticating user token {{(pid=80245) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:30:32.262797 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.fernet_utils [None req-cfcaa629-385d-4fe0-bdf5-b83ac401c217 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80245) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 08:30:32.578409 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfcaa629-385d-4fe0-bdf5-b83ac401c217 admin admin] Validating token access rules against request {{(pid=80245) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:30:32.586347 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4094584-7c30-45e1-b2ff-4fa1daaf7aad service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4f40410b7674687ab5362ac982248ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4faef13aa50e4e36b6c265fb1398fa14', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80245) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:30:32.588115 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4094584-7c30-45e1-b2ff-4fa1daaf7aad service cinder] REQUEST_METHOD: `GET` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:30:32.589000 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4094584-7c30-45e1-b2ff-4fa1daaf7aad service cinder] SCRIPT_NAME: `/identity` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:30:32.589432 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4094584-7c30-45e1-b2ff-4fa1daaf7aad service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:30:32.596106 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.fernet_utils [None req-e4094584-7c30-45e1-b2ff-4fa1daaf7aad service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80245) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 08:30:32.811624 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4094584-7c30-45e1-b2ff-4fa1daaf7aad service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:30:32.813259 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4094584-7c30-45e1-b2ff-4fa1daaf7aad service cinder] RBAC: Authorization granted {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:30:32.856678 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: [pid: 80245|app: 0|req: 878/1771] 10.222.0.47 () {60 vars in 1365 bytes} [Thu Jul 21 08:30:32 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 597 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 08:30:33.239468 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cae1c3ea-ca17-4bf1-8fdd-59fe4f696ee2 None None] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:30:33.241509 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.fernet_utils [None req-cae1c3ea-ca17-4bf1-8fdd-59fe4f696ee2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80244) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 08:30:33.372944 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44e06b11-d1e5-4538-b126-1c6c062e5965 None None] REQUEST_METHOD: `POST` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:30:33.373755 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44e06b11-d1e5-4538-b126-1c6c062e5965 None None] SCRIPT_NAME: `/identity` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:30:33.374074 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44e06b11-d1e5-4538-b126-1c6c062e5965 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:30:33.380416 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG dbcounter [-] [80245] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80245) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:30:33.544880 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cae1c3ea-ca17-4bf1-8fdd-59fe4f696ee2 None None] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:30:33.548971 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.auth.core [None req-44e06b11-d1e5-4538-b126-1c6c062e5965 None None] MFA Rules not processed for user `2ef7f1e91e554a1f88eb6a63e60a475f`. Rule list: `[]` (Enabled: `True`). {{(pid=80245) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 08:30:33.552754 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.fernet_utils [None req-44e06b11-d1e5-4538-b126-1c6c062e5965 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80245) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 08:30:33.555448 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e04a11d0-c83b-436e-b36d-37c5b955be12 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '42d15fc0515945acacbbffcf2356c7a5', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:30:33.556643 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e04a11d0-c83b-436e-b36d-37c5b955be12 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:30:33.557071 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e04a11d0-c83b-436e-b36d-37c5b955be12 admin admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:30:33.558087 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e04a11d0-c83b-436e-b36d-37c5b955be12 admin admin] PATH_INFO: `/v3/projects/d2b04a0c9dcd4f08a6db8d8847819ac0` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:30:33.562539 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e04a11d0-c83b-436e-b36d-37c5b955be12 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d2b04a0c9dcd4f08a6db8d8847819ac0)` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:30:33.565825 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e04a11d0-c83b-436e-b36d-37c5b955be12 admin admin] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:30:33.614009 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG dbcounter [-] [80244] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80244) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:30:33.832873 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: [pid: 80245|app: 0|req: 879/1772] 10.222.0.47 () {58 vars in 1009 bytes} [Thu Jul 21 08:30:33 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 461 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 08:30:33.852119 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.notifications [None req-e04a11d0-c83b-436e-b36d-37c5b955be12 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd2b04a0c9dcd4f08a6db8d8847819ac0', 'request_id': 'req-e04a11d0-c83b-436e-b36d-37c5b955be12'} {{(pid=80244) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 08:30:33.883633 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.notifications [None req-e04a11d0-c83b-436e-b36d-37c5b955be12 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd2b04a0c9dcd4f08a6db8d8847819ac0', 'request_id': 'req-e04a11d0-c83b-436e-b36d-37c5b955be12'} {{(pid=80244) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 08:30:33.917817 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.notifications [None req-e04a11d0-c83b-436e-b36d-37c5b955be12 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd2b04a0c9dcd4f08a6db8d8847819ac0', 'request_id': 'req-e04a11d0-c83b-436e-b36d-37c5b955be12'} {{(pid=80244) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 08:30:33.992265 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.notifications [None req-e04a11d0-c83b-436e-b36d-37c5b955be12 admin admin] The token cache is being invalidate because project d2b04a0c9dcd4f08a6db8d8847819ac0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80244) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 21 08:30:33.992622 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.notifications [None req-e04a11d0-c83b-436e-b36d-37c5b955be12 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80244) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 08:30:34.001514 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.notifications [None req-e04a11d0-c83b-436e-b36d-37c5b955be12 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80244) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 08:30:34.005373 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 894/1773] 10.222.0.47 () {58 vars in 1232 bytes} [Thu Jul 21 08:30:33 2022] DELETE /identity/v3/projects/d2b04a0c9dcd4f08a6db8d8847819ac0 => generated 0 bytes in 766 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 08:30:34.010959 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f20efeb-b58e-4700-a8e0-e4c2d5ad4470 None None] REQUEST_METHOD: `POST` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:30:34.011575 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f20efeb-b58e-4700-a8e0-e4c2d5ad4470 None None] SCRIPT_NAME: `/identity` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:30:34.011653 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f20efeb-b58e-4700-a8e0-e4c2d5ad4470 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:30:34.050520 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.auth.core [None req-7f20efeb-b58e-4700-a8e0-e4c2d5ad4470 None None] MFA Rules not processed for user `2dce40b8d74c41ec94cc36e6ff05d412`. Rule list: `[]` (Enabled: `True`). {{(pid=80245) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 08:30:34.052554 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.fernet_utils [None req-7f20efeb-b58e-4700-a8e0-e4c2d5ad4470 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80245) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 08:30:34.169310 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: [pid: 80245|app: 0|req: 880/1774] 10.222.0.47 () {58 vars in 957 bytes} [Thu Jul 21 08:30:34 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 159 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 08:30:34.177364 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb97dd49-6372-41a6-a3e5-c3acfb573d77 None None] REQUEST_METHOD: `POST` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:30:34.177735 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb97dd49-6372-41a6-a3e5-c3acfb573d77 None None] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:30:34.178118 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb97dd49-6372-41a6-a3e5-c3acfb573d77 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:30:34.200405 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f20efeb-b58e-4700-a8e0-e4c2d5ad4470 None None] Authenticating user token {{(pid=80245) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:30:34.202115 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.fernet_utils [None req-7f20efeb-b58e-4700-a8e0-e4c2d5ad4470 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80245) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 08:30:34.288417 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.auth.core [None req-fb97dd49-6372-41a6-a3e5-c3acfb573d77 None None] MFA Rules not processed for user `2dce40b8d74c41ec94cc36e6ff05d412`. Rule list: `[]` (Enabled: `True`). {{(pid=80244) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 08:30:34.294735 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.fernet_utils [None req-fb97dd49-6372-41a6-a3e5-c3acfb573d77 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80244) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 08:30:34.354586 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 895/1775] 10.222.0.47 () {58 vars in 957 bytes} [Thu Jul 21 08:30:34 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 178 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 08:30:34.356872 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb97dd49-6372-41a6-a3e5-c3acfb573d77 None None] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:30:34.401958 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb97dd49-6372-41a6-a3e5-c3acfb573d77 None None] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:30:34.420925 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bdc544f-314b-43b0-a164-9d3ce10d9849 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '42d15fc0515945acacbbffcf2356c7a5', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:30:34.422026 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bdc544f-314b-43b0-a164-9d3ce10d9849 admin admin] REQUEST_METHOD: `GET` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:30:34.422532 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bdc544f-314b-43b0-a164-9d3ce10d9849 admin admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:30:34.423099 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bdc544f-314b-43b0-a164-9d3ce10d9849 admin admin] PATH_INFO: `/v3/domains` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:30:34.426807 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bdc544f-314b-43b0-a164-9d3ce10d9849 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80244) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 08:30:34.427317 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bdc544f-314b-43b0-a164-9d3ce10d9849 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:30:34.429519 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bdc544f-314b-43b0-a164-9d3ce10d9849 admin admin] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:30:34.477145 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 896/1776] 10.222.0.47 () {58 vars in 1152 bytes} [Thu Jul 21 08:30:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 122 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 08:30:34.482073 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bdc544f-314b-43b0-a164-9d3ce10d9849 admin admin] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:30:34.511182 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bdc544f-314b-43b0-a164-9d3ce10d9849 admin admin] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:30:34.538242 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab29dc50-3875-44f1-b841-2ebd553d08ec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '42d15fc0515945acacbbffcf2356c7a5', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:30:34.539375 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab29dc50-3875-44f1-b841-2ebd553d08ec admin admin] REQUEST_METHOD: `POST` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:30:34.540208 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab29dc50-3875-44f1-b841-2ebd553d08ec admin admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:30:34.540648 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab29dc50-3875-44f1-b841-2ebd553d08ec admin admin] PATH_INFO: `/v3/projects` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:30:34.541531 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab29dc50-3875-44f1-b841-2ebd553d08ec admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:30:34.558558 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab29dc50-3875-44f1-b841-2ebd553d08ec admin admin] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:30:34.633467 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG dbcounter [-] [80245] Writing DB stats keystone:SELECT=100 {{(pid=80245) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:30:34.673754 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f20efeb-b58e-4700-a8e0-e4c2d5ad4470 None None] Validating token access rules against request {{(pid=80245) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:30:34.679418 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 897/1777] 10.222.0.47 () {60 vars in 1152 bytes} [Thu Jul 21 08:30:34 2022] POST /identity/v3/projects => generated 375 bytes in 197 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 21 08:30:34.687885 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab29dc50-3875-44f1-b841-2ebd553d08ec admin admin] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:30:34.692548 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.fernet_utils [None req-ab29dc50-3875-44f1-b841-2ebd553d08ec admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80244) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 08:30:34.965155 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab29dc50-3875-44f1-b841-2ebd553d08ec admin admin] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:30:34.987823 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb340ba9-6417-44d8-ab99-d644e8e3c5e9 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '302351411fc84248a6b166b25d21609f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4faef13aa50e4e36b6c265fb1398fa14', 'project_domain_id': 'default', 'roles': ['service', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80245) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:30:34.991562 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb340ba9-6417-44d8-ab99-d644e8e3c5e9 service swift] REQUEST_METHOD: `GET` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:30:34.991562 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb340ba9-6417-44d8-ab99-d644e8e3c5e9 service swift] SCRIPT_NAME: `/identity` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:30:34.991562 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb340ba9-6417-44d8-ab99-d644e8e3c5e9 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:30:34.994563 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb9cbd79-234a-4abe-bc9a-06909601a1d0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '42d15fc0515945acacbbffcf2356c7a5', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:30:34.995510 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb9cbd79-234a-4abe-bc9a-06909601a1d0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:30:34.995830 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb9cbd79-234a-4abe-bc9a-06909601a1d0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:30:34.996068 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb9cbd79-234a-4abe-bc9a-06909601a1d0 admin admin] PATH_INFO: `/v3/users/2d412a765e97490f9c83b00b1363209b` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:30:34.997455 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.fernet_utils [None req-cb340ba9-6417-44d8-ab99-d644e8e3c5e9 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80245) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 08:30:35.002513 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb9cbd79-234a-4abe-bc9a-06909601a1d0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=2d412a765e97490f9c83b00b1363209b)` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:30:35.004591 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb9cbd79-234a-4abe-bc9a-06909601a1d0 admin admin] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:30:35.136103 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.notifications [None req-eb9cbd79-234a-4abe-bc9a-06909601a1d0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2d412a765e97490f9c83b00b1363209b'} {{(pid=80244) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 08:30:35.150377 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.notifications [None req-eb9cbd79-234a-4abe-bc9a-06909601a1d0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2d412a765e97490f9c83b00b1363209b'} {{(pid=80244) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 08:30:35.160737 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG dbcounter [-] [80244] Writing DB stats keystone:SELECT=82,keystone:DELETE=15,keystone:INSERT=3 {{(pid=80244) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:30:35.188163 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.notifications [None req-eb9cbd79-234a-4abe-bc9a-06909601a1d0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2d412a765e97490f9c83b00b1363209b'} {{(pid=80244) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 08:30:35.200878 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.notifications [None req-eb9cbd79-234a-4abe-bc9a-06909601a1d0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2d412a765e97490f9c83b00b1363209b'} {{(pid=80244) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 08:30:35.212137 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.notifications [None req-eb9cbd79-234a-4abe-bc9a-06909601a1d0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2d412a765e97490f9c83b00b1363209b'} {{(pid=80244) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 08:30:35.221661 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 898/1778] 10.222.0.47 () {58 vars in 1223 bytes} [Thu Jul 21 08:30:34 2022] DELETE /identity/v3/users/2d412a765e97490f9c83b00b1363209b => generated 0 bytes in 535 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 08:30:35.225086 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb9cbd79-234a-4abe-bc9a-06909601a1d0 admin admin] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:30:35.226057 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.fernet_utils [None req-eb9cbd79-234a-4abe-bc9a-06909601a1d0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80244) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 08:30:35.342895 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb340ba9-6417-44d8-ab99-d644e8e3c5e9 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:30:35.346079 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb340ba9-6417-44d8-ab99-d644e8e3c5e9 service swift] RBAC: Authorization granted {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:30:35.369521 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: [pid: 80245|app: 0|req: 881/1779] 10.222.0.47 () {60 vars in 1384 bytes} [Thu Jul 21 08:30:34 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 1170 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Jul 21 08:30:35.376107 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d078aebc-44da-436f-b4ab-6bc01a53c09e None None] REQUEST_METHOD: `POST` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:30:35.377944 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d078aebc-44da-436f-b4ab-6bc01a53c09e None None] SCRIPT_NAME: `/identity` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:30:35.377944 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d078aebc-44da-436f-b4ab-6bc01a53c09e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:30:35.437814 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.auth.core [None req-d078aebc-44da-436f-b4ab-6bc01a53c09e None None] MFA Rules not processed for user `2dce40b8d74c41ec94cc36e6ff05d412`. Rule list: `[]` (Enabled: `True`). {{(pid=80245) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 08:30:35.445262 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.fernet_utils [None req-d078aebc-44da-436f-b4ab-6bc01a53c09e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80245) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 08:30:35.452063 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb9cbd79-234a-4abe-bc9a-06909601a1d0 admin admin] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:30:35.477594 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b96ece51-a1ec-4847-8b1f-253c263bab6d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '42d15fc0515945acacbbffcf2356c7a5', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:30:35.484023 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b96ece51-a1ec-4847-8b1f-253c263bab6d admin admin] REQUEST_METHOD: `POST` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:30:35.484926 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b96ece51-a1ec-4847-8b1f-253c263bab6d admin admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:30:35.486973 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b96ece51-a1ec-4847-8b1f-253c263bab6d admin admin] PATH_INFO: `/v3/users` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:30:35.492303 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b96ece51-a1ec-4847-8b1f-253c263bab6d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:30:35.509640 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b96ece51-a1ec-4847-8b1f-253c263bab6d admin admin] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:30:35.523996 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: [pid: 80245|app: 0|req: 882/1780] 10.222.0.47 () {58 vars in 957 bytes} [Thu Jul 21 08:30:35 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 154 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 08:30:35.642971 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 899/1781] 10.222.0.47 () {60 vars in 1143 bytes} [Thu Jul 21 08:30:35 2022] POST /identity/v3/users => generated 288 bytes in 421 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 08:30:35.651309 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d078aebc-44da-436f-b4ab-6bc01a53c09e None None] Authenticating user token {{(pid=80245) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:30:35.679677 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d078aebc-44da-436f-b4ab-6bc01a53c09e None None] Validating token access rules against request {{(pid=80245) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:30:35.736319 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27dd7599-e56f-4c07-80b0-a686c00749ca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '42d15fc0515945acacbbffcf2356c7a5', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80245) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:30:35.739109 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27dd7599-e56f-4c07-80b0-a686c00749ca admin admin] REQUEST_METHOD: `GET` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:30:35.744076 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27dd7599-e56f-4c07-80b0-a686c00749ca admin admin] SCRIPT_NAME: `/identity` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:30:35.746342 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27dd7599-e56f-4c07-80b0-a686c00749ca admin admin] PATH_INFO: `/v3/roles` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:30:35.754553 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27dd7599-e56f-4c07-80b0-a686c00749ca admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:30:35.773037 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27dd7599-e56f-4c07-80b0-a686c00749ca admin admin] RBAC: Authorization granted {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:30:35.831518 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b96ece51-a1ec-4847-8b1f-253c263bab6d admin admin] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:30:35.834932 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.fernet_utils [None req-b96ece51-a1ec-4847-8b1f-253c263bab6d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80244) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 08:30:35.860509 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: [pid: 80245|app: 0|req: 883/1782] 10.222.0.47 () {58 vars in 1121 bytes} [Thu Jul 21 08:30:35 2022] GET /identity/v3/roles => generated 1420 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 08:30:35.861747 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27dd7599-e56f-4c07-80b0-a686c00749ca admin admin] Authenticating user token {{(pid=80245) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:30:35.869372 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b96ece51-a1ec-4847-8b1f-253c263bab6d admin admin] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:30:35.903278 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85f7d705-9d77-4a69-81ca-3a60ca18641b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '42d15fc0515945acacbbffcf2356c7a5', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:30:35.904216 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85f7d705-9d77-4a69-81ca-3a60ca18641b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:30:35.905100 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85f7d705-9d77-4a69-81ca-3a60ca18641b admin admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:30:35.905376 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85f7d705-9d77-4a69-81ca-3a60ca18641b admin admin] PATH_INFO: `/v3/users/8b727a8c76f141b79e9d996019da54a5` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:30:35.911300 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27dd7599-e56f-4c07-80b0-a686c00749ca admin admin] Validating token access rules against request {{(pid=80245) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:30:35.915712 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85f7d705-9d77-4a69-81ca-3a60ca18641b admin admin] RBAC: Authorizing `identity:delete_user(user_id=8b727a8c76f141b79e9d996019da54a5)` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:30:35.917752 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85f7d705-9d77-4a69-81ca-3a60ca18641b admin admin] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:30:35.932929 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89334278-f962-4810-ab3e-44a203749deb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '42d15fc0515945acacbbffcf2356c7a5', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80245) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:30:35.933774 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89334278-f962-4810-ab3e-44a203749deb admin admin] REQUEST_METHOD: `GET` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:30:35.935491 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89334278-f962-4810-ab3e-44a203749deb admin admin] SCRIPT_NAME: `/identity` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:30:35.935491 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89334278-f962-4810-ab3e-44a203749deb admin admin] PATH_INFO: `/v3/roles` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:30:35.938166 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89334278-f962-4810-ab3e-44a203749deb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:30:35.947478 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89334278-f962-4810-ab3e-44a203749deb admin admin] RBAC: Authorization granted {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:30:35.994365 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: [pid: 80245|app: 0|req: 884/1783] 10.222.0.47 () {58 vars in 1121 bytes} [Thu Jul 21 08:30:35 2022] GET /identity/v3/roles => generated 1420 bytes in 133 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 08:30:35.996329 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89334278-f962-4810-ab3e-44a203749deb admin admin] Authenticating user token {{(pid=80245) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:30:36.010224 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.fernet_utils [None req-89334278-f962-4810-ab3e-44a203749deb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80245) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 08:30:36.095104 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.notifications [None req-85f7d705-9d77-4a69-81ca-3a60ca18641b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8b727a8c76f141b79e9d996019da54a5'} {{(pid=80244) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 08:30:36.115497 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.notifications [None req-85f7d705-9d77-4a69-81ca-3a60ca18641b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8b727a8c76f141b79e9d996019da54a5'} {{(pid=80244) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 08:30:36.163982 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.notifications [None req-85f7d705-9d77-4a69-81ca-3a60ca18641b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8b727a8c76f141b79e9d996019da54a5'} {{(pid=80244) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 08:30:36.167544 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.notifications [None req-85f7d705-9d77-4a69-81ca-3a60ca18641b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8b727a8c76f141b79e9d996019da54a5'} {{(pid=80244) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 08:30:36.203171 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.notifications [None req-85f7d705-9d77-4a69-81ca-3a60ca18641b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8b727a8c76f141b79e9d996019da54a5'} {{(pid=80244) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 08:30:36.233975 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 900/1784] 10.222.0.47 () {58 vars in 1223 bytes} [Thu Jul 21 08:30:35 2022] DELETE /identity/v3/users/8b727a8c76f141b79e9d996019da54a5 => generated 0 bytes in 400 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 08:30:36.233975 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85f7d705-9d77-4a69-81ca-3a60ca18641b admin admin] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:30:36.239970 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.fernet_utils [None req-85f7d705-9d77-4a69-81ca-3a60ca18641b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80244) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 08:30:36.356486 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89334278-f962-4810-ab3e-44a203749deb admin admin] Validating token access rules against request {{(pid=80245) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:30:36.429780 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG dbcounter [-] [80244] Writing DB stats keystone:DELETE=12,keystone:SELECT=84,keystone:INSERT=4 {{(pid=80244) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:30:36.466348 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG dbcounter [-] [80245] Writing DB stats keystone:SELECT=100 {{(pid=80245) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:30:36.512172 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85f7d705-9d77-4a69-81ca-3a60ca18641b admin admin] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:30:36.533578 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfa526be-53e5-42b8-8072-666407c40fae service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20ca3c3bd3774691b3374cfbfec6e494', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4faef13aa50e4e36b6c265fb1398fa14', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:30:36.538390 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfa526be-53e5-42b8-8072-666407c40fae service neutron] REQUEST_METHOD: `GET` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:30:36.539862 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfa526be-53e5-42b8-8072-666407c40fae service neutron] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:30:36.540797 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfa526be-53e5-42b8-8072-666407c40fae service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:30:36.554148 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.fernet_utils [None req-dfa526be-53e5-42b8-8072-666407c40fae service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80244) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 08:30:36.587044 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-167743e7-8a9f-4937-9e42-1549eda6237b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4f40410b7674687ab5362ac982248ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4faef13aa50e4e36b6c265fb1398fa14', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80245) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:30:36.589517 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-167743e7-8a9f-4937-9e42-1549eda6237b service cinder] REQUEST_METHOD: `GET` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:30:36.591214 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-167743e7-8a9f-4937-9e42-1549eda6237b service cinder] SCRIPT_NAME: `/identity` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:30:36.592985 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-167743e7-8a9f-4937-9e42-1549eda6237b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:30:36.600972 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.fernet_utils [None req-167743e7-8a9f-4937-9e42-1549eda6237b service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80245) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 08:30:36.733759 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfa526be-53e5-42b8-8072-666407c40fae service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:30:36.736850 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfa526be-53e5-42b8-8072-666407c40fae service neutron] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:30:36.752112 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 901/1785] 10.222.0.47 () {60 vars in 1365 bytes} [Thu Jul 21 08:30:36 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 520 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 08:30:36.753445 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfa526be-53e5-42b8-8072-666407c40fae service neutron] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:30:36.755242 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.fernet_utils [None req-dfa526be-53e5-42b8-8072-666407c40fae service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80244) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 08:30:36.774317 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfa526be-53e5-42b8-8072-666407c40fae service neutron] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:30:36.788065 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f0b8464-6e7e-408c-8bd2-6a30e1e263f7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '42d15fc0515945acacbbffcf2356c7a5', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:30:36.790811 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f0b8464-6e7e-408c-8bd2-6a30e1e263f7 admin admin] REQUEST_METHOD: `PUT` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:30:36.794108 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f0b8464-6e7e-408c-8bd2-6a30e1e263f7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:30:36.794108 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f0b8464-6e7e-408c-8bd2-6a30e1e263f7 admin admin] PATH_INFO: `/v3/projects/3aabc4c5aa8b4e5698680ec589143be6/users/e35645c0565c4de29bac3e0ad8f1f245/roles/3a074738221e4601b5205715ba87d240` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:30:36.810705 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-167743e7-8a9f-4937-9e42-1549eda6237b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:30:36.812253 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-167743e7-8a9f-4937-9e42-1549eda6237b service cinder] RBAC: Authorization granted {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:30:36.824884 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f0b8464-6e7e-408c-8bd2-6a30e1e263f7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3aabc4c5aa8b4e5698680ec589143be6, user_id=e35645c0565c4de29bac3e0ad8f1f245, role_id=3a074738221e4601b5205715ba87d240)` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:30:36.833260 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f0b8464-6e7e-408c-8bd2-6a30e1e263f7 admin admin] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:30:36.835236 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: [pid: 80245|app: 0|req: 885/1786] 10.222.0.47 () {60 vars in 1365 bytes} [Thu Jul 21 08:30:35 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 840 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 08:30:36.852880 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da2374fa-ccda-4c5d-9ddd-4fc782ec2709 None None] REQUEST_METHOD: `POST` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:30:36.853171 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da2374fa-ccda-4c5d-9ddd-4fc782ec2709 None None] SCRIPT_NAME: `/identity` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:30:36.853460 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da2374fa-ccda-4c5d-9ddd-4fc782ec2709 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:30:36.914171 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 902/1787] 10.222.0.47 () {60 vars in 1482 bytes} [Thu Jul 21 08:30:36 2022] PUT /identity/v3/projects/3aabc4c5aa8b4e5698680ec589143be6/users/e35645c0565c4de29bac3e0ad8f1f245/roles/3a074738221e4601b5205715ba87d240 => generated 0 bytes in 150 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 08:30:36.920775 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc2a6e54-232e-47ca-a38f-a83310a1f23d None None] REQUEST_METHOD: `POST` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:30:36.923209 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc2a6e54-232e-47ca-a38f-a83310a1f23d None None] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:30:36.923880 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc2a6e54-232e-47ca-a38f-a83310a1f23d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:30:36.960582 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.auth.core [None req-da2374fa-ccda-4c5d-9ddd-4fc782ec2709 None None] MFA Rules not processed for user `2dce40b8d74c41ec94cc36e6ff05d412`. Rule list: `[]` (Enabled: `True`). {{(pid=80245) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 08:30:36.968638 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.fernet_utils [None req-da2374fa-ccda-4c5d-9ddd-4fc782ec2709 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80245) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 08:30:37.058450 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.auth.core [None req-fc2a6e54-232e-47ca-a38f-a83310a1f23d None None] MFA Rules not processed for user `e35645c0565c4de29bac3e0ad8f1f245`. Rule list: `[]` (Enabled: `True`). {{(pid=80244) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 08:30:37.064927 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.fernet_utils [None req-fc2a6e54-232e-47ca-a38f-a83310a1f23d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80244) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 08:30:37.271916 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: [pid: 80245|app: 0|req: 886/1788] 10.222.0.47 () {58 vars in 957 bytes} [Thu Jul 21 08:30:36 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 424 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 08:30:37.276690 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 903/1789] 10.222.0.47 () {58 vars in 957 bytes} [Thu Jul 21 08:30:36 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 362 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 08:30:37.282637 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da2374fa-ccda-4c5d-9ddd-4fc782ec2709 None None] Authenticating user token {{(pid=80245) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:30:37.309889 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc2a6e54-232e-47ca-a38f-a83310a1f23d None None] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:30:37.317055 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da2374fa-ccda-4c5d-9ddd-4fc782ec2709 None None] Validating token access rules against request {{(pid=80245) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:30:37.330797 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7e02b0c-de65-487b-b236-b2de0fb200c1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '42d15fc0515945acacbbffcf2356c7a5', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80245) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:30:37.331536 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7e02b0c-de65-487b-b236-b2de0fb200c1 admin admin] REQUEST_METHOD: `POST` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:30:37.331827 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7e02b0c-de65-487b-b236-b2de0fb200c1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:30:37.332108 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7e02b0c-de65-487b-b236-b2de0fb200c1 admin admin] PATH_INFO: `/v3/projects` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:30:37.333441 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7e02b0c-de65-487b-b236-b2de0fb200c1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:30:37.338361 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7e02b0c-de65-487b-b236-b2de0fb200c1 admin admin] RBAC: Authorization granted {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:30:37.442930 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: [pid: 80245|app: 0|req: 887/1790] 10.222.0.47 () {60 vars in 1152 bytes} [Thu Jul 21 08:30:37 2022] POST /identity/v3/projects => generated 377 bytes in 160 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 08:30:37.451060 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7e02b0c-de65-487b-b236-b2de0fb200c1 admin admin] Authenticating user token {{(pid=80245) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:30:37.517755 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc2a6e54-232e-47ca-a38f-a83310a1f23d None None] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:30:37.540439 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG dbcounter [-] [80244] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80244) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:30:37.689134 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7e02b0c-de65-487b-b236-b2de0fb200c1 admin admin] Validating token access rules against request {{(pid=80245) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:30:37.711873 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9ac2d9f-d952-4059-8a72-2c4130ebd606 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '42d15fc0515945acacbbffcf2356c7a5', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80245) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:30:37.713401 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9ac2d9f-d952-4059-8a72-2c4130ebd606 admin admin] REQUEST_METHOD: `POST` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:30:37.713693 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9ac2d9f-d952-4059-8a72-2c4130ebd606 admin admin] SCRIPT_NAME: `/identity` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:30:37.715988 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9ac2d9f-d952-4059-8a72-2c4130ebd606 admin admin] PATH_INFO: `/v3/users` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:30:37.717207 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9ac2d9f-d952-4059-8a72-2c4130ebd606 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:30:37.723402 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9ac2d9f-d952-4059-8a72-2c4130ebd606 admin admin] RBAC: Authorization granted {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:30:37.736985 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3543c222-8726-470d-9ffe-ccde5f157ad4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20ca3c3bd3774691b3374cfbfec6e494', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4faef13aa50e4e36b6c265fb1398fa14', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:30:37.739137 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3543c222-8726-470d-9ffe-ccde5f157ad4 service neutron] REQUEST_METHOD: `GET` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:30:37.739465 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3543c222-8726-470d-9ffe-ccde5f157ad4 service neutron] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:30:37.739734 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3543c222-8726-470d-9ffe-ccde5f157ad4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:30:37.740681 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG dbcounter [-] [80245] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80245) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:30:37.781203 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3543c222-8726-470d-9ffe-ccde5f157ad4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:30:37.788310 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3543c222-8726-470d-9ffe-ccde5f157ad4 service neutron] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:30:37.794519 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: [pid: 80245|app: 0|req: 888/1791] 10.222.0.47 () {60 vars in 1143 bytes} [Thu Jul 21 08:30:37 2022] POST /identity/v3/users => generated 295 bytes in 341 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 08:30:37.796227 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9ac2d9f-d952-4059-8a72-2c4130ebd606 admin admin] Authenticating user token {{(pid=80245) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:30:37.825519 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 904/1792] 10.222.0.47 () {60 vars in 1365 bytes} [Thu Jul 21 08:30:37 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 514 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 08:30:37.852848 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9ac2d9f-d952-4059-8a72-2c4130ebd606 admin admin] Validating token access rules against request {{(pid=80245) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:30:37.873893 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f201c96-2011-46ee-9e71-0a95aa58a4a6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '42d15fc0515945acacbbffcf2356c7a5', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80245) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:30:37.876334 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f201c96-2011-46ee-9e71-0a95aa58a4a6 admin admin] REQUEST_METHOD: `GET` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:30:37.876334 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f201c96-2011-46ee-9e71-0a95aa58a4a6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:30:37.876334 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f201c96-2011-46ee-9e71-0a95aa58a4a6 admin admin] PATH_INFO: `/v3/roles` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:30:37.876334 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f201c96-2011-46ee-9e71-0a95aa58a4a6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:30:37.898064 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f201c96-2011-46ee-9e71-0a95aa58a4a6 admin admin] RBAC: Authorization granted {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:30:37.984757 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: [pid: 80245|app: 0|req: 889/1793] 10.222.0.47 () {58 vars in 1121 bytes} [Thu Jul 21 08:30:37 2022] GET /identity/v3/roles => generated 1420 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 08:30:37.988502 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3543c222-8726-470d-9ffe-ccde5f157ad4 service neutron] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:30:38.027238 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3543c222-8726-470d-9ffe-ccde5f157ad4 service neutron] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:30:38.038787 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6103d24-3768-4c67-bb52-21f91c875e16 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '42d15fc0515945acacbbffcf2356c7a5', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:30:38.040042 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6103d24-3768-4c67-bb52-21f91c875e16 admin admin] REQUEST_METHOD: `PUT` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:30:38.040850 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6103d24-3768-4c67-bb52-21f91c875e16 admin admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:30:38.042662 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6103d24-3768-4c67-bb52-21f91c875e16 admin admin] PATH_INFO: `/v3/projects/217c0284201f4cf294e98a44d8269e3c/users/b60c6a5b83434534ac72bf2d555c5848/roles/e29870f6e9e2411a9db0d5bf45f32ca5` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:30:38.109366 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6103d24-3768-4c67-bb52-21f91c875e16 admin admin] RBAC: Authorizing `identity:create_grant(project_id=217c0284201f4cf294e98a44d8269e3c, user_id=b60c6a5b83434534ac72bf2d555c5848, role_id=e29870f6e9e2411a9db0d5bf45f32ca5)` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:30:38.135006 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6103d24-3768-4c67-bb52-21f91c875e16 admin admin] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:30:38.193528 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 905/1794] 10.222.0.47 () {60 vars in 1482 bytes} [Thu Jul 21 08:30:37 2022] PUT /identity/v3/projects/217c0284201f4cf294e98a44d8269e3c/users/b60c6a5b83434534ac72bf2d555c5848/roles/e29870f6e9e2411a9db0d5bf45f32ca5 => generated 0 bytes in 203 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 08:30:38.193708 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f201c96-2011-46ee-9e71-0a95aa58a4a6 admin admin] Authenticating user token {{(pid=80245) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:30:38.205523 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e08fe41c-f3ac-4681-b2ff-fb1f967814cc None None] REQUEST_METHOD: `POST` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:30:38.213775 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f201c96-2011-46ee-9e71-0a95aa58a4a6 admin admin] Validating token access rules against request {{(pid=80245) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:30:38.216165 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e08fe41c-f3ac-4681-b2ff-fb1f967814cc None None] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:30:38.218458 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e08fe41c-f3ac-4681-b2ff-fb1f967814cc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:30:38.346138 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.auth.core [None req-e08fe41c-f3ac-4681-b2ff-fb1f967814cc None None] MFA Rules not processed for user `20ca3c3bd3774691b3374cfbfec6e494`. Rule list: `[]` (Enabled: `True`). {{(pid=80244) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 08:30:38.365995 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.fernet_utils [None req-e08fe41c-f3ac-4681-b2ff-fb1f967814cc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80244) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 08:30:38.525198 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f6d04f2-7771-425c-abf7-7c781eefb552 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '42d15fc0515945acacbbffcf2356c7a5', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80245) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:30:38.527974 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f6d04f2-7771-425c-abf7-7c781eefb552 admin admin] REQUEST_METHOD: `GET` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:30:38.528738 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f6d04f2-7771-425c-abf7-7c781eefb552 admin admin] SCRIPT_NAME: `/identity` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:30:38.530126 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f6d04f2-7771-425c-abf7-7c781eefb552 admin admin] PATH_INFO: `/v3/roles` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:30:38.533021 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f6d04f2-7771-425c-abf7-7c781eefb552 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:30:38.539720 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f6d04f2-7771-425c-abf7-7c781eefb552 admin admin] RBAC: Authorization granted {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:30:38.590706 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: [pid: 80245|app: 0|req: 890/1795] 10.222.0.47 () {58 vars in 1121 bytes} [Thu Jul 21 08:30:38 2022] GET /identity/v3/roles => generated 1420 bytes in 397 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 21 08:30:38.592536 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f6d04f2-7771-425c-abf7-7c781eefb552 admin admin] Authenticating user token {{(pid=80245) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:30:38.605294 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 906/1796] 10.222.0.18 () {58 vars in 1015 bytes} [Thu Jul 21 08:30:38 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 407 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 08:30:38.626216 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f6d04f2-7771-425c-abf7-7c781eefb552 admin admin] Validating token access rules against request {{(pid=80245) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:30:38.648024 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca431d6c-ffde-402c-b340-93d2df6f56ed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '42d15fc0515945acacbbffcf2356c7a5', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80245) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:30:38.650466 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca431d6c-ffde-402c-b340-93d2df6f56ed admin admin] REQUEST_METHOD: `PUT` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:30:38.650466 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca431d6c-ffde-402c-b340-93d2df6f56ed admin admin] SCRIPT_NAME: `/identity` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:30:38.650466 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca431d6c-ffde-402c-b340-93d2df6f56ed admin admin] PATH_INFO: `/v3/projects/217c0284201f4cf294e98a44d8269e3c/users/b60c6a5b83434534ac72bf2d555c5848/roles/e29870f6e9e2411a9db0d5bf45f32ca5` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:30:38.666872 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca431d6c-ffde-402c-b340-93d2df6f56ed admin admin] RBAC: Authorizing `identity:create_grant(project_id=217c0284201f4cf294e98a44d8269e3c, user_id=b60c6a5b83434534ac72bf2d555c5848, role_id=e29870f6e9e2411a9db0d5bf45f32ca5)` {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:30:38.670232 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca431d6c-ffde-402c-b340-93d2df6f56ed admin admin] RBAC: Authorization granted {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:30:38.739175 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: [pid: 80245|app: 0|req: 891/1797] 10.222.0.47 () {60 vars in 1482 bytes} [Thu Jul 21 08:30:38 2022] PUT /identity/v3/projects/217c0284201f4cf294e98a44d8269e3c/users/b60c6a5b83434534ac72bf2d555c5848/roles/e29870f6e9e2411a9db0d5bf45f32ca5 => generated 0 bytes in 147 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 08:30:38.756337 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3758a32b-0857-4e02-8030-2e30c1ee802b None None] REQUEST_METHOD: `POST` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:30:38.756337 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3758a32b-0857-4e02-8030-2e30c1ee802b None None] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:30:38.758137 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3758a32b-0857-4e02-8030-2e30c1ee802b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:30:38.857326 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.auth.core [None req-3758a32b-0857-4e02-8030-2e30c1ee802b None None] MFA Rules not processed for user `b60c6a5b83434534ac72bf2d555c5848`. Rule list: `[]` (Enabled: `True`). {{(pid=80244) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 08:30:38.861019 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.fernet_utils [None req-3758a32b-0857-4e02-8030-2e30c1ee802b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80244) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 08:30:39.023652 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca431d6c-ffde-402c-b340-93d2df6f56ed admin admin] Authenticating user token {{(pid=80245) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:30:39.025657 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.fernet_utils [None req-ca431d6c-ffde-402c-b340-93d2df6f56ed admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80245) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 08:30:39.227972 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 907/1798] 10.222.0.47 () {58 vars in 957 bytes} [Thu Jul 21 08:30:38 2022] POST /identity/v3/auth/tokens => generated 3311 bytes in 473 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 08:30:39.247246 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3758a32b-0857-4e02-8030-2e30c1ee802b None None] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:30:39.253196 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.fernet_utils [None req-3758a32b-0857-4e02-8030-2e30c1ee802b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80244) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 08:30:39.312205 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca431d6c-ffde-402c-b340-93d2df6f56ed admin admin] Validating token access rules against request {{(pid=80245) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:30:39.331267 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f067bc5c-ae58-4ed4-be09-b795b2ec3ef7 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4f40410b7674687ab5362ac982248ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4faef13aa50e4e36b6c265fb1398fa14', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80245) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:30:39.333956 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f067bc5c-ae58-4ed4-be09-b795b2ec3ef7 service cinder] REQUEST_METHOD: `GET` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:30:39.335546 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f067bc5c-ae58-4ed4-be09-b795b2ec3ef7 service cinder] SCRIPT_NAME: `/identity` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:30:39.335846 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f067bc5c-ae58-4ed4-be09-b795b2ec3ef7 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:30:39.352514 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.fernet_utils [None req-f067bc5c-ae58-4ed4-be09-b795b2ec3ef7 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80245) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 08:30:39.393214 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG dbcounter [-] [80244] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80244) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:30:39.506593 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3758a32b-0857-4e02-8030-2e30c1ee802b None None] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:30:39.525294 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-620fa997-8cfa-4bc8-a096-5bdccc38c658 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20ca3c3bd3774691b3374cfbfec6e494', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4faef13aa50e4e36b6c265fb1398fa14', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:30:39.526379 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-620fa997-8cfa-4bc8-a096-5bdccc38c658 service neutron] REQUEST_METHOD: `GET` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:30:39.526906 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-620fa997-8cfa-4bc8-a096-5bdccc38c658 service neutron] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:30:39.527419 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-620fa997-8cfa-4bc8-a096-5bdccc38c658 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:30:39.554811 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-620fa997-8cfa-4bc8-a096-5bdccc38c658 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:30:39.558564 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-620fa997-8cfa-4bc8-a096-5bdccc38c658 service neutron] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:30:39.568728 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f067bc5c-ae58-4ed4-be09-b795b2ec3ef7 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:30:39.570752 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f067bc5c-ae58-4ed4-be09-b795b2ec3ef7 service cinder] RBAC: Authorization granted {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:30:39.581149 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 908/1799] 10.222.0.47 () {60 vars in 1365 bytes} [Thu Jul 21 08:30:39 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 334 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 08:30:39.582728 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-620fa997-8cfa-4bc8-a096-5bdccc38c658 service neutron] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:30:39.584660 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.fernet_utils [None req-620fa997-8cfa-4bc8-a096-5bdccc38c658 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80244) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 08:30:39.603568 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-620fa997-8cfa-4bc8-a096-5bdccc38c658 service neutron] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:30:39.609355 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e0bf0f5-7ae1-4b9b-8df0-f4e42210072e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4f40410b7674687ab5362ac982248ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4faef13aa50e4e36b6c265fb1398fa14', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:30:39.610097 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e0bf0f5-7ae1-4b9b-8df0-f4e42210072e service cinder] REQUEST_METHOD: `GET` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:30:39.610634 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e0bf0f5-7ae1-4b9b-8df0-f4e42210072e service cinder] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:30:39.611076 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e0bf0f5-7ae1-4b9b-8df0-f4e42210072e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:30:39.634845 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: [pid: 80245|app: 0|req: 892/1800] 10.222.0.47 () {60 vars in 1365 bytes} [Thu Jul 21 08:30:39 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 612 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 08:30:39.636268 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f067bc5c-ae58-4ed4-be09-b795b2ec3ef7 service cinder] Authenticating user token {{(pid=80245) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:30:39.660099 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f067bc5c-ae58-4ed4-be09-b795b2ec3ef7 service cinder] Validating token access rules against request {{(pid=80245) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:30:39.669113 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cc4b855-8712-4c4d-95dc-03ffe0efd855 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20ca3c3bd3774691b3374cfbfec6e494', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4faef13aa50e4e36b6c265fb1398fa14', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80245) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:30:39.670794 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cc4b855-8712-4c4d-95dc-03ffe0efd855 service neutron] REQUEST_METHOD: `GET` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:30:39.672106 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cc4b855-8712-4c4d-95dc-03ffe0efd855 service neutron] SCRIPT_NAME: `/identity` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:30:39.672871 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cc4b855-8712-4c4d-95dc-03ffe0efd855 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:30:39.701526 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cc4b855-8712-4c4d-95dc-03ffe0efd855 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:30:39.705368 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cc4b855-8712-4c4d-95dc-03ffe0efd855 service neutron] RBAC: Authorization granted {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:30:39.717281 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG dbcounter [-] [80245] Writing DB stats keystone:INSERT=4,keystone:SELECT=96 {{(pid=80245) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:30:39.750515 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: [pid: 80245|app: 0|req: 893/1801] 10.222.0.47 () {60 vars in 1365 bytes} [Thu Jul 21 08:30:39 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 110 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 08:30:39.751944 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cc4b855-8712-4c4d-95dc-03ffe0efd855 service neutron] Authenticating user token {{(pid=80245) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:30:39.793079 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cc4b855-8712-4c4d-95dc-03ffe0efd855 service neutron] Validating token access rules against request {{(pid=80245) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:30:39.836700 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9930581-ae65-444f-a74c-6b81d2e2f31e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20ca3c3bd3774691b3374cfbfec6e494', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4faef13aa50e4e36b6c265fb1398fa14', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80245) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:30:39.847618 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9930581-ae65-444f-a74c-6b81d2e2f31e service neutron] REQUEST_METHOD: `GET` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:30:39.849504 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9930581-ae65-444f-a74c-6b81d2e2f31e service neutron] SCRIPT_NAME: `/identity` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:30:39.850470 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9930581-ae65-444f-a74c-6b81d2e2f31e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:30:39.892321 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e0bf0f5-7ae1-4b9b-8df0-f4e42210072e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:30:39.901776 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9930581-ae65-444f-a74c-6b81d2e2f31e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:30:39.904886 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9930581-ae65-444f-a74c-6b81d2e2f31e service neutron] RBAC: Authorization granted {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:30:39.908721 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e0bf0f5-7ae1-4b9b-8df0-f4e42210072e service cinder] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:30:39.916835 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: [pid: 80245|app: 0|req: 894/1802] 10.222.0.47 () {60 vars in 1365 bytes} [Thu Jul 21 08:30:39 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 165 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 08:30:39.979121 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 909/1803] 10.222.0.47 () {60 vars in 1365 bytes} [Thu Jul 21 08:30:39 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 397 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 08:30:41.880677 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9930581-ae65-444f-a74c-6b81d2e2f31e service neutron] Authenticating user token {{(pid=80245) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:30:41.885745 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.fernet_utils [None req-e9930581-ae65-444f-a74c-6b81d2e2f31e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80245) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 08:30:42.179834 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e0bf0f5-7ae1-4b9b-8df0-f4e42210072e service cinder] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:30:42.181619 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.fernet_utils [None req-8e0bf0f5-7ae1-4b9b-8df0-f4e42210072e service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80244) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 08:30:42.244083 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e0bf0f5-7ae1-4b9b-8df0-f4e42210072e service cinder] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:30:42.247509 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9930581-ae65-444f-a74c-6b81d2e2f31e service neutron] Validating token access rules against request {{(pid=80245) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:30:42.271666 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6280637-09ec-42e9-876d-9094a13f1764 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '42d15fc0515945acacbbffcf2356c7a5', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80245) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:30:42.272566 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6280637-09ec-42e9-876d-9094a13f1764 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:30:42.272860 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6280637-09ec-42e9-876d-9094a13f1764 admin admin] SCRIPT_NAME: `/identity` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:30:42.273138 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6280637-09ec-42e9-876d-9094a13f1764 admin admin] PATH_INFO: `/v3/projects/4af2477817324d4cb2e5e86cb5d3d103` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:30:42.280559 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6280637-09ec-42e9-876d-9094a13f1764 admin admin] RBAC: Authorizing `identity:delete_project(project_id=4af2477817324d4cb2e5e86cb5d3d103)` {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:30:42.289435 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6280637-09ec-42e9-876d-9094a13f1764 admin admin] RBAC: Authorization granted {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:30:42.295623 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b524f49-56c4-4fd5-9674-22dd6c219d0a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '42d15fc0515945acacbbffcf2356c7a5', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:30:42.298405 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b524f49-56c4-4fd5-9674-22dd6c219d0a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:30:42.298709 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b524f49-56c4-4fd5-9674-22dd6c219d0a admin admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:30:42.298980 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b524f49-56c4-4fd5-9674-22dd6c219d0a admin admin] PATH_INFO: `/v3/projects/16c2903b2f4c473eb05b9a29920ae497` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:30:42.316009 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b524f49-56c4-4fd5-9674-22dd6c219d0a admin admin] RBAC: Authorizing `identity:delete_project(project_id=16c2903b2f4c473eb05b9a29920ae497)` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:30:42.326886 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b524f49-56c4-4fd5-9674-22dd6c219d0a admin admin] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:30:42.518485 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.notifications [None req-e6280637-09ec-42e9-876d-9094a13f1764 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4af2477817324d4cb2e5e86cb5d3d103', 'request_id': 'req-e6280637-09ec-42e9-876d-9094a13f1764'} {{(pid=80245) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 08:30:42.545763 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.notifications [None req-e6280637-09ec-42e9-876d-9094a13f1764 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4af2477817324d4cb2e5e86cb5d3d103', 'request_id': 'req-e6280637-09ec-42e9-876d-9094a13f1764'} {{(pid=80245) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 08:30:42.583800 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.notifications [None req-e6280637-09ec-42e9-876d-9094a13f1764 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4af2477817324d4cb2e5e86cb5d3d103', 'request_id': 'req-e6280637-09ec-42e9-876d-9094a13f1764'} {{(pid=80245) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 08:30:42.633068 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.notifications [None req-5b524f49-56c4-4fd5-9674-22dd6c219d0a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '16c2903b2f4c473eb05b9a29920ae497', 'request_id': 'req-5b524f49-56c4-4fd5-9674-22dd6c219d0a'} {{(pid=80244) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 08:30:42.636627 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.notifications [None req-e6280637-09ec-42e9-876d-9094a13f1764 admin admin] The token cache is being invalidate because project 4af2477817324d4cb2e5e86cb5d3d103 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80245) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 21 08:30:42.637755 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.notifications [None req-e6280637-09ec-42e9-876d-9094a13f1764 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80245) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 08:30:42.639633 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.notifications [None req-e6280637-09ec-42e9-876d-9094a13f1764 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80245) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 08:30:42.645086 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: [pid: 80245|app: 0|req: 895/1804] 10.222.0.47 () {58 vars in 1232 bytes} [Thu Jul 21 08:30:41 2022] DELETE /identity/v3/projects/4af2477817324d4cb2e5e86cb5d3d103 => generated 0 bytes in 766 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 08:30:42.648856 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6280637-09ec-42e9-876d-9094a13f1764 admin admin] Authenticating user token {{(pid=80245) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:30:42.652825 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.fernet_utils [None req-e6280637-09ec-42e9-876d-9094a13f1764 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80245) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 08:30:42.674181 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.notifications [None req-5b524f49-56c4-4fd5-9674-22dd6c219d0a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '16c2903b2f4c473eb05b9a29920ae497', 'request_id': 'req-5b524f49-56c4-4fd5-9674-22dd6c219d0a'} {{(pid=80244) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 08:30:42.711805 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.notifications [None req-5b524f49-56c4-4fd5-9674-22dd6c219d0a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '16c2903b2f4c473eb05b9a29920ae497', 'request_id': 'req-5b524f49-56c4-4fd5-9674-22dd6c219d0a'} {{(pid=80244) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 08:30:42.795984 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.notifications [None req-5b524f49-56c4-4fd5-9674-22dd6c219d0a admin admin] The token cache is being invalidate because project 16c2903b2f4c473eb05b9a29920ae497 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80244) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 21 08:30:42.799724 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.notifications [None req-5b524f49-56c4-4fd5-9674-22dd6c219d0a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80244) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 08:30:42.802478 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.notifications [None req-5b524f49-56c4-4fd5-9674-22dd6c219d0a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80244) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 08:30:42.808342 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 910/1805] 10.222.0.47 () {58 vars in 1232 bytes} [Thu Jul 21 08:30:42 2022] DELETE /identity/v3/projects/16c2903b2f4c473eb05b9a29920ae497 => generated 0 bytes in 629 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 08:30:42.812478 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b524f49-56c4-4fd5-9674-22dd6c219d0a admin admin] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:30:42.814367 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.fernet_utils [None req-5b524f49-56c4-4fd5-9674-22dd6c219d0a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80244) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 08:30:43.027536 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6280637-09ec-42e9-876d-9094a13f1764 admin admin] Validating token access rules against request {{(pid=80245) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:30:43.060188 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG dbcounter [-] [80244] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80244) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:30:43.142928 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG dbcounter [-] [80245] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80245) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:30:43.151973 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b524f49-56c4-4fd5-9674-22dd6c219d0a admin admin] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:30:43.191653 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7b80454-4980-445b-88bc-0f8abd9f2e83 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '42d15fc0515945acacbbffcf2356c7a5', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:30:43.195798 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7b80454-4980-445b-88bc-0f8abd9f2e83 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:30:43.196026 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7b80454-4980-445b-88bc-0f8abd9f2e83 admin admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:30:43.198744 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7b80454-4980-445b-88bc-0f8abd9f2e83 admin admin] PATH_INFO: `/v3/users/04c2ae4751b1424eb9ae873706a5150b` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:30:43.215859 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7b80454-4980-445b-88bc-0f8abd9f2e83 admin admin] RBAC: Authorizing `identity:delete_user(user_id=04c2ae4751b1424eb9ae873706a5150b)` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:30:43.223862 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7b80454-4980-445b-88bc-0f8abd9f2e83 admin admin] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:30:43.385104 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ef75d7b-0f48-458e-821e-99f076f8d5f3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '42d15fc0515945acacbbffcf2356c7a5', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80245) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:30:43.389427 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ef75d7b-0f48-458e-821e-99f076f8d5f3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:30:43.389938 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ef75d7b-0f48-458e-821e-99f076f8d5f3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:30:43.391053 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ef75d7b-0f48-458e-821e-99f076f8d5f3 admin admin] PATH_INFO: `/v3/users/21e4296be8d245aa847b55c655d403c5` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:30:43.403081 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ef75d7b-0f48-458e-821e-99f076f8d5f3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=21e4296be8d245aa847b55c655d403c5)` {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:30:43.408820 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ef75d7b-0f48-458e-821e-99f076f8d5f3 admin admin] RBAC: Authorization granted {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:30:43.490612 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.notifications [None req-a7b80454-4980-445b-88bc-0f8abd9f2e83 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '04c2ae4751b1424eb9ae873706a5150b'} {{(pid=80244) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 08:30:43.515309 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.notifications [None req-a7b80454-4980-445b-88bc-0f8abd9f2e83 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '04c2ae4751b1424eb9ae873706a5150b'} {{(pid=80244) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 08:30:43.573641 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.notifications [None req-a7b80454-4980-445b-88bc-0f8abd9f2e83 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '04c2ae4751b1424eb9ae873706a5150b'} {{(pid=80244) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 08:30:43.585109 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.notifications [None req-a7b80454-4980-445b-88bc-0f8abd9f2e83 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '04c2ae4751b1424eb9ae873706a5150b'} {{(pid=80244) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 08:30:43.608699 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.notifications [None req-a7b80454-4980-445b-88bc-0f8abd9f2e83 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '04c2ae4751b1424eb9ae873706a5150b'} {{(pid=80244) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 08:30:43.609999 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.notifications [None req-8ef75d7b-0f48-458e-821e-99f076f8d5f3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '21e4296be8d245aa847b55c655d403c5'} {{(pid=80245) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 08:30:43.626768 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.notifications [None req-8ef75d7b-0f48-458e-821e-99f076f8d5f3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '21e4296be8d245aa847b55c655d403c5'} {{(pid=80245) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 08:30:43.639592 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 911/1806] 10.222.0.47 () {58 vars in 1223 bytes} [Thu Jul 21 08:30:42 2022] DELETE /identity/v3/users/04c2ae4751b1424eb9ae873706a5150b => generated 0 bytes in 827 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 08:30:43.691503 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.notifications [None req-8ef75d7b-0f48-458e-821e-99f076f8d5f3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '21e4296be8d245aa847b55c655d403c5'} {{(pid=80245) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 08:30:43.703557 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.notifications [None req-8ef75d7b-0f48-458e-821e-99f076f8d5f3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '21e4296be8d245aa847b55c655d403c5'} {{(pid=80245) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 08:30:43.704526 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.notifications [None req-8ef75d7b-0f48-458e-821e-99f076f8d5f3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '21e4296be8d245aa847b55c655d403c5'} {{(pid=80245) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 08:30:43.714049 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: [pid: 80245|app: 0|req: 896/1807] 10.222.0.47 () {58 vars in 1223 bytes} [Thu Jul 21 08:30:42 2022] DELETE /identity/v3/users/21e4296be8d245aa847b55c655d403c5 => generated 0 bytes in 1066 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 08:30:44.713161 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-102865f6-a620-46d3-87ba-14bb967c5d50 None None] REQUEST_METHOD: `GET` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:30:44.717703 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-102865f6-a620-46d3-87ba-14bb967c5d50 None None] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:30:44.718302 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-102865f6-a620-46d3-87ba-14bb967c5d50 None None] PATH_INFO: `/` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:30:44.723467 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 912/1808] 10.222.0.47 () {54 vars in 921 bytes} [Thu Jul 21 08:30:44 2022] GET /identity => generated 270 bytes in 14 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 08:30:44.796027 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f17b9919-ac83-42a4-8940-d1769495325a None None] REQUEST_METHOD: `POST` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:30:44.799218 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f17b9919-ac83-42a4-8940-d1769495325a None None] SCRIPT_NAME: `/identity` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:30:44.800373 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f17b9919-ac83-42a4-8940-d1769495325a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:30:44.879129 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.auth.core [None req-f17b9919-ac83-42a4-8940-d1769495325a None None] MFA Rules not processed for user `ac2b5b99a36442148fe743269054e991`. Rule list: `[]` (Enabled: `True`). {{(pid=80245) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 08:30:44.881057 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.fernet_utils [None req-f17b9919-ac83-42a4-8940-d1769495325a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80245) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 08:30:45.069866 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: [pid: 80245|app: 0|req: 897/1809] 10.222.0.47 () {58 vars in 1018 bytes} [Thu Jul 21 08:30:44 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 275 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 08:30:45.121155 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-102865f6-a620-46d3-87ba-14bb967c5d50 None None] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:30:45.125097 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.fernet_utils [None req-102865f6-a620-46d3-87ba-14bb967c5d50 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80244) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 08:30:45.180278 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-102865f6-a620-46d3-87ba-14bb967c5d50 None None] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:30:45.187633 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-756bc4c0-13b3-433d-aecd-ae0cf777e896 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2b5b99a36442148fe743269054e991', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4faef13aa50e4e36b6c265fb1398fa14', 'project_domain_id': 'default', 'roles': ['member', 'service', 'ResellerAdmin', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:30:45.188455 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-756bc4c0-13b3-433d-aecd-ae0cf777e896 service nova] REQUEST_METHOD: `GET` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:30:45.189334 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-756bc4c0-13b3-433d-aecd-ae0cf777e896 service nova] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:30:45.190498 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-756bc4c0-13b3-433d-aecd-ae0cf777e896 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:30:45.278329 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-756bc4c0-13b3-433d-aecd-ae0cf777e896 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:30:45.284342 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-756bc4c0-13b3-433d-aecd-ae0cf777e896 service nova] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:30:45.354927 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 913/1810] 10.222.0.47 () {60 vars in 1365 bytes} [Thu Jul 21 08:30:45 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 234 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 08:30:46.194344 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f17b9919-ac83-42a4-8940-d1769495325a None None] Authenticating user token {{(pid=80245) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:30:46.196761 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.fernet_utils [None req-f17b9919-ac83-42a4-8940-d1769495325a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80245) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 08:30:46.405349 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG dbcounter [-] [80245] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80245) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:30:46.428633 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-756bc4c0-13b3-433d-aecd-ae0cf777e896 service nova] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:30:46.436517 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.fernet_utils [None req-756bc4c0-13b3-433d-aecd-ae0cf777e896 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80244) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 08:30:46.513213 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f17b9919-ac83-42a4-8940-d1769495325a None None] Validating token access rules against request {{(pid=80245) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:30:46.533170 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15964153-4d9c-4d4c-8434-a5389ead7e44 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '42d15fc0515945acacbbffcf2356c7a5', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80245) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:30:46.534352 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15964153-4d9c-4d4c-8434-a5389ead7e44 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:30:46.535264 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15964153-4d9c-4d4c-8434-a5389ead7e44 admin admin] SCRIPT_NAME: `/identity` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:30:46.535828 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15964153-4d9c-4d4c-8434-a5389ead7e44 admin admin] PATH_INFO: `/v3/projects/ebfbe7fe67e94ce0be4508e814147208` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:30:46.537412 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15964153-4d9c-4d4c-8434-a5389ead7e44 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ebfbe7fe67e94ce0be4508e814147208)` {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:30:46.539636 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15964153-4d9c-4d4c-8434-a5389ead7e44 admin admin] RBAC: Authorization granted {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:30:46.622304 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.notifications [None req-15964153-4d9c-4d4c-8434-a5389ead7e44 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ebfbe7fe67e94ce0be4508e814147208', 'request_id': 'req-15964153-4d9c-4d4c-8434-a5389ead7e44'} {{(pid=80245) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 08:30:46.638820 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.notifications [None req-15964153-4d9c-4d4c-8434-a5389ead7e44 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ebfbe7fe67e94ce0be4508e814147208', 'request_id': 'req-15964153-4d9c-4d4c-8434-a5389ead7e44'} {{(pid=80245) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 08:30:46.648621 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.notifications [None req-15964153-4d9c-4d4c-8434-a5389ead7e44 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ebfbe7fe67e94ce0be4508e814147208', 'request_id': 'req-15964153-4d9c-4d4c-8434-a5389ead7e44'} {{(pid=80245) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 08:30:46.673436 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.notifications [None req-15964153-4d9c-4d4c-8434-a5389ead7e44 admin admin] The token cache is being invalidate because project ebfbe7fe67e94ce0be4508e814147208 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80245) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 21 08:30:46.677136 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.notifications [None req-15964153-4d9c-4d4c-8434-a5389ead7e44 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80245) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 08:30:46.679049 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.notifications [None req-15964153-4d9c-4d4c-8434-a5389ead7e44 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80245) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 08:30:46.686872 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: [pid: 80245|app: 0|req: 898/1811] 10.222.0.47 () {58 vars in 1232 bytes} [Thu Jul 21 08:30:46 2022] DELETE /identity/v3/projects/ebfbe7fe67e94ce0be4508e814147208 => generated 0 bytes in 492 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 08:30:46.693039 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-143072d7-a9b8-4400-a354-73044ba4a76a None None] REQUEST_METHOD: `POST` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:30:46.693690 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-143072d7-a9b8-4400-a354-73044ba4a76a None None] SCRIPT_NAME: `/identity` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:30:46.694415 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-143072d7-a9b8-4400-a354-73044ba4a76a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:30:46.718980 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-756bc4c0-13b3-433d-aecd-ae0cf777e896 service nova] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:30:46.735550 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.auth.core [None req-143072d7-a9b8-4400-a354-73044ba4a76a None None] MFA Rules not processed for user `2dce40b8d74c41ec94cc36e6ff05d412`. Rule list: `[]` (Enabled: `True`). {{(pid=80245) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 08:30:46.739236 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.fernet_utils [None req-143072d7-a9b8-4400-a354-73044ba4a76a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80245) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 08:30:46.836395 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54a279ed-7098-4190-bb21-d6314c738bb4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '42d15fc0515945acacbbffcf2356c7a5', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:30:46.837778 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54a279ed-7098-4190-bb21-d6314c738bb4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:30:46.838437 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54a279ed-7098-4190-bb21-d6314c738bb4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:30:46.838931 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54a279ed-7098-4190-bb21-d6314c738bb4 admin admin] PATH_INFO: `/v3/projects/a0ca7cbc167d47a8b5e26af30f6f7fa1` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:30:46.841508 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54a279ed-7098-4190-bb21-d6314c738bb4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a0ca7cbc167d47a8b5e26af30f6f7fa1)` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:30:46.844090 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54a279ed-7098-4190-bb21-d6314c738bb4 admin admin] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:30:46.847868 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG dbcounter [-] [80244] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80244) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:30:46.881925 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: [pid: 80245|app: 0|req: 899/1812] 10.222.0.47 () {58 vars in 957 bytes} [Thu Jul 21 08:30:46 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 181 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 08:30:46.895921 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f884878d-a9ed-409d-9be9-92d81b75fc0e None None] REQUEST_METHOD: `POST` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:30:46.896946 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f884878d-a9ed-409d-9be9-92d81b75fc0e None None] SCRIPT_NAME: `/identity` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:30:46.897279 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f884878d-a9ed-409d-9be9-92d81b75fc0e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:30:46.940140 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.notifications [None req-54a279ed-7098-4190-bb21-d6314c738bb4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a0ca7cbc167d47a8b5e26af30f6f7fa1', 'request_id': 'req-54a279ed-7098-4190-bb21-d6314c738bb4'} {{(pid=80244) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 08:30:46.943670 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.auth.core [None req-f884878d-a9ed-409d-9be9-92d81b75fc0e None None] MFA Rules not processed for user `2dce40b8d74c41ec94cc36e6ff05d412`. Rule list: `[]` (Enabled: `True`). {{(pid=80245) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 08:30:46.945518 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.fernet_utils [None req-f884878d-a9ed-409d-9be9-92d81b75fc0e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80245) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 08:30:46.956844 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.notifications [None req-54a279ed-7098-4190-bb21-d6314c738bb4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a0ca7cbc167d47a8b5e26af30f6f7fa1', 'request_id': 'req-54a279ed-7098-4190-bb21-d6314c738bb4'} {{(pid=80244) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 08:30:46.971403 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.notifications [None req-54a279ed-7098-4190-bb21-d6314c738bb4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a0ca7cbc167d47a8b5e26af30f6f7fa1', 'request_id': 'req-54a279ed-7098-4190-bb21-d6314c738bb4'} {{(pid=80244) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 08:30:46.985171 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.notifications [None req-54a279ed-7098-4190-bb21-d6314c738bb4 admin admin] The token cache is being invalidate because project a0ca7cbc167d47a8b5e26af30f6f7fa1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80244) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 21 08:30:46.985535 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.notifications [None req-54a279ed-7098-4190-bb21-d6314c738bb4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80244) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 08:30:46.986333 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.notifications [None req-54a279ed-7098-4190-bb21-d6314c738bb4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80244) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 08:30:46.991871 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 914/1813] 10.222.0.47 () {58 vars in 1232 bytes} [Thu Jul 21 08:30:46 2022] DELETE /identity/v3/projects/a0ca7cbc167d47a8b5e26af30f6f7fa1 => generated 0 bytes in 566 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 21 08:30:46.993680 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4b628f3-33f5-47af-8a4f-b4c3e2a4890c None None] REQUEST_METHOD: `POST` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:30:46.994064 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4b628f3-33f5-47af-8a4f-b4c3e2a4890c None None] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:30:46.994449 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4b628f3-33f5-47af-8a4f-b4c3e2a4890c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:30:47.018885 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.auth.core [None req-b4b628f3-33f5-47af-8a4f-b4c3e2a4890c None None] MFA Rules not processed for user `2dce40b8d74c41ec94cc36e6ff05d412`. Rule list: `[]` (Enabled: `True`). {{(pid=80244) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 08:30:47.020170 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.fernet_utils [None req-b4b628f3-33f5-47af-8a4f-b4c3e2a4890c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80244) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 08:30:47.050670 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: [pid: 80245|app: 0|req: 900/1814] 10.222.0.47 () {58 vars in 957 bytes} [Thu Jul 21 08:30:46 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 21 08:30:47.051837 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f884878d-a9ed-409d-9be9-92d81b75fc0e None None] Authenticating user token {{(pid=80245) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:30:47.060087 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG dbcounter [-] [80245] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80245) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:30:47.065000 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f884878d-a9ed-409d-9be9-92d81b75fc0e None None] Validating token access rules against request {{(pid=80245) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:30:47.072727 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e181eeb5-cd54-4c7e-93e2-7d2d3cbaf6cb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '42d15fc0515945acacbbffcf2356c7a5', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80245) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:30:47.074270 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e181eeb5-cd54-4c7e-93e2-7d2d3cbaf6cb admin admin] REQUEST_METHOD: `GET` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:30:47.074846 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e181eeb5-cd54-4c7e-93e2-7d2d3cbaf6cb admin admin] SCRIPT_NAME: `/identity` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:30:47.075422 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e181eeb5-cd54-4c7e-93e2-7d2d3cbaf6cb admin admin] PATH_INFO: `/v3/domains` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:30:47.076973 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e181eeb5-cd54-4c7e-93e2-7d2d3cbaf6cb admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80245) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 08:30:47.077437 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e181eeb5-cd54-4c7e-93e2-7d2d3cbaf6cb admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:30:47.079495 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e181eeb5-cd54-4c7e-93e2-7d2d3cbaf6cb admin admin] RBAC: Authorization granted {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:30:47.095587 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: [pid: 80245|app: 0|req: 901/1815] 10.222.0.47 () {58 vars in 1152 bytes} [Thu Jul 21 08:30:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 21 08:30:47.098993 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad28a719-dadf-4a5e-b93a-4f8cc7eeca30 None None] REQUEST_METHOD: `GET` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:30:47.099289 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad28a719-dadf-4a5e-b93a-4f8cc7eeca30 None None] SCRIPT_NAME: `/identity` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:30:47.100495 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad28a719-dadf-4a5e-b93a-4f8cc7eeca30 None None] PATH_INFO: `/` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:30:47.102084 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: [pid: 80245|app: 0|req: 902/1816] 10.222.0.47 () {54 vars in 921 bytes} [Thu Jul 21 08:30:47 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 08:30:47.102696 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad28a719-dadf-4a5e-b93a-4f8cc7eeca30 None None] Authenticating user token {{(pid=80245) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:30:47.124667 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad28a719-dadf-4a5e-b93a-4f8cc7eeca30 None None] Validating token access rules against request {{(pid=80245) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:30:47.129789 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc48ae92-b752-4a59-9691-d4248de23d0f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '42d15fc0515945acacbbffcf2356c7a5', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80245) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:30:47.130495 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc48ae92-b752-4a59-9691-d4248de23d0f admin admin] REQUEST_METHOD: `POST` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:30:47.130775 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc48ae92-b752-4a59-9691-d4248de23d0f admin admin] SCRIPT_NAME: `/identity` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:30:47.131051 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc48ae92-b752-4a59-9691-d4248de23d0f admin admin] PATH_INFO: `/v3/projects` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:30:47.131785 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc48ae92-b752-4a59-9691-d4248de23d0f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:30:47.134097 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc48ae92-b752-4a59-9691-d4248de23d0f admin admin] RBAC: Authorization granted {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:30:47.139945 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 915/1817] 10.222.0.47 () {58 vars in 957 bytes} [Thu Jul 21 08:30:46 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 147 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 21 08:30:47.141327 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62c071fd-c641-4e1c-a656-8c6c2ea91caf None None] REQUEST_METHOD: `POST` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:30:47.141637 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62c071fd-c641-4e1c-a656-8c6c2ea91caf None None] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:30:47.141904 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62c071fd-c641-4e1c-a656-8c6c2ea91caf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:30:47.168483 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.auth.core [None req-62c071fd-c641-4e1c-a656-8c6c2ea91caf None None] MFA Rules not processed for user `ac2b5b99a36442148fe743269054e991`. Rule list: `[]` (Enabled: `True`). {{(pid=80244) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 08:30:47.170522 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.fernet_utils [None req-62c071fd-c641-4e1c-a656-8c6c2ea91caf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80244) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 08:30:47.187406 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: [pid: 80245|app: 0|req: 903/1818] 10.222.0.47 () {60 vars in 1152 bytes} [Thu Jul 21 08:30:47 2022] POST /identity/v3/projects => generated 367 bytes in 80 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 08:30:47.189161 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02696bb4-e553-4207-8b71-ae1f55c8c902 None None] REQUEST_METHOD: `POST` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:30:47.189803 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02696bb4-e553-4207-8b71-ae1f55c8c902 None None] SCRIPT_NAME: `/identity` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:30:47.190277 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02696bb4-e553-4207-8b71-ae1f55c8c902 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:30:47.210684 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.auth.core [None req-02696bb4-e553-4207-8b71-ae1f55c8c902 None None] MFA Rules not processed for user `2dce40b8d74c41ec94cc36e6ff05d412`. Rule list: `[]` (Enabled: `True`). {{(pid=80245) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 08:30:47.211730 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.fernet_utils [None req-02696bb4-e553-4207-8b71-ae1f55c8c902 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80245) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 08:30:47.274515 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 916/1819] 10.222.0.47 () {58 vars in 1018 bytes} [Thu Jul 21 08:30:47 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 134 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 08:30:47.275775 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62c071fd-c641-4e1c-a656-8c6c2ea91caf None None] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:30:47.287620 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62c071fd-c641-4e1c-a656-8c6c2ea91caf None None] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:30:47.309927 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG dbcounter [-] [80244] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80244) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:30:47.313211 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: [pid: 80245|app: 0|req: 904/1820] 10.222.0.47 () {58 vars in 957 bytes} [Thu Jul 21 08:30:47 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 08:30:47.314051 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02696bb4-e553-4207-8b71-ae1f55c8c902 None None] Authenticating user token {{(pid=80245) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:30:47.315524 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.fernet_utils [None req-02696bb4-e553-4207-8b71-ae1f55c8c902 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80245) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 08:30:47.373428 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-355e31d3-b735-4988-851b-dee61a643848 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '42d15fc0515945acacbbffcf2356c7a5', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:30:47.374535 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-355e31d3-b735-4988-851b-dee61a643848 admin admin] REQUEST_METHOD: `POST` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:30:47.374801 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-355e31d3-b735-4988-851b-dee61a643848 admin admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:30:47.375082 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-355e31d3-b735-4988-851b-dee61a643848 admin admin] PATH_INFO: `/v3/users` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:30:47.375856 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-355e31d3-b735-4988-851b-dee61a643848 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:30:47.377842 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-355e31d3-b735-4988-851b-dee61a643848 admin admin] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:30:47.400363 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 917/1821] 10.222.0.47 () {60 vars in 1143 bytes} [Thu Jul 21 08:30:47 2022] POST /identity/v3/users => generated 284 bytes in 125 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 08:30:47.401208 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-355e31d3-b735-4988-851b-dee61a643848 admin admin] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:30:47.410134 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02696bb4-e553-4207-8b71-ae1f55c8c902 None None] Validating token access rules against request {{(pid=80245) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:30:47.414449 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-355e31d3-b735-4988-851b-dee61a643848 admin admin] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:30:47.417911 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd1f0541-b34c-4b6c-8ba5-a43a36ca8f37 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2b5b99a36442148fe743269054e991', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4faef13aa50e4e36b6c265fb1398fa14', 'project_domain_id': 'default', 'roles': ['member', 'service', 'ResellerAdmin', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80245) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:30:47.418652 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd1f0541-b34c-4b6c-8ba5-a43a36ca8f37 service nova] REQUEST_METHOD: `GET` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:30:47.419004 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd1f0541-b34c-4b6c-8ba5-a43a36ca8f37 service nova] SCRIPT_NAME: `/identity` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:30:47.419369 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd1f0541-b34c-4b6c-8ba5-a43a36ca8f37 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:30:47.420391 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-862a7282-354b-4a03-afff-7fc7f862da5f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '42d15fc0515945acacbbffcf2356c7a5', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:30:47.421272 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-862a7282-354b-4a03-afff-7fc7f862da5f admin admin] REQUEST_METHOD: `GET` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:30:47.421475 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-862a7282-354b-4a03-afff-7fc7f862da5f admin admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:30:47.421758 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-862a7282-354b-4a03-afff-7fc7f862da5f admin admin] PATH_INFO: `/v3/domains` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:30:47.422503 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-862a7282-354b-4a03-afff-7fc7f862da5f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80244) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 08:30:47.422817 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-862a7282-354b-4a03-afff-7fc7f862da5f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:30:47.424481 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-862a7282-354b-4a03-afff-7fc7f862da5f admin admin] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:30:47.430970 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd1f0541-b34c-4b6c-8ba5-a43a36ca8f37 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:30:47.432754 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd1f0541-b34c-4b6c-8ba5-a43a36ca8f37 service nova] RBAC: Authorization granted {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:30:47.444502 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 918/1822] 10.222.0.47 () {58 vars in 1152 bytes} [Thu Jul 21 08:30:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 08:30:47.445439 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: [pid: 80245|app: 0|req: 905/1823] 10.222.0.47 () {60 vars in 1365 bytes} [Thu Jul 21 08:30:47 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 132 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 08:30:47.448884 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd1f0541-b34c-4b6c-8ba5-a43a36ca8f37 service nova] Authenticating user token {{(pid=80245) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:30:47.455794 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-862a7282-354b-4a03-afff-7fc7f862da5f admin admin] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:30:47.460748 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd1f0541-b34c-4b6c-8ba5-a43a36ca8f37 service nova] Validating token access rules against request {{(pid=80245) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:30:47.466581 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e57fd05e-cba6-4dad-903f-9d044e7dbe28 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '42d15fc0515945acacbbffcf2356c7a5', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80245) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:30:47.467264 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e57fd05e-cba6-4dad-903f-9d044e7dbe28 admin admin] REQUEST_METHOD: `POST` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:30:47.467440 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-862a7282-354b-4a03-afff-7fc7f862da5f admin admin] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:30:47.467749 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e57fd05e-cba6-4dad-903f-9d044e7dbe28 admin admin] SCRIPT_NAME: `/identity` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:30:47.467874 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e57fd05e-cba6-4dad-903f-9d044e7dbe28 admin admin] PATH_INFO: `/v3/projects` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:30:47.468902 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e57fd05e-cba6-4dad-903f-9d044e7dbe28 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:30:47.471582 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e57fd05e-cba6-4dad-903f-9d044e7dbe28 admin admin] RBAC: Authorization granted {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:30:47.490101 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2b761d3-f5c0-4bcd-b171-733861bb0fe4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '42d15fc0515945acacbbffcf2356c7a5', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:30:47.490964 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2b761d3-f5c0-4bcd-b171-733861bb0fe4 admin admin] REQUEST_METHOD: `GET` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:30:47.491252 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2b761d3-f5c0-4bcd-b171-733861bb0fe4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:30:47.491579 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2b761d3-f5c0-4bcd-b171-733861bb0fe4 admin admin] PATH_INFO: `/v3/roles` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:30:47.492320 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2b761d3-f5c0-4bcd-b171-733861bb0fe4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:30:47.493943 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2b761d3-f5c0-4bcd-b171-733861bb0fe4 admin admin] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:30:47.505556 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 919/1824] 10.222.0.47 () {58 vars in 1121 bytes} [Thu Jul 21 08:30:47 2022] GET /identity/v3/roles => generated 1420 bytes in 60 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 08:30:47.509494 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: [pid: 80245|app: 0|req: 906/1825] 10.222.0.47 () {60 vars in 1152 bytes} [Thu Jul 21 08:30:47 2022] POST /identity/v3/projects => generated 365 bytes in 61 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 21 08:30:47.510618 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2b761d3-f5c0-4bcd-b171-733861bb0fe4 admin admin] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:30:47.516382 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e57fd05e-cba6-4dad-903f-9d044e7dbe28 admin admin] Authenticating user token {{(pid=80245) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:30:47.522442 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2b761d3-f5c0-4bcd-b171-733861bb0fe4 admin admin] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:30:47.555577 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG dbcounter [-] [80245] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80245) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:30:47.591726 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de756ef1-e118-4f2c-8f2e-68fd4fa51c91 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '42d15fc0515945acacbbffcf2356c7a5', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:30:47.592627 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de756ef1-e118-4f2c-8f2e-68fd4fa51c91 admin admin] REQUEST_METHOD: `GET` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:30:47.593003 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de756ef1-e118-4f2c-8f2e-68fd4fa51c91 admin admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:30:47.593379 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de756ef1-e118-4f2c-8f2e-68fd4fa51c91 admin admin] PATH_INFO: `/v3/roles` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:30:47.594212 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de756ef1-e118-4f2c-8f2e-68fd4fa51c91 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:30:47.595944 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de756ef1-e118-4f2c-8f2e-68fd4fa51c91 admin admin] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:30:47.600463 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e57fd05e-cba6-4dad-903f-9d044e7dbe28 admin admin] Validating token access rules against request {{(pid=80245) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:30:47.606155 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6c00b5b-b440-4934-a02a-4f0e8c52facf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '42d15fc0515945acacbbffcf2356c7a5', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80245) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:30:47.606878 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6c00b5b-b440-4934-a02a-4f0e8c52facf admin admin] REQUEST_METHOD: `POST` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:30:47.606998 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 920/1826] 10.222.0.47 () {58 vars in 1121 bytes} [Thu Jul 21 08:30:47 2022] GET /identity/v3/roles => generated 1420 bytes in 97 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 08:30:47.607123 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6c00b5b-b440-4934-a02a-4f0e8c52facf admin admin] SCRIPT_NAME: `/identity` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:30:47.607499 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6c00b5b-b440-4934-a02a-4f0e8c52facf admin admin] PATH_INFO: `/v3/users` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:30:47.608521 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6c00b5b-b440-4934-a02a-4f0e8c52facf admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:30:47.610276 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6c00b5b-b440-4934-a02a-4f0e8c52facf admin admin] RBAC: Authorization granted {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:30:47.611082 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de756ef1-e118-4f2c-8f2e-68fd4fa51c91 admin admin] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:30:47.621129 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de756ef1-e118-4f2c-8f2e-68fd4fa51c91 admin admin] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:30:47.628075 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec211ab9-0823-4e7e-8089-dd2c527fcfb2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '42d15fc0515945acacbbffcf2356c7a5', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:30:47.628976 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec211ab9-0823-4e7e-8089-dd2c527fcfb2 admin admin] REQUEST_METHOD: `PUT` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:30:47.629504 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec211ab9-0823-4e7e-8089-dd2c527fcfb2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:30:47.629782 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec211ab9-0823-4e7e-8089-dd2c527fcfb2 admin admin] PATH_INFO: `/v3/projects/bd6dd7e116d24597860060d16a21ff26/users/7f6f75688c704dacb841845cab2e608f/roles/3a074738221e4601b5205715ba87d240` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:30:47.639460 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: [pid: 80245|app: 0|req: 907/1827] 10.222.0.47 () {60 vars in 1143 bytes} [Thu Jul 21 08:30:47 2022] POST /identity/v3/users => generated 283 bytes in 120 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 08:30:47.640519 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6c00b5b-b440-4934-a02a-4f0e8c52facf admin admin] Authenticating user token {{(pid=80245) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:30:47.647397 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec211ab9-0823-4e7e-8089-dd2c527fcfb2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=bd6dd7e116d24597860060d16a21ff26, user_id=7f6f75688c704dacb841845cab2e608f, role_id=3a074738221e4601b5205715ba87d240)` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:30:47.650755 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec211ab9-0823-4e7e-8089-dd2c527fcfb2 admin admin] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:30:47.656440 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6c00b5b-b440-4934-a02a-4f0e8c52facf admin admin] Validating token access rules against request {{(pid=80245) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:30:47.661729 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47ed21d5-e2f2-43fd-9769-c094d20df545 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '42d15fc0515945acacbbffcf2356c7a5', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80245) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:30:47.662421 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47ed21d5-e2f2-43fd-9769-c094d20df545 admin admin] REQUEST_METHOD: `GET` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:30:47.662714 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47ed21d5-e2f2-43fd-9769-c094d20df545 admin admin] SCRIPT_NAME: `/identity` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:30:47.662993 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47ed21d5-e2f2-43fd-9769-c094d20df545 admin admin] PATH_INFO: `/v3/roles` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:30:47.663767 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47ed21d5-e2f2-43fd-9769-c094d20df545 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:30:47.665380 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47ed21d5-e2f2-43fd-9769-c094d20df545 admin admin] RBAC: Authorization granted {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:30:47.666099 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 921/1828] 10.222.0.47 () {60 vars in 1482 bytes} [Thu Jul 21 08:30:47 2022] PUT /identity/v3/projects/bd6dd7e116d24597860060d16a21ff26/users/7f6f75688c704dacb841845cab2e608f/roles/3a074738221e4601b5205715ba87d240 => generated 0 bytes in 55 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 08:30:47.679544 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: [pid: 80245|app: 0|req: 908/1829] 10.222.0.47 () {58 vars in 1121 bytes} [Thu Jul 21 08:30:47 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 08:30:47.679787 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-277367e2-cf27-48e4-b69b-e5c2c1f21505 None None] REQUEST_METHOD: `POST` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:30:47.679787 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-277367e2-cf27-48e4-b69b-e5c2c1f21505 None None] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:30:47.679787 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-277367e2-cf27-48e4-b69b-e5c2c1f21505 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:30:47.681312 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47ed21d5-e2f2-43fd-9769-c094d20df545 admin admin] Authenticating user token {{(pid=80245) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:30:47.692375 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47ed21d5-e2f2-43fd-9769-c094d20df545 admin admin] Validating token access rules against request {{(pid=80245) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:30:47.705446 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.auth.core [None req-277367e2-cf27-48e4-b69b-e5c2c1f21505 None None] MFA Rules not processed for user `7f6f75688c704dacb841845cab2e608f`. Rule list: `[]` (Enabled: `True`). {{(pid=80244) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 08:30:47.706655 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.fernet_utils [None req-277367e2-cf27-48e4-b69b-e5c2c1f21505 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80244) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 08:30:47.775114 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a342379a-0196-4cd3-b590-e155012e8a36 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '42d15fc0515945acacbbffcf2356c7a5', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80245) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:30:47.775879 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a342379a-0196-4cd3-b590-e155012e8a36 admin admin] REQUEST_METHOD: `GET` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:30:47.776262 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a342379a-0196-4cd3-b590-e155012e8a36 admin admin] SCRIPT_NAME: `/identity` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:30:47.776649 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a342379a-0196-4cd3-b590-e155012e8a36 admin admin] PATH_INFO: `/v3/roles` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:30:47.777494 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a342379a-0196-4cd3-b590-e155012e8a36 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:30:47.779442 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a342379a-0196-4cd3-b590-e155012e8a36 admin admin] RBAC: Authorization granted {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:30:47.781672 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG dbcounter [-] [80244] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80244) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:30:47.798814 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: [pid: 80245|app: 0|req: 909/1830] 10.222.0.47 () {58 vars in 1121 bytes} [Thu Jul 21 08:30:47 2022] GET /identity/v3/roles => generated 1420 bytes in 115 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 08:30:47.800394 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a342379a-0196-4cd3-b590-e155012e8a36 admin admin] Authenticating user token {{(pid=80245) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:30:47.812525 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a342379a-0196-4cd3-b590-e155012e8a36 admin admin] Validating token access rules against request {{(pid=80245) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:30:47.813011 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 922/1831] 10.222.0.47 () {58 vars in 957 bytes} [Thu Jul 21 08:30:47 2022] POST /identity/v3/auth/tokens => generated 3234 bytes in 137 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 21 08:30:47.814969 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-277367e2-cf27-48e4-b69b-e5c2c1f21505 None None] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:30:47.817693 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dc296d0-5e0e-43b3-ba67-d219ec58ddbb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '42d15fc0515945acacbbffcf2356c7a5', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80245) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:30:47.818593 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dc296d0-5e0e-43b3-ba67-d219ec58ddbb admin admin] REQUEST_METHOD: `PUT` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:30:47.818876 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dc296d0-5e0e-43b3-ba67-d219ec58ddbb admin admin] SCRIPT_NAME: `/identity` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:30:47.819154 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dc296d0-5e0e-43b3-ba67-d219ec58ddbb admin admin] PATH_INFO: `/v3/projects/6401b0739da443cda479f51a5a87e522/users/ce63a95cc2aa4ff6844cf0a1b49c648d/roles/3a074738221e4601b5205715ba87d240` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:30:47.828446 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-277367e2-cf27-48e4-b69b-e5c2c1f21505 None None] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:30:47.834772 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a62292fa-8516-48c8-afdc-e5754463aa5c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '42d15fc0515945acacbbffcf2356c7a5', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:30:47.835168 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a62292fa-8516-48c8-afdc-e5754463aa5c admin admin] REQUEST_METHOD: `POST` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:30:47.835504 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a62292fa-8516-48c8-afdc-e5754463aa5c admin admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:30:47.835786 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a62292fa-8516-48c8-afdc-e5754463aa5c admin admin] PATH_INFO: `/v3/projects` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:30:47.836494 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a62292fa-8516-48c8-afdc-e5754463aa5c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:30:47.837186 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dc296d0-5e0e-43b3-ba67-d219ec58ddbb admin admin] RBAC: Authorizing `identity:create_grant(project_id=6401b0739da443cda479f51a5a87e522, user_id=ce63a95cc2aa4ff6844cf0a1b49c648d, role_id=3a074738221e4601b5205715ba87d240)` {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:30:47.838427 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a62292fa-8516-48c8-afdc-e5754463aa5c admin admin] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:30:47.839883 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dc296d0-5e0e-43b3-ba67-d219ec58ddbb admin admin] RBAC: Authorization granted {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:30:47.859420 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: [pid: 80245|app: 0|req: 910/1832] 10.222.0.47 () {60 vars in 1482 bytes} [Thu Jul 21 08:30:47 2022] PUT /identity/v3/projects/6401b0739da443cda479f51a5a87e522/users/ce63a95cc2aa4ff6844cf0a1b49c648d/roles/3a074738221e4601b5205715ba87d240 => generated 0 bytes in 56 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 08:30:47.867121 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57bde712-0322-427b-9385-777a9a360585 None None] REQUEST_METHOD: `POST` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:30:47.868474 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57bde712-0322-427b-9385-777a9a360585 None None] SCRIPT_NAME: `/identity` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:30:47.868765 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57bde712-0322-427b-9385-777a9a360585 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:30:47.880386 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 923/1833] 10.222.0.47 () {60 vars in 1152 bytes} [Thu Jul 21 08:30:47 2022] POST /identity/v3/projects => generated 369 bytes in 66 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 08:30:47.884397 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a62292fa-8516-48c8-afdc-e5754463aa5c admin admin] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:30:47.919310 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.auth.core [None req-57bde712-0322-427b-9385-777a9a360585 None None] MFA Rules not processed for user `ce63a95cc2aa4ff6844cf0a1b49c648d`. Rule list: `[]` (Enabled: `True`). {{(pid=80245) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 08:30:47.921199 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.fernet_utils [None req-57bde712-0322-427b-9385-777a9a360585 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80245) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 08:30:47.986029 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a62292fa-8516-48c8-afdc-e5754463aa5c admin admin] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:30:47.990869 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb87dca9-a532-4758-adf9-db0807c89218 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '42d15fc0515945acacbbffcf2356c7a5', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:30:47.991579 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb87dca9-a532-4758-adf9-db0807c89218 admin admin] REQUEST_METHOD: `POST` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:30:47.991941 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb87dca9-a532-4758-adf9-db0807c89218 admin admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:30:47.992146 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb87dca9-a532-4758-adf9-db0807c89218 admin admin] PATH_INFO: `/v3/users` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:30:47.992843 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb87dca9-a532-4758-adf9-db0807c89218 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:30:47.994185 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: [pid: 80245|app: 0|req: 911/1834] 10.222.0.47 () {58 vars in 957 bytes} [Thu Jul 21 08:30:47 2022] POST /identity/v3/auth/tokens => generated 3232 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 08:30:47.994941 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb87dca9-a532-4758-adf9-db0807c89218 admin admin] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:30:47.998427 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57bde712-0322-427b-9385-777a9a360585 None None] Authenticating user token {{(pid=80245) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:30:48.009376 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57bde712-0322-427b-9385-777a9a360585 None None] Validating token access rules against request {{(pid=80245) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:30:48.015027 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a69418c-92ee-436d-a12c-cbe53518bf27 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '42d15fc0515945acacbbffcf2356c7a5', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80245) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:30:48.015715 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a69418c-92ee-436d-a12c-cbe53518bf27 admin admin] REQUEST_METHOD: `POST` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:30:48.016001 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a69418c-92ee-436d-a12c-cbe53518bf27 admin admin] SCRIPT_NAME: `/identity` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:30:48.016104 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 924/1835] 10.222.0.47 () {60 vars in 1143 bytes} [Thu Jul 21 08:30:47 2022] POST /identity/v3/users => generated 291 bytes in 132 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 08:30:48.016571 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a69418c-92ee-436d-a12c-cbe53518bf27 admin admin] PATH_INFO: `/v3/projects` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:30:48.017265 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a69418c-92ee-436d-a12c-cbe53518bf27 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:30:48.019166 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a69418c-92ee-436d-a12c-cbe53518bf27 admin admin] RBAC: Authorization granted {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:30:48.020402 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb87dca9-a532-4758-adf9-db0807c89218 admin admin] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:30:48.030583 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG dbcounter [-] [80245] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80245) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:30:48.030940 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb87dca9-a532-4758-adf9-db0807c89218 admin admin] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:30:48.036249 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0af1ae63-b326-4f93-9f48-567149f03b02 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '42d15fc0515945acacbbffcf2356c7a5', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:30:48.037068 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0af1ae63-b326-4f93-9f48-567149f03b02 admin admin] REQUEST_METHOD: `GET` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:30:48.037313 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0af1ae63-b326-4f93-9f48-567149f03b02 admin admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:30:48.037637 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0af1ae63-b326-4f93-9f48-567149f03b02 admin admin] PATH_INFO: `/v3/roles` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:30:48.038528 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0af1ae63-b326-4f93-9f48-567149f03b02 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:30:48.040399 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0af1ae63-b326-4f93-9f48-567149f03b02 admin admin] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:30:48.048083 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: [pid: 80245|app: 0|req: 912/1836] 10.222.0.47 () {60 vars in 1152 bytes} [Thu Jul 21 08:30:47 2022] POST /identity/v3/projects => generated 365 bytes in 50 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 21 08:30:48.050935 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 925/1837] 10.222.0.47 () {58 vars in 1121 bytes} [Thu Jul 21 08:30:48 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 21 08:30:48.052240 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a69418c-92ee-436d-a12c-cbe53518bf27 admin admin] Authenticating user token {{(pid=80245) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:30:48.056889 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0af1ae63-b326-4f93-9f48-567149f03b02 admin admin] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:30:48.066954 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0af1ae63-b326-4f93-9f48-567149f03b02 admin admin] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:30:48.116843 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a69418c-92ee-436d-a12c-cbe53518bf27 admin admin] Validating token access rules against request {{(pid=80245) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:30:48.121736 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2c76ce8-3c44-4af0-97d9-d76ba4b5b30c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '42d15fc0515945acacbbffcf2356c7a5', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80245) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:30:48.122415 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2c76ce8-3c44-4af0-97d9-d76ba4b5b30c admin admin] REQUEST_METHOD: `POST` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:30:48.122707 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2c76ce8-3c44-4af0-97d9-d76ba4b5b30c admin admin] SCRIPT_NAME: `/identity` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:30:48.122984 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2c76ce8-3c44-4af0-97d9-d76ba4b5b30c admin admin] PATH_INFO: `/v3/users` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:30:48.123719 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2c76ce8-3c44-4af0-97d9-d76ba4b5b30c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:30:48.124891 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aff30d6d-94a9-45f7-adad-aa8a4f7577cd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '42d15fc0515945acacbbffcf2356c7a5', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:30:48.125644 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2c76ce8-3c44-4af0-97d9-d76ba4b5b30c admin admin] RBAC: Authorization granted {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:30:48.125807 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aff30d6d-94a9-45f7-adad-aa8a4f7577cd admin admin] REQUEST_METHOD: `PUT` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:30:48.126088 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aff30d6d-94a9-45f7-adad-aa8a4f7577cd admin admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:30:48.126364 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aff30d6d-94a9-45f7-adad-aa8a4f7577cd admin admin] PATH_INFO: `/v3/projects/5c0104061b6a49aca5799d0caf5530bd/users/8ce9f8b8342b40429f17c7e7becf6208/roles/e29870f6e9e2411a9db0d5bf45f32ca5` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:30:48.143128 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aff30d6d-94a9-45f7-adad-aa8a4f7577cd admin admin] RBAC: Authorizing `identity:create_grant(project_id=5c0104061b6a49aca5799d0caf5530bd, user_id=8ce9f8b8342b40429f17c7e7becf6208, role_id=e29870f6e9e2411a9db0d5bf45f32ca5)` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:30:48.145825 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aff30d6d-94a9-45f7-adad-aa8a4f7577cd admin admin] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:30:48.149912 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: [pid: 80245|app: 0|req: 913/1838] 10.222.0.47 () {60 vars in 1143 bytes} [Thu Jul 21 08:30:48 2022] POST /identity/v3/users => generated 289 bytes in 95 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 21 08:30:48.150972 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2c76ce8-3c44-4af0-97d9-d76ba4b5b30c admin admin] Authenticating user token {{(pid=80245) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:30:48.162103 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2c76ce8-3c44-4af0-97d9-d76ba4b5b30c admin admin] Validating token access rules against request {{(pid=80245) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:30:48.163258 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 926/1839] 10.222.0.47 () {60 vars in 1482 bytes} [Thu Jul 21 08:30:48 2022] PUT /identity/v3/projects/5c0104061b6a49aca5799d0caf5530bd/users/8ce9f8b8342b40429f17c7e7becf6208/roles/e29870f6e9e2411a9db0d5bf45f32ca5 => generated 0 bytes in 106 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 21 08:30:48.167176 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aff30d6d-94a9-45f7-adad-aa8a4f7577cd admin admin] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:30:48.199615 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG dbcounter [-] [80244] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80244) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:30:48.223990 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6a45d31-896b-4e68-92b2-cf963a60b3a7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '42d15fc0515945acacbbffcf2356c7a5', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80245) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:30:48.225291 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6a45d31-896b-4e68-92b2-cf963a60b3a7 admin admin] REQUEST_METHOD: `GET` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:30:48.225575 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6a45d31-896b-4e68-92b2-cf963a60b3a7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:30:48.225854 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6a45d31-896b-4e68-92b2-cf963a60b3a7 admin admin] PATH_INFO: `/v3/roles` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:30:48.226593 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6a45d31-896b-4e68-92b2-cf963a60b3a7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:30:48.228249 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6a45d31-896b-4e68-92b2-cf963a60b3a7 admin admin] RBAC: Authorization granted {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:30:48.235163 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aff30d6d-94a9-45f7-adad-aa8a4f7577cd admin admin] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:30:48.239193 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: [pid: 80245|app: 0|req: 914/1840] 10.222.0.47 () {58 vars in 1121 bytes} [Thu Jul 21 08:30:48 2022] GET /identity/v3/roles => generated 1420 bytes in 88 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 21 08:30:48.241198 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-371de084-7f53-4a31-b1de-ccd344aa6283 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '42d15fc0515945acacbbffcf2356c7a5', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:30:48.241833 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-371de084-7f53-4a31-b1de-ccd344aa6283 admin admin] REQUEST_METHOD: `GET` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:30:48.242232 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-371de084-7f53-4a31-b1de-ccd344aa6283 admin admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:30:48.242389 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-371de084-7f53-4a31-b1de-ccd344aa6283 admin admin] PATH_INFO: `/v3/roles` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:30:48.243122 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-371de084-7f53-4a31-b1de-ccd344aa6283 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:30:48.243814 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6a45d31-896b-4e68-92b2-cf963a60b3a7 admin admin] Authenticating user token {{(pid=80245) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:30:48.245092 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-371de084-7f53-4a31-b1de-ccd344aa6283 admin admin] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:30:48.253704 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6a45d31-896b-4e68-92b2-cf963a60b3a7 admin admin] Validating token access rules against request {{(pid=80245) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:30:48.258973 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 927/1841] 10.222.0.47 () {58 vars in 1121 bytes} [Thu Jul 21 08:30:48 2022] GET /identity/v3/roles => generated 1420 bytes in 89 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 21 08:30:48.259100 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c522b92-2ca4-4ece-b116-e6affa3ed760 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '42d15fc0515945acacbbffcf2356c7a5', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80245) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:30:48.259967 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c522b92-2ca4-4ece-b116-e6affa3ed760 admin admin] REQUEST_METHOD: `PUT` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:30:48.260071 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-371de084-7f53-4a31-b1de-ccd344aa6283 admin admin] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:30:48.260253 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c522b92-2ca4-4ece-b116-e6affa3ed760 admin admin] SCRIPT_NAME: `/identity` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:30:48.260561 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c522b92-2ca4-4ece-b116-e6affa3ed760 admin admin] PATH_INFO: `/v3/projects/fa64e380f53a4d999af86be40d4fa83e/users/2311fea5afca4496a5f2bcdb3cb7d6d5/roles/e29870f6e9e2411a9db0d5bf45f32ca5` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:30:48.270485 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-371de084-7f53-4a31-b1de-ccd344aa6283 admin admin] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:30:48.275931 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1ee15e1-64bf-4dd2-865b-8ea748cae75e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '42d15fc0515945acacbbffcf2356c7a5', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:30:48.276834 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1ee15e1-64bf-4dd2-865b-8ea748cae75e admin admin] REQUEST_METHOD: `PUT` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:30:48.277117 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1ee15e1-64bf-4dd2-865b-8ea748cae75e admin admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:30:48.277395 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1ee15e1-64bf-4dd2-865b-8ea748cae75e admin admin] PATH_INFO: `/v3/projects/5c0104061b6a49aca5799d0caf5530bd/users/8ce9f8b8342b40429f17c7e7becf6208/roles/e29870f6e9e2411a9db0d5bf45f32ca5` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:30:48.277609 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c522b92-2ca4-4ece-b116-e6affa3ed760 admin admin] RBAC: Authorizing `identity:create_grant(project_id=fa64e380f53a4d999af86be40d4fa83e, user_id=2311fea5afca4496a5f2bcdb3cb7d6d5, role_id=e29870f6e9e2411a9db0d5bf45f32ca5)` {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:30:48.280004 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1ee15e1-64bf-4dd2-865b-8ea748cae75e admin admin] RBAC: Authorizing `identity:create_grant(project_id=5c0104061b6a49aca5799d0caf5530bd, user_id=8ce9f8b8342b40429f17c7e7becf6208, role_id=e29870f6e9e2411a9db0d5bf45f32ca5)` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:30:48.280979 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c522b92-2ca4-4ece-b116-e6affa3ed760 admin admin] RBAC: Authorization granted {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:30:48.282864 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1ee15e1-64bf-4dd2-865b-8ea748cae75e admin admin] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:30:48.294046 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 928/1842] 10.222.0.47 () {60 vars in 1482 bytes} [Thu Jul 21 08:30:48 2022] PUT /identity/v3/projects/5c0104061b6a49aca5799d0caf5530bd/users/8ce9f8b8342b40429f17c7e7becf6208/roles/e29870f6e9e2411a9db0d5bf45f32ca5 => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 21 08:30:48.303570 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: [pid: 80245|app: 0|req: 915/1843] 10.222.0.47 () {60 vars in 1482 bytes} [Thu Jul 21 08:30:48 2022] PUT /identity/v3/projects/fa64e380f53a4d999af86be40d4fa83e/users/2311fea5afca4496a5f2bcdb3cb7d6d5/roles/e29870f6e9e2411a9db0d5bf45f32ca5 => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 21 08:30:48.303691 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1ee15e1-64bf-4dd2-865b-8ea748cae75e admin admin] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:30:48.305578 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb09c06d-6169-4cc0-8260-48514f8beb14 None None] REQUEST_METHOD: `POST` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:30:48.305881 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb09c06d-6169-4cc0-8260-48514f8beb14 None None] SCRIPT_NAME: `/identity` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:30:48.306159 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb09c06d-6169-4cc0-8260-48514f8beb14 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:30:48.334745 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.auth.core [None req-eb09c06d-6169-4cc0-8260-48514f8beb14 None None] MFA Rules not processed for user `8ce9f8b8342b40429f17c7e7becf6208`. Rule list: `[]` (Enabled: `True`). {{(pid=80245) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 08:30:48.336352 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.fernet_utils [None req-eb09c06d-6169-4cc0-8260-48514f8beb14 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80245) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 08:30:48.378201 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1ee15e1-64bf-4dd2-865b-8ea748cae75e admin admin] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:30:48.384915 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG dbcounter [-] [80245] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80245) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:30:48.421235 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: [pid: 80245|app: 0|req: 916/1844] 10.222.0.47 () {58 vars in 957 bytes} [Thu Jul 21 08:30:48 2022] POST /identity/v3/auth/tokens => generated 3303 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 21 08:30:48.432530 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb09c06d-6169-4cc0-8260-48514f8beb14 None None] Authenticating user token {{(pid=80245) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:30:48.434151 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.fernet_utils [None req-eb09c06d-6169-4cc0-8260-48514f8beb14 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80245) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 08:30:48.452268 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c21260d3-6f3f-485f-870b-b5fd46fed278 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '42d15fc0515945acacbbffcf2356c7a5', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:30:48.452955 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c21260d3-6f3f-485f-870b-b5fd46fed278 admin admin] REQUEST_METHOD: `GET` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:30:48.453248 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c21260d3-6f3f-485f-870b-b5fd46fed278 admin admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:30:48.453527 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c21260d3-6f3f-485f-870b-b5fd46fed278 admin admin] PATH_INFO: `/v3/roles` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:30:48.454286 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c21260d3-6f3f-485f-870b-b5fd46fed278 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:30:48.455978 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c21260d3-6f3f-485f-870b-b5fd46fed278 admin admin] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:30:48.479461 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 929/1845] 10.222.0.47 () {58 vars in 1121 bytes} [Thu Jul 21 08:30:48 2022] GET /identity/v3/roles => generated 1420 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 08:30:48.481519 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c21260d3-6f3f-485f-870b-b5fd46fed278 admin admin] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:30:48.496416 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c21260d3-6f3f-485f-870b-b5fd46fed278 admin admin] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:30:48.501820 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aedb9898-8cab-4142-9503-e1f0c02add9f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '42d15fc0515945acacbbffcf2356c7a5', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:30:48.502722 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aedb9898-8cab-4142-9503-e1f0c02add9f admin admin] REQUEST_METHOD: `PUT` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:30:48.503010 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aedb9898-8cab-4142-9503-e1f0c02add9f admin admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:30:48.503292 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aedb9898-8cab-4142-9503-e1f0c02add9f admin admin] PATH_INFO: `/v3/projects/fa64e380f53a4d999af86be40d4fa83e/users/2311fea5afca4496a5f2bcdb3cb7d6d5/roles/e29870f6e9e2411a9db0d5bf45f32ca5` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:30:48.505710 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aedb9898-8cab-4142-9503-e1f0c02add9f admin admin] RBAC: Authorizing `identity:create_grant(project_id=fa64e380f53a4d999af86be40d4fa83e, user_id=2311fea5afca4496a5f2bcdb3cb7d6d5, role_id=e29870f6e9e2411a9db0d5bf45f32ca5)` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:30:48.508427 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aedb9898-8cab-4142-9503-e1f0c02add9f admin admin] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:30:48.508576 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb09c06d-6169-4cc0-8260-48514f8beb14 None None] Validating token access rules against request {{(pid=80245) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:30:48.514273 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2e81c55-5236-454e-9806-110aada587db service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4f40410b7674687ab5362ac982248ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4faef13aa50e4e36b6c265fb1398fa14', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80245) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:30:48.515487 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2e81c55-5236-454e-9806-110aada587db service cinder] REQUEST_METHOD: `GET` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:30:48.515710 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2e81c55-5236-454e-9806-110aada587db service cinder] SCRIPT_NAME: `/identity` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:30:48.516194 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2e81c55-5236-454e-9806-110aada587db service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:30:48.520352 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 930/1846] 10.222.0.47 () {60 vars in 1482 bytes} [Thu Jul 21 08:30:48 2022] PUT /identity/v3/projects/fa64e380f53a4d999af86be40d4fa83e/users/2311fea5afca4496a5f2bcdb3cb7d6d5/roles/e29870f6e9e2411a9db0d5bf45f32ca5 => generated 0 bytes in 41 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 21 08:30:48.528679 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2e81c55-5236-454e-9806-110aada587db service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:30:48.530249 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2e81c55-5236-454e-9806-110aada587db service cinder] RBAC: Authorization granted {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:30:48.531139 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a1e732b-91d5-44ba-b64a-4807c086f1ba None None] REQUEST_METHOD: `POST` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:30:48.531470 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a1e732b-91d5-44ba-b64a-4807c086f1ba None None] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:30:48.531752 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a1e732b-91d5-44ba-b64a-4807c086f1ba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:30:48.543810 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: [pid: 80245|app: 0|req: 917/1847] 10.222.0.47 () {60 vars in 1365 bytes} [Thu Jul 21 08:30:48 2022] GET /identity/v3/auth/tokens => generated 3303 bytes in 112 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 21 08:30:48.560862 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.auth.core [None req-0a1e732b-91d5-44ba-b64a-4807c086f1ba None None] MFA Rules not processed for user `2311fea5afca4496a5f2bcdb3cb7d6d5`. Rule list: `[]` (Enabled: `True`). {{(pid=80244) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 08:30:48.562181 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.fernet_utils [None req-0a1e732b-91d5-44ba-b64a-4807c086f1ba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80244) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 08:30:48.590644 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2e81c55-5236-454e-9806-110aada587db service cinder] Authenticating user token {{(pid=80245) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:30:48.592032 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.fernet_utils [None req-a2e81c55-5236-454e-9806-110aada587db service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80245) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 08:30:48.596809 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG dbcounter [-] [80244] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80244) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:30:48.603747 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2e81c55-5236-454e-9806-110aada587db service cinder] Validating token access rules against request {{(pid=80245) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:30:48.637649 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 931/1848] 10.222.0.47 () {58 vars in 957 bytes} [Thu Jul 21 08:30:48 2022] POST /identity/v3/auth/tokens => generated 3299 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 21 08:30:48.649726 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a1e732b-91d5-44ba-b64a-4807c086f1ba None None] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:30:48.659719 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54d9be9a-c02e-48b5-b604-eb81b4e0c599 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4f40410b7674687ab5362ac982248ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4faef13aa50e4e36b6c265fb1398fa14', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80245) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:30:48.660384 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54d9be9a-c02e-48b5-b604-eb81b4e0c599 service cinder] REQUEST_METHOD: `GET` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:30:48.660753 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54d9be9a-c02e-48b5-b604-eb81b4e0c599 service cinder] SCRIPT_NAME: `/identity` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:30:48.660926 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54d9be9a-c02e-48b5-b604-eb81b4e0c599 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:30:48.707236 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a1e732b-91d5-44ba-b64a-4807c086f1ba None None] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:30:48.710859 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bdeceaa-df41-4af5-8614-7e712980c9ca service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4f40410b7674687ab5362ac982248ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4faef13aa50e4e36b6c265fb1398fa14', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:30:48.711539 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bdeceaa-df41-4af5-8614-7e712980c9ca service cinder] REQUEST_METHOD: `GET` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:30:48.711828 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bdeceaa-df41-4af5-8614-7e712980c9ca service cinder] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:30:48.712108 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bdeceaa-df41-4af5-8614-7e712980c9ca service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:30:48.719561 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54d9be9a-c02e-48b5-b604-eb81b4e0c599 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:30:48.721007 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54d9be9a-c02e-48b5-b604-eb81b4e0c599 service cinder] RBAC: Authorization granted {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:30:48.721251 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bdeceaa-df41-4af5-8614-7e712980c9ca service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:30:48.722683 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bdeceaa-df41-4af5-8614-7e712980c9ca service cinder] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:30:48.729997 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: [pid: 80245|app: 0|req: 918/1849] 10.222.0.47 () {60 vars in 1365 bytes} [Thu Jul 21 08:30:48 2022] GET /identity/v3/auth/tokens => generated 3234 bytes in 140 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 21 08:30:48.733346 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 932/1850] 10.222.0.47 () {60 vars in 1365 bytes} [Thu Jul 21 08:30:48 2022] GET /identity/v3/auth/tokens => generated 3299 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 21 08:30:48.809917 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54d9be9a-c02e-48b5-b604-eb81b4e0c599 service cinder] Authenticating user token {{(pid=80245) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:30:48.819675 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54d9be9a-c02e-48b5-b604-eb81b4e0c599 service cinder] Validating token access rules against request {{(pid=80245) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:30:48.823453 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63e7a402-7ec7-43e3-b650-b36e5a8cf3a1 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4f40410b7674687ab5362ac982248ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4faef13aa50e4e36b6c265fb1398fa14', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80245) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:30:48.824108 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63e7a402-7ec7-43e3-b650-b36e5a8cf3a1 service cinder] REQUEST_METHOD: `GET` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:30:48.824390 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63e7a402-7ec7-43e3-b650-b36e5a8cf3a1 service cinder] SCRIPT_NAME: `/identity` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:30:48.824667 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63e7a402-7ec7-43e3-b650-b36e5a8cf3a1 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:30:48.866969 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG dbcounter [-] [80245] Writing DB stats keystone:SELECT=100 {{(pid=80245) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:30:48.898740 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63e7a402-7ec7-43e3-b650-b36e5a8cf3a1 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:30:48.900343 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63e7a402-7ec7-43e3-b650-b36e5a8cf3a1 service cinder] RBAC: Authorization granted {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:30:48.910510 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: [pid: 80245|app: 0|req: 919/1851] 10.222.0.47 () {60 vars in 1365 bytes} [Thu Jul 21 08:30:48 2022] GET /identity/v3/auth/tokens => generated 3232 bytes in 101 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 08:30:49.554368 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bdeceaa-df41-4af5-8614-7e712980c9ca service cinder] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:30:49.564209 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bdeceaa-df41-4af5-8614-7e712980c9ca service cinder] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:30:49.568199 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6a3f6c9-c6a4-460a-88e2-d995b9d26a00 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4f40410b7674687ab5362ac982248ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4faef13aa50e4e36b6c265fb1398fa14', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:30:49.568867 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6a3f6c9-c6a4-460a-88e2-d995b9d26a00 service cinder] REQUEST_METHOD: `GET` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:30:49.569284 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6a3f6c9-c6a4-460a-88e2-d995b9d26a00 service cinder] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:30:49.569436 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6a3f6c9-c6a4-460a-88e2-d995b9d26a00 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:30:49.571302 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.fernet_utils [None req-f6a3f6c9-c6a4-460a-88e2-d995b9d26a00 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80244) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 08:30:49.654193 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6a3f6c9-c6a4-460a-88e2-d995b9d26a00 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:30:49.656043 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6a3f6c9-c6a4-460a-88e2-d995b9d26a00 service cinder] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:30:49.671029 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 933/1852] 10.222.0.47 () {60 vars in 1365 bytes} [Thu Jul 21 08:30:49 2022] GET /identity/v3/auth/tokens => generated 3311 bytes in 117 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 08:30:52.591716 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63e7a402-7ec7-43e3-b650-b36e5a8cf3a1 service cinder] Authenticating user token {{(pid=80245) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:30:52.670691 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63e7a402-7ec7-43e3-b650-b36e5a8cf3a1 service cinder] Validating token access rules against request {{(pid=80245) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:30:52.677524 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5266953e-793d-40ee-b69c-18b4e0d463b9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '42d15fc0515945acacbbffcf2356c7a5', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80245) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:30:52.678983 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5266953e-793d-40ee-b69c-18b4e0d463b9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:30:52.679462 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5266953e-793d-40ee-b69c-18b4e0d463b9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:30:52.679902 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5266953e-793d-40ee-b69c-18b4e0d463b9 admin admin] PATH_INFO: `/v3/users/7f6f75688c704dacb841845cab2e608f` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:30:52.681293 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5266953e-793d-40ee-b69c-18b4e0d463b9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=7f6f75688c704dacb841845cab2e608f)` {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:30:52.683493 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5266953e-793d-40ee-b69c-18b4e0d463b9 admin admin] RBAC: Authorization granted {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:30:52.741875 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.notifications [None req-5266953e-793d-40ee-b69c-18b4e0d463b9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7f6f75688c704dacb841845cab2e608f'} {{(pid=80245) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 08:30:52.750437 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.notifications [None req-5266953e-793d-40ee-b69c-18b4e0d463b9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7f6f75688c704dacb841845cab2e608f'} {{(pid=80245) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 08:30:52.764402 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.notifications [None req-5266953e-793d-40ee-b69c-18b4e0d463b9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7f6f75688c704dacb841845cab2e608f'} {{(pid=80245) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 08:30:52.771775 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.notifications [None req-5266953e-793d-40ee-b69c-18b4e0d463b9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7f6f75688c704dacb841845cab2e608f'} {{(pid=80245) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 08:30:52.772513 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.notifications [None req-5266953e-793d-40ee-b69c-18b4e0d463b9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7f6f75688c704dacb841845cab2e608f'} {{(pid=80245) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 08:30:52.778509 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: [pid: 80245|app: 0|req: 920/1853] 10.222.0.47 () {58 vars in 1223 bytes} [Thu Jul 21 08:30:52 2022] DELETE /identity/v3/users/7f6f75688c704dacb841845cab2e608f => generated 0 bytes in 187 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 21 08:30:52.783650 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8ef4af7-3172-4669-888c-e791fa6e70e8 None None] REQUEST_METHOD: `POST` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:30:52.784139 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8ef4af7-3172-4669-888c-e791fa6e70e8 None None] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:30:52.784212 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8ef4af7-3172-4669-888c-e791fa6e70e8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:30:52.802005 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.auth.core [None req-b8ef4af7-3172-4669-888c-e791fa6e70e8 None None] MFA Rules not processed for user `2dce40b8d74c41ec94cc36e6ff05d412`. Rule list: `[]` (Enabled: `True`). {{(pid=80244) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 08:30:52.803201 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.fernet_utils [None req-b8ef4af7-3172-4669-888c-e791fa6e70e8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80244) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 08:30:52.861226 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 934/1854] 10.222.0.47 () {58 vars in 957 bytes} [Thu Jul 21 08:30:52 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 08:30:52.871022 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5266953e-793d-40ee-b69c-18b4e0d463b9 admin admin] Authenticating user token {{(pid=80245) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:30:52.872604 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.fernet_utils [None req-5266953e-793d-40ee-b69c-18b4e0d463b9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80245) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 08:30:52.942165 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5266953e-793d-40ee-b69c-18b4e0d463b9 admin admin] Validating token access rules against request {{(pid=80245) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:30:52.946496 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acd61b6c-c6ff-4446-9eda-b57ae4abe6ae service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20ca3c3bd3774691b3374cfbfec6e494', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4faef13aa50e4e36b6c265fb1398fa14', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80245) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:30:52.947567 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acd61b6c-c6ff-4446-9eda-b57ae4abe6ae service neutron] REQUEST_METHOD: `GET` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:30:52.947753 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acd61b6c-c6ff-4446-9eda-b57ae4abe6ae service neutron] SCRIPT_NAME: `/identity` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:30:52.948065 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acd61b6c-c6ff-4446-9eda-b57ae4abe6ae service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:30:52.956786 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG dbcounter [-] [80245] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80245) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:30:52.960081 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acd61b6c-c6ff-4446-9eda-b57ae4abe6ae service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:30:52.962121 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acd61b6c-c6ff-4446-9eda-b57ae4abe6ae service neutron] RBAC: Authorization granted {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:30:52.976607 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: [pid: 80245|app: 0|req: 921/1855] 10.222.0.47 () {60 vars in 1365 bytes} [Thu Jul 21 08:30:52 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 106 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 21 08:30:53.381727 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8ef4af7-3172-4669-888c-e791fa6e70e8 None None] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:30:53.384147 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.fernet_utils [None req-b8ef4af7-3172-4669-888c-e791fa6e70e8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80244) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 08:30:53.398778 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG dbcounter [-] [80244] Writing DB stats keystone:SELECT=100 {{(pid=80244) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:30:53.411909 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8ef4af7-3172-4669-888c-e791fa6e70e8 None None] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:30:53.417093 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0a0772f-7944-4ec5-b468-85599e869111 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '42d15fc0515945acacbbffcf2356c7a5', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:30:53.417981 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0a0772f-7944-4ec5-b468-85599e869111 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:30:53.418263 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0a0772f-7944-4ec5-b468-85599e869111 admin admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:30:53.418536 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0a0772f-7944-4ec5-b468-85599e869111 admin admin] PATH_INFO: `/v3/projects/bd6dd7e116d24597860060d16a21ff26` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:30:53.419904 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0a0772f-7944-4ec5-b468-85599e869111 admin admin] RBAC: Authorizing `identity:delete_project(project_id=bd6dd7e116d24597860060d16a21ff26)` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:30:53.421902 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0a0772f-7944-4ec5-b468-85599e869111 admin admin] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:30:53.485801 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.notifications [None req-d0a0772f-7944-4ec5-b468-85599e869111 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'bd6dd7e116d24597860060d16a21ff26', 'request_id': 'req-d0a0772f-7944-4ec5-b468-85599e869111'} {{(pid=80244) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 08:30:53.495709 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.notifications [None req-d0a0772f-7944-4ec5-b468-85599e869111 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'bd6dd7e116d24597860060d16a21ff26', 'request_id': 'req-d0a0772f-7944-4ec5-b468-85599e869111'} {{(pid=80244) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 08:30:53.503057 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.notifications [None req-d0a0772f-7944-4ec5-b468-85599e869111 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'bd6dd7e116d24597860060d16a21ff26', 'request_id': 'req-d0a0772f-7944-4ec5-b468-85599e869111'} {{(pid=80244) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 08:30:53.515292 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.notifications [None req-d0a0772f-7944-4ec5-b468-85599e869111 admin admin] The token cache is being invalidate because project bd6dd7e116d24597860060d16a21ff26 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80244) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 21 08:30:53.515712 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.notifications [None req-d0a0772f-7944-4ec5-b468-85599e869111 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80244) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 08:30:53.516377 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.notifications [None req-d0a0772f-7944-4ec5-b468-85599e869111 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80244) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 08:30:53.518350 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 935/1856] 10.222.0.47 () {58 vars in 1232 bytes} [Thu Jul 21 08:30:53 2022] DELETE /identity/v3/projects/bd6dd7e116d24597860060d16a21ff26 => generated 0 bytes in 138 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 08:30:53.522220 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acd61b6c-c6ff-4446-9eda-b57ae4abe6ae service neutron] Authenticating user token {{(pid=80245) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:30:53.535045 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acd61b6c-c6ff-4446-9eda-b57ae4abe6ae service neutron] Validating token access rules against request {{(pid=80245) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:30:53.593202 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-213bf0da-72ba-4267-b194-3797240a1eff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '42d15fc0515945acacbbffcf2356c7a5', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80245) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:30:53.594083 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-213bf0da-72ba-4267-b194-3797240a1eff admin admin] REQUEST_METHOD: `DELETE` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:30:53.594396 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-213bf0da-72ba-4267-b194-3797240a1eff admin admin] SCRIPT_NAME: `/identity` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:30:53.594675 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-213bf0da-72ba-4267-b194-3797240a1eff admin admin] PATH_INFO: `/v3/users/8ce9f8b8342b40429f17c7e7becf6208` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:30:53.595893 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-213bf0da-72ba-4267-b194-3797240a1eff admin admin] RBAC: Authorizing `identity:delete_user(user_id=8ce9f8b8342b40429f17c7e7becf6208)` {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:30:53.597829 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-213bf0da-72ba-4267-b194-3797240a1eff admin admin] RBAC: Authorization granted {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:30:53.643145 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.notifications [None req-213bf0da-72ba-4267-b194-3797240a1eff admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8ce9f8b8342b40429f17c7e7becf6208'} {{(pid=80245) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 08:30:53.650211 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.notifications [None req-213bf0da-72ba-4267-b194-3797240a1eff admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8ce9f8b8342b40429f17c7e7becf6208'} {{(pid=80245) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 08:30:53.662336 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.notifications [None req-213bf0da-72ba-4267-b194-3797240a1eff admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8ce9f8b8342b40429f17c7e7becf6208'} {{(pid=80245) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 08:30:53.669353 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.notifications [None req-213bf0da-72ba-4267-b194-3797240a1eff admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8ce9f8b8342b40429f17c7e7becf6208'} {{(pid=80245) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 08:30:53.669999 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.notifications [None req-213bf0da-72ba-4267-b194-3797240a1eff admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8ce9f8b8342b40429f17c7e7becf6208'} {{(pid=80245) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 08:30:53.675906 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: [pid: 80245|app: 0|req: 922/1857] 10.222.0.47 () {58 vars in 1223 bytes} [Thu Jul 21 08:30:53 2022] DELETE /identity/v3/users/8ce9f8b8342b40429f17c7e7becf6208 => generated 0 bytes in 154 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 08:30:53.947643 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0a0772f-7944-4ec5-b468-85599e869111 admin admin] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:30:53.949092 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.fernet_utils [None req-d0a0772f-7944-4ec5-b468-85599e869111 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80244) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 08:30:53.963095 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0a0772f-7944-4ec5-b468-85599e869111 admin admin] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:30:54.022612 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eefce2a-468a-4f06-9c91-de2072675264 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '42d15fc0515945acacbbffcf2356c7a5', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:30:54.023438 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eefce2a-468a-4f06-9c91-de2072675264 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:30:54.023726 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eefce2a-468a-4f06-9c91-de2072675264 admin admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:30:54.024003 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eefce2a-468a-4f06-9c91-de2072675264 admin admin] PATH_INFO: `/v3/projects/5c0104061b6a49aca5799d0caf5530bd` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:30:54.025204 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2eefce2a-468a-4f06-9c91-de2072675264 admin admin] RBAC: Authorizing `identity:delete_project(project_id=5c0104061b6a49aca5799d0caf5530bd)` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:30:54.027140 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2eefce2a-468a-4f06-9c91-de2072675264 admin admin] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:30:54.073768 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.notifications [None req-2eefce2a-468a-4f06-9c91-de2072675264 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5c0104061b6a49aca5799d0caf5530bd', 'request_id': 'req-2eefce2a-468a-4f06-9c91-de2072675264'} {{(pid=80244) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 08:30:54.081809 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.notifications [None req-2eefce2a-468a-4f06-9c91-de2072675264 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5c0104061b6a49aca5799d0caf5530bd', 'request_id': 'req-2eefce2a-468a-4f06-9c91-de2072675264'} {{(pid=80244) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 08:30:54.088393 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.notifications [None req-2eefce2a-468a-4f06-9c91-de2072675264 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5c0104061b6a49aca5799d0caf5530bd', 'request_id': 'req-2eefce2a-468a-4f06-9c91-de2072675264'} {{(pid=80244) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 08:30:54.099915 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.notifications [None req-2eefce2a-468a-4f06-9c91-de2072675264 admin admin] The token cache is being invalidate because project 5c0104061b6a49aca5799d0caf5530bd was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80244) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 21 08:30:54.100198 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.notifications [None req-2eefce2a-468a-4f06-9c91-de2072675264 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80244) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 08:30:54.100849 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.notifications [None req-2eefce2a-468a-4f06-9c91-de2072675264 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80244) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 08:30:54.102538 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 936/1858] 10.222.0.47 () {58 vars in 1232 bytes} [Thu Jul 21 08:30:53 2022] DELETE /identity/v3/projects/5c0104061b6a49aca5799d0caf5530bd => generated 0 bytes in 156 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 08:30:54.108077 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95b9b46d-e3b8-40a0-bf29-2568425d0f2d None None] REQUEST_METHOD: `POST` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:30:54.108363 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95b9b46d-e3b8-40a0-bf29-2568425d0f2d None None] SCRIPT_NAME: `/identity` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:30:54.108644 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95b9b46d-e3b8-40a0-bf29-2568425d0f2d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:30:54.128550 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.auth.core [None req-95b9b46d-e3b8-40a0-bf29-2568425d0f2d None None] MFA Rules not processed for user `2dce40b8d74c41ec94cc36e6ff05d412`. Rule list: `[]` (Enabled: `True`). {{(pid=80245) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 08:30:54.129999 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.fernet_utils [None req-95b9b46d-e3b8-40a0-bf29-2568425d0f2d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80245) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 08:30:54.202625 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: [pid: 80245|app: 0|req: 923/1859] 10.222.0.47 () {58 vars in 957 bytes} [Thu Jul 21 08:30:54 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 08:30:54.212056 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdec0bf1-5872-4331-931a-71d0f34ccdea None None] REQUEST_METHOD: `POST` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:30:54.212390 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdec0bf1-5872-4331-931a-71d0f34ccdea None None] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:30:54.212684 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdec0bf1-5872-4331-931a-71d0f34ccdea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:30:54.238788 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.auth.core [None req-cdec0bf1-5872-4331-931a-71d0f34ccdea None None] MFA Rules not processed for user `2dce40b8d74c41ec94cc36e6ff05d412`. Rule list: `[]` (Enabled: `True`). {{(pid=80244) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 08:30:54.240634 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.fernet_utils [None req-cdec0bf1-5872-4331-931a-71d0f34ccdea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80244) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 08:30:54.256152 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 937/1860] 10.222.0.47 () {58 vars in 957 bytes} [Thu Jul 21 08:30:54 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 46 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 08:30:54.258552 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95b9b46d-e3b8-40a0-bf29-2568425d0f2d None None] Authenticating user token {{(pid=80245) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:30:54.280038 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95b9b46d-e3b8-40a0-bf29-2568425d0f2d None None] Validating token access rules against request {{(pid=80245) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:30:54.294276 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65bb4111-9df0-48c2-beb6-84b32e3b39c0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '42d15fc0515945acacbbffcf2356c7a5', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80245) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:30:54.295980 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65bb4111-9df0-48c2-beb6-84b32e3b39c0 admin admin] REQUEST_METHOD: `GET` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:30:54.296264 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65bb4111-9df0-48c2-beb6-84b32e3b39c0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:30:54.296653 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65bb4111-9df0-48c2-beb6-84b32e3b39c0 admin admin] PATH_INFO: `/v3/domains` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:30:54.297404 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65bb4111-9df0-48c2-beb6-84b32e3b39c0 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80245) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 08:30:54.297712 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65bb4111-9df0-48c2-beb6-84b32e3b39c0 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:30:54.299347 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65bb4111-9df0-48c2-beb6-84b32e3b39c0 admin admin] RBAC: Authorization granted {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:30:54.318321 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: [pid: 80245|app: 0|req: 924/1861] 10.222.0.47 () {58 vars in 1152 bytes} [Thu Jul 21 08:30:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 57 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 08:30:54.321057 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdec0bf1-5872-4331-931a-71d0f34ccdea None None] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:30:54.331101 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdec0bf1-5872-4331-931a-71d0f34ccdea None None] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:30:54.343313 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de7a3613-8bac-4d14-9c44-2593d5f309e4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '42d15fc0515945acacbbffcf2356c7a5', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:30:54.344100 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de7a3613-8bac-4d14-9c44-2593d5f309e4 admin admin] REQUEST_METHOD: `POST` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:30:54.344487 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de7a3613-8bac-4d14-9c44-2593d5f309e4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:30:54.344834 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de7a3613-8bac-4d14-9c44-2593d5f309e4 admin admin] PATH_INFO: `/v3/projects` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:30:54.345592 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de7a3613-8bac-4d14-9c44-2593d5f309e4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:30:54.347558 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de7a3613-8bac-4d14-9c44-2593d5f309e4 admin admin] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:30:54.356248 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG dbcounter [-] [80244] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=80244) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:30:54.375646 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 938/1862] 10.222.0.47 () {60 vars in 1152 bytes} [Thu Jul 21 08:30:54 2022] POST /identity/v3/projects => generated 385 bytes in 56 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 08:30:54.380038 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65bb4111-9df0-48c2-beb6-84b32e3b39c0 admin admin] Authenticating user token {{(pid=80245) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:30:54.385173 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG dbcounter [-] [80245] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80245) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:30:54.392303 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65bb4111-9df0-48c2-beb6-84b32e3b39c0 admin admin] Validating token access rules against request {{(pid=80245) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:30:54.450189 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd53f94e-929e-4bac-9cec-8bb8823d5474 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '42d15fc0515945acacbbffcf2356c7a5', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80245) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:30:54.450973 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd53f94e-929e-4bac-9cec-8bb8823d5474 admin admin] REQUEST_METHOD: `POST` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:30:54.451262 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd53f94e-929e-4bac-9cec-8bb8823d5474 admin admin] SCRIPT_NAME: `/identity` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:30:54.451573 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd53f94e-929e-4bac-9cec-8bb8823d5474 admin admin] PATH_INFO: `/v3/users` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:30:54.452281 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd53f94e-929e-4bac-9cec-8bb8823d5474 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:30:54.454201 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd53f94e-929e-4bac-9cec-8bb8823d5474 admin admin] RBAC: Authorization granted {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:30:54.478636 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: [pid: 80245|app: 0|req: 925/1863] 10.222.0.47 () {60 vars in 1143 bytes} [Thu Jul 21 08:30:54 2022] POST /identity/v3/users => generated 293 bytes in 99 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 08:30:54.479512 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de7a3613-8bac-4d14-9c44-2593d5f309e4 admin admin] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:30:54.491037 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de7a3613-8bac-4d14-9c44-2593d5f309e4 admin admin] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:30:54.495764 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f74f657a-ec8f-4d8a-88a2-e225ed70979b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '42d15fc0515945acacbbffcf2356c7a5', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:30:54.496440 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f74f657a-ec8f-4d8a-88a2-e225ed70979b admin admin] REQUEST_METHOD: `GET` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:30:54.496723 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f74f657a-ec8f-4d8a-88a2-e225ed70979b admin admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:30:54.496998 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f74f657a-ec8f-4d8a-88a2-e225ed70979b admin admin] PATH_INFO: `/v3/roles` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:30:54.497784 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f74f657a-ec8f-4d8a-88a2-e225ed70979b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:30:54.499415 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f74f657a-ec8f-4d8a-88a2-e225ed70979b admin admin] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:30:54.510386 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 939/1864] 10.222.0.47 () {58 vars in 1121 bytes} [Thu Jul 21 08:30:54 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 21 08:30:54.514567 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd53f94e-929e-4bac-9cec-8bb8823d5474 admin admin] Authenticating user token {{(pid=80245) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:30:54.524936 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd53f94e-929e-4bac-9cec-8bb8823d5474 admin admin] Validating token access rules against request {{(pid=80245) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:30:54.530247 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a828efe-fb3d-426a-81ff-05b6dfedbfdd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '42d15fc0515945acacbbffcf2356c7a5', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80245) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:30:54.530853 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a828efe-fb3d-426a-81ff-05b6dfedbfdd admin admin] REQUEST_METHOD: `GET` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:30:54.531138 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a828efe-fb3d-426a-81ff-05b6dfedbfdd admin admin] SCRIPT_NAME: `/identity` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:30:54.531448 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a828efe-fb3d-426a-81ff-05b6dfedbfdd admin admin] PATH_INFO: `/v3/roles` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:30:54.532178 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a828efe-fb3d-426a-81ff-05b6dfedbfdd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:30:54.533818 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a828efe-fb3d-426a-81ff-05b6dfedbfdd admin admin] RBAC: Authorization granted {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:30:54.545135 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: [pid: 80245|app: 0|req: 926/1865] 10.222.0.47 () {58 vars in 1121 bytes} [Thu Jul 21 08:30:54 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 08:30:54.549454 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f74f657a-ec8f-4d8a-88a2-e225ed70979b admin admin] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:30:54.559938 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f74f657a-ec8f-4d8a-88a2-e225ed70979b admin admin] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:30:54.567036 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45937ebb-d6d8-45cb-ad0c-44cc701bf517 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '42d15fc0515945acacbbffcf2356c7a5', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:30:54.568114 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45937ebb-d6d8-45cb-ad0c-44cc701bf517 admin admin] REQUEST_METHOD: `PUT` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:30:54.568408 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45937ebb-d6d8-45cb-ad0c-44cc701bf517 admin admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:30:54.568688 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45937ebb-d6d8-45cb-ad0c-44cc701bf517 admin admin] PATH_INFO: `/v3/projects/5a420c50a1754e97884677f8e5dfaae6/users/81556d24089045c7b11f028737494382/roles/3a074738221e4601b5205715ba87d240` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:30:54.584620 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45937ebb-d6d8-45cb-ad0c-44cc701bf517 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5a420c50a1754e97884677f8e5dfaae6, user_id=81556d24089045c7b11f028737494382, role_id=3a074738221e4601b5205715ba87d240)` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:30:54.587564 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45937ebb-d6d8-45cb-ad0c-44cc701bf517 admin admin] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:30:54.602040 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 940/1866] 10.222.0.47 () {60 vars in 1482 bytes} [Thu Jul 21 08:30:54 2022] PUT /identity/v3/projects/5a420c50a1754e97884677f8e5dfaae6/users/81556d24089045c7b11f028737494382/roles/3a074738221e4601b5205715ba87d240 => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 08:30:54.612897 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57210bbd-4f34-49d5-8ae2-8f36d899d8d9 None None] REQUEST_METHOD: `POST` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:30:54.613349 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57210bbd-4f34-49d5-8ae2-8f36d899d8d9 None None] SCRIPT_NAME: `/identity` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:30:54.613481 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57210bbd-4f34-49d5-8ae2-8f36d899d8d9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:30:54.640233 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.auth.core [None req-57210bbd-4f34-49d5-8ae2-8f36d899d8d9 None None] MFA Rules not processed for user `81556d24089045c7b11f028737494382`. Rule list: `[]` (Enabled: `True`). {{(pid=80245) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 08:30:54.641611 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.fernet_utils [None req-57210bbd-4f34-49d5-8ae2-8f36d899d8d9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80245) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 08:30:54.704431 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: [pid: 80245|app: 0|req: 927/1867] 10.222.0.47 () {58 vars in 957 bytes} [Thu Jul 21 08:30:54 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 21 08:30:54.709058 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45937ebb-d6d8-45cb-ad0c-44cc701bf517 admin admin] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:30:54.771068 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45937ebb-d6d8-45cb-ad0c-44cc701bf517 admin admin] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:30:54.775684 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43934ed4-2a0c-448e-abd1-50cacb667a74 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '42d15fc0515945acacbbffcf2356c7a5', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:30:54.776369 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43934ed4-2a0c-448e-abd1-50cacb667a74 admin admin] REQUEST_METHOD: `POST` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:30:54.776653 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43934ed4-2a0c-448e-abd1-50cacb667a74 admin admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:30:54.776913 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43934ed4-2a0c-448e-abd1-50cacb667a74 admin admin] PATH_INFO: `/v3/projects` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:30:54.777630 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43934ed4-2a0c-448e-abd1-50cacb667a74 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:30:54.780158 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43934ed4-2a0c-448e-abd1-50cacb667a74 admin admin] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:30:54.804872 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 941/1868] 10.222.0.47 () {60 vars in 1152 bytes} [Thu Jul 21 08:30:54 2022] POST /identity/v3/projects => generated 385 bytes in 96 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 08:30:54.810037 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57210bbd-4f34-49d5-8ae2-8f36d899d8d9 None None] Authenticating user token {{(pid=80245) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:30:54.820830 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57210bbd-4f34-49d5-8ae2-8f36d899d8d9 None None] Validating token access rules against request {{(pid=80245) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:30:54.876771 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9260bbee-43da-4e6d-a6b0-7a2e7f265a55 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '42d15fc0515945acacbbffcf2356c7a5', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80245) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:30:54.877486 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9260bbee-43da-4e6d-a6b0-7a2e7f265a55 admin admin] REQUEST_METHOD: `POST` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:30:54.877781 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9260bbee-43da-4e6d-a6b0-7a2e7f265a55 admin admin] SCRIPT_NAME: `/identity` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:30:54.878050 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9260bbee-43da-4e6d-a6b0-7a2e7f265a55 admin admin] PATH_INFO: `/v3/users` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:30:54.878762 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9260bbee-43da-4e6d-a6b0-7a2e7f265a55 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:30:54.880988 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9260bbee-43da-4e6d-a6b0-7a2e7f265a55 admin admin] RBAC: Authorization granted {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:30:54.897450 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG dbcounter [-] [80245] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=80245) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:30:54.904431 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: [pid: 80245|app: 0|req: 928/1869] 10.222.0.47 () {60 vars in 1143 bytes} [Thu Jul 21 08:30:54 2022] POST /identity/v3/users => generated 299 bytes in 95 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 21 08:30:54.908460 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43934ed4-2a0c-448e-abd1-50cacb667a74 admin admin] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:30:54.918864 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43934ed4-2a0c-448e-abd1-50cacb667a74 admin admin] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:30:54.923554 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a20c6f1-e4d6-4f09-b308-1cab6ce4c784 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '42d15fc0515945acacbbffcf2356c7a5', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:30:54.924226 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a20c6f1-e4d6-4f09-b308-1cab6ce4c784 admin admin] REQUEST_METHOD: `GET` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:30:54.924520 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a20c6f1-e4d6-4f09-b308-1cab6ce4c784 admin admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:30:54.924803 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a20c6f1-e4d6-4f09-b308-1cab6ce4c784 admin admin] PATH_INFO: `/v3/roles` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:30:54.925531 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a20c6f1-e4d6-4f09-b308-1cab6ce4c784 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:30:54.927133 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a20c6f1-e4d6-4f09-b308-1cab6ce4c784 admin admin] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:30:54.937758 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 942/1870] 10.222.0.47 () {58 vars in 1121 bytes} [Thu Jul 21 08:30:54 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 21 08:30:54.941982 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9260bbee-43da-4e6d-a6b0-7a2e7f265a55 admin admin] Authenticating user token {{(pid=80245) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:30:54.952848 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9260bbee-43da-4e6d-a6b0-7a2e7f265a55 admin admin] Validating token access rules against request {{(pid=80245) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:30:54.957501 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddd3c732-1d6c-47c4-8ea2-76ee2453ecca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '42d15fc0515945acacbbffcf2356c7a5', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80245) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:30:54.958406 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddd3c732-1d6c-47c4-8ea2-76ee2453ecca admin admin] REQUEST_METHOD: `PUT` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:30:54.958696 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddd3c732-1d6c-47c4-8ea2-76ee2453ecca admin admin] SCRIPT_NAME: `/identity` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:30:54.958977 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddd3c732-1d6c-47c4-8ea2-76ee2453ecca admin admin] PATH_INFO: `/v3/projects/ba9298146fbb492a8a03deb611cec1d8/users/a0c04abc7b2f4fb4ba164a2fbbcfe290/roles/e29870f6e9e2411a9db0d5bf45f32ca5` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:30:54.974208 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddd3c732-1d6c-47c4-8ea2-76ee2453ecca admin admin] RBAC: Authorizing `identity:create_grant(project_id=ba9298146fbb492a8a03deb611cec1d8, user_id=a0c04abc7b2f4fb4ba164a2fbbcfe290, role_id=e29870f6e9e2411a9db0d5bf45f32ca5)` {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:30:54.976897 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddd3c732-1d6c-47c4-8ea2-76ee2453ecca admin admin] RBAC: Authorization granted {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:30:54.989902 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: [pid: 80245|app: 0|req: 929/1871] 10.222.0.47 () {60 vars in 1482 bytes} [Thu Jul 21 08:30:54 2022] PUT /identity/v3/projects/ba9298146fbb492a8a03deb611cec1d8/users/a0c04abc7b2f4fb4ba164a2fbbcfe290/roles/e29870f6e9e2411a9db0d5bf45f32ca5 => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 08:30:54.993967 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a20c6f1-e4d6-4f09-b308-1cab6ce4c784 admin admin] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:30:55.005088 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a20c6f1-e4d6-4f09-b308-1cab6ce4c784 admin admin] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:30:55.061198 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09010ae1-69cc-4dfc-9fd3-ceb593c38133 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '42d15fc0515945acacbbffcf2356c7a5', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:30:55.061792 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09010ae1-69cc-4dfc-9fd3-ceb593c38133 admin admin] REQUEST_METHOD: `GET` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:30:55.062076 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09010ae1-69cc-4dfc-9fd3-ceb593c38133 admin admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:30:55.062356 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09010ae1-69cc-4dfc-9fd3-ceb593c38133 admin admin] PATH_INFO: `/v3/roles` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:30:55.063113 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09010ae1-69cc-4dfc-9fd3-ceb593c38133 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:30:55.064778 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09010ae1-69cc-4dfc-9fd3-ceb593c38133 admin admin] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:30:55.079415 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 943/1872] 10.222.0.47 () {58 vars in 1121 bytes} [Thu Jul 21 08:30:54 2022] GET /identity/v3/roles => generated 1420 bytes in 83 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 08:30:55.080831 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddd3c732-1d6c-47c4-8ea2-76ee2453ecca admin admin] Authenticating user token {{(pid=80245) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:30:55.094672 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddd3c732-1d6c-47c4-8ea2-76ee2453ecca admin admin] Validating token access rules against request {{(pid=80245) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:30:55.099353 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8edcba7-a247-4f43-be19-c7863c5d0156 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '42d15fc0515945acacbbffcf2356c7a5', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80245) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:30:55.100241 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8edcba7-a247-4f43-be19-c7863c5d0156 admin admin] REQUEST_METHOD: `PUT` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:30:55.100522 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8edcba7-a247-4f43-be19-c7863c5d0156 admin admin] SCRIPT_NAME: `/identity` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:30:55.100795 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8edcba7-a247-4f43-be19-c7863c5d0156 admin admin] PATH_INFO: `/v3/projects/ba9298146fbb492a8a03deb611cec1d8/users/a0c04abc7b2f4fb4ba164a2fbbcfe290/roles/e29870f6e9e2411a9db0d5bf45f32ca5` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:30:55.102759 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8edcba7-a247-4f43-be19-c7863c5d0156 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ba9298146fbb492a8a03deb611cec1d8, user_id=a0c04abc7b2f4fb4ba164a2fbbcfe290, role_id=e29870f6e9e2411a9db0d5bf45f32ca5)` {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:30:55.105456 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8edcba7-a247-4f43-be19-c7863c5d0156 admin admin] RBAC: Authorization granted {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:30:55.116285 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: [pid: 80245|app: 0|req: 930/1873] 10.222.0.47 () {60 vars in 1482 bytes} [Thu Jul 21 08:30:55 2022] PUT /identity/v3/projects/ba9298146fbb492a8a03deb611cec1d8/users/a0c04abc7b2f4fb4ba164a2fbbcfe290/roles/e29870f6e9e2411a9db0d5bf45f32ca5 => generated 0 bytes in 36 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 08:30:55.128313 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c8dedac-2100-466f-8aee-790f23b93a0f None None] REQUEST_METHOD: `POST` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:30:55.128610 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c8dedac-2100-466f-8aee-790f23b93a0f None None] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:30:55.128894 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c8dedac-2100-466f-8aee-790f23b93a0f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:30:55.156757 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.auth.core [None req-4c8dedac-2100-466f-8aee-790f23b93a0f None None] MFA Rules not processed for user `a0c04abc7b2f4fb4ba164a2fbbcfe290`. Rule list: `[]` (Enabled: `True`). {{(pid=80244) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 08:30:55.157986 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.fernet_utils [None req-4c8dedac-2100-466f-8aee-790f23b93a0f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80244) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 08:30:55.169690 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG dbcounter [-] [80244] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80244) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:30:55.229072 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 944/1874] 10.222.0.47 () {58 vars in 957 bytes} [Thu Jul 21 08:30:55 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 08:30:55.241524 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8edcba7-a247-4f43-be19-c7863c5d0156 admin admin] Authenticating user token {{(pid=80245) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:30:55.243260 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.fernet_utils [None req-e8edcba7-a247-4f43-be19-c7863c5d0156 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80245) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 08:30:55.334359 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8edcba7-a247-4f43-be19-c7863c5d0156 admin admin] Validating token access rules against request {{(pid=80245) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:30:55.350578 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f4ca9fa-9a6c-4f4b-bcb6-662aa4b048f3 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4f40410b7674687ab5362ac982248ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4faef13aa50e4e36b6c265fb1398fa14', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80245) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:30:55.351662 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f4ca9fa-9a6c-4f4b-bcb6-662aa4b048f3 service cinder] REQUEST_METHOD: `GET` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:30:55.354442 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f4ca9fa-9a6c-4f4b-bcb6-662aa4b048f3 service cinder] SCRIPT_NAME: `/identity` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:30:55.354759 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f4ca9fa-9a6c-4f4b-bcb6-662aa4b048f3 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:30:55.387745 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f4ca9fa-9a6c-4f4b-bcb6-662aa4b048f3 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:30:55.392932 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f4ca9fa-9a6c-4f4b-bcb6-662aa4b048f3 service cinder] RBAC: Authorization granted {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:30:55.420625 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: [pid: 80245|app: 0|req: 931/1875] 10.222.0.47 () {60 vars in 1365 bytes} [Thu Jul 21 08:30:55 2022] GET /identity/v3/auth/tokens => generated 3319 bytes in 179 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 08:30:55.462140 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c8dedac-2100-466f-8aee-790f23b93a0f None None] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:30:55.554830 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c8dedac-2100-466f-8aee-790f23b93a0f None None] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:30:55.563227 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daf21268-3f1c-44d5-a586-2f93f1955d96 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '42d15fc0515945acacbbffcf2356c7a5', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:30:55.564388 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daf21268-3f1c-44d5-a586-2f93f1955d96 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:30:55.564727 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daf21268-3f1c-44d5-a586-2f93f1955d96 admin admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:30:55.565308 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daf21268-3f1c-44d5-a586-2f93f1955d96 admin admin] PATH_INFO: `/v3/users/81556d24089045c7b11f028737494382` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:30:55.567641 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daf21268-3f1c-44d5-a586-2f93f1955d96 admin admin] RBAC: Authorizing `identity:delete_user(user_id=81556d24089045c7b11f028737494382)` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:30:55.570679 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daf21268-3f1c-44d5-a586-2f93f1955d96 admin admin] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:30:55.644277 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.notifications [None req-daf21268-3f1c-44d5-a586-2f93f1955d96 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '81556d24089045c7b11f028737494382'} {{(pid=80244) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 08:30:55.657324 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.notifications [None req-daf21268-3f1c-44d5-a586-2f93f1955d96 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '81556d24089045c7b11f028737494382'} {{(pid=80244) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 08:30:55.683487 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.notifications [None req-daf21268-3f1c-44d5-a586-2f93f1955d96 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '81556d24089045c7b11f028737494382'} {{(pid=80244) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 08:30:55.684714 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.notifications [None req-daf21268-3f1c-44d5-a586-2f93f1955d96 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '81556d24089045c7b11f028737494382'} {{(pid=80244) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 08:30:55.703218 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.notifications [None req-daf21268-3f1c-44d5-a586-2f93f1955d96 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '81556d24089045c7b11f028737494382'} {{(pid=80244) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 08:30:55.716487 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 945/1876] 10.222.0.47 () {58 vars in 1223 bytes} [Thu Jul 21 08:30:55 2022] DELETE /identity/v3/users/81556d24089045c7b11f028737494382 => generated 0 bytes in 255 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 08:30:55.721476 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16b84a9f-018a-4a29-88a5-a90475aa9172 None None] REQUEST_METHOD: `POST` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:30:55.721983 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16b84a9f-018a-4a29-88a5-a90475aa9172 None None] SCRIPT_NAME: `/identity` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:30:55.722418 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16b84a9f-018a-4a29-88a5-a90475aa9172 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:30:55.752012 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.auth.core [None req-16b84a9f-018a-4a29-88a5-a90475aa9172 None None] MFA Rules not processed for user `2dce40b8d74c41ec94cc36e6ff05d412`. Rule list: `[]` (Enabled: `True`). {{(pid=80245) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 08:30:55.753307 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.fernet_utils [None req-16b84a9f-018a-4a29-88a5-a90475aa9172 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80245) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 08:30:55.857756 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: [pid: 80245|app: 0|req: 932/1877] 10.222.0.47 () {58 vars in 957 bytes} [Thu Jul 21 08:30:55 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 137 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 08:30:55.897224 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daf21268-3f1c-44d5-a586-2f93f1955d96 admin admin] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:30:55.899155 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.fernet_utils [None req-daf21268-3f1c-44d5-a586-2f93f1955d96 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80244) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 08:30:55.936952 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG dbcounter [-] [80244] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80244) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:30:55.993125 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daf21268-3f1c-44d5-a586-2f93f1955d96 admin admin] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:30:55.997628 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42c18cc8-1bb0-4f57-a3d7-4b46f825c6dc service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20ca3c3bd3774691b3374cfbfec6e494', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4faef13aa50e4e36b6c265fb1398fa14', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:30:55.998379 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42c18cc8-1bb0-4f57-a3d7-4b46f825c6dc service neutron] REQUEST_METHOD: `GET` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:30:55.998739 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42c18cc8-1bb0-4f57-a3d7-4b46f825c6dc service neutron] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:30:55.999085 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42c18cc8-1bb0-4f57-a3d7-4b46f825c6dc service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:30:56.009820 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42c18cc8-1bb0-4f57-a3d7-4b46f825c6dc service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:30:56.011835 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42c18cc8-1bb0-4f57-a3d7-4b46f825c6dc service neutron] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:30:56.023574 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 946/1878] 10.222.0.47 () {60 vars in 1365 bytes} [Thu Jul 21 08:30:55 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 127 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 08:30:56.509063 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16b84a9f-018a-4a29-88a5-a90475aa9172 None None] Authenticating user token {{(pid=80245) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:30:56.511116 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.fernet_utils [None req-16b84a9f-018a-4a29-88a5-a90475aa9172 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80245) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 08:30:56.524767 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16b84a9f-018a-4a29-88a5-a90475aa9172 None None] Validating token access rules against request {{(pid=80245) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:30:56.533249 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-761c85a5-20bd-4c40-a09e-aa8636d09007 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '42d15fc0515945acacbbffcf2356c7a5', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80245) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:30:56.535384 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-761c85a5-20bd-4c40-a09e-aa8636d09007 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:30:56.535384 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-761c85a5-20bd-4c40-a09e-aa8636d09007 admin admin] SCRIPT_NAME: `/identity` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:30:56.536598 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-761c85a5-20bd-4c40-a09e-aa8636d09007 admin admin] PATH_INFO: `/v3/projects/5a420c50a1754e97884677f8e5dfaae6` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:30:56.539790 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-761c85a5-20bd-4c40-a09e-aa8636d09007 admin admin] RBAC: Authorizing `identity:delete_project(project_id=5a420c50a1754e97884677f8e5dfaae6)` {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:30:56.545078 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-761c85a5-20bd-4c40-a09e-aa8636d09007 admin admin] RBAC: Authorization granted {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:30:56.622091 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.notifications [None req-761c85a5-20bd-4c40-a09e-aa8636d09007 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5a420c50a1754e97884677f8e5dfaae6', 'request_id': 'req-761c85a5-20bd-4c40-a09e-aa8636d09007'} {{(pid=80245) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 08:30:56.632090 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.notifications [None req-761c85a5-20bd-4c40-a09e-aa8636d09007 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5a420c50a1754e97884677f8e5dfaae6', 'request_id': 'req-761c85a5-20bd-4c40-a09e-aa8636d09007'} {{(pid=80245) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 08:30:56.639992 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG dbcounter [-] [80245] Writing DB stats keystone:SELECT=92,keystone:INSERT=2,keystone:DELETE=6 {{(pid=80245) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:30:56.641617 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.notifications [None req-761c85a5-20bd-4c40-a09e-aa8636d09007 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5a420c50a1754e97884677f8e5dfaae6', 'request_id': 'req-761c85a5-20bd-4c40-a09e-aa8636d09007'} {{(pid=80245) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 08:30:56.660721 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.notifications [None req-761c85a5-20bd-4c40-a09e-aa8636d09007 admin admin] The token cache is being invalidate because project 5a420c50a1754e97884677f8e5dfaae6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80245) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 21 08:30:56.661138 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.notifications [None req-761c85a5-20bd-4c40-a09e-aa8636d09007 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80245) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 08:30:56.662441 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.notifications [None req-761c85a5-20bd-4c40-a09e-aa8636d09007 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80245) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 08:30:56.667458 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: [pid: 80245|app: 0|req: 933/1879] 10.222.0.47 () {58 vars in 1232 bytes} [Thu Jul 21 08:30:56 2022] DELETE /identity/v3/projects/5a420c50a1754e97884677f8e5dfaae6 => generated 0 bytes in 156 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 08:30:56.669637 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42c18cc8-1bb0-4f57-a3d7-4b46f825c6dc service neutron] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:30:56.683901 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42c18cc8-1bb0-4f57-a3d7-4b46f825c6dc service neutron] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:30:56.762206 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3158e4a-b934-4aec-b339-9ae3b14d6d9e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '42d15fc0515945acacbbffcf2356c7a5', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:30:56.763661 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3158e4a-b934-4aec-b339-9ae3b14d6d9e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:30:56.763953 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3158e4a-b934-4aec-b339-9ae3b14d6d9e admin admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:30:56.764233 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3158e4a-b934-4aec-b339-9ae3b14d6d9e admin admin] PATH_INFO: `/v3/users/a0c04abc7b2f4fb4ba164a2fbbcfe290` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:30:56.765472 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3158e4a-b934-4aec-b339-9ae3b14d6d9e admin admin] RBAC: Authorizing `identity:delete_user(user_id=a0c04abc7b2f4fb4ba164a2fbbcfe290)` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:30:56.768416 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-761c85a5-20bd-4c40-a09e-aa8636d09007 admin admin] Authenticating user token {{(pid=80245) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:30:56.768883 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3158e4a-b934-4aec-b339-9ae3b14d6d9e admin admin] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:30:56.771838 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.fernet_utils [None req-761c85a5-20bd-4c40-a09e-aa8636d09007 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80245) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 08:30:56.790152 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-761c85a5-20bd-4c40-a09e-aa8636d09007 admin admin] Validating token access rules against request {{(pid=80245) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:30:56.795557 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82a5f8ba-3048-4bb4-94c3-b7a047676b1d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '42d15fc0515945acacbbffcf2356c7a5', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80245) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:30:56.796589 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82a5f8ba-3048-4bb4-94c3-b7a047676b1d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:30:56.796876 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82a5f8ba-3048-4bb4-94c3-b7a047676b1d admin admin] SCRIPT_NAME: `/identity` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:30:56.797158 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82a5f8ba-3048-4bb4-94c3-b7a047676b1d admin admin] PATH_INFO: `/v3/users/7c9ed461f0e5488f84eef4ede64db72f` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:30:56.798653 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82a5f8ba-3048-4bb4-94c3-b7a047676b1d admin admin] RBAC: Authorizing `identity:delete_user(user_id=7c9ed461f0e5488f84eef4ede64db72f)` {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:30:56.800539 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82a5f8ba-3048-4bb4-94c3-b7a047676b1d admin admin] RBAC: Authorization granted {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:30:56.849165 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.notifications [None req-c3158e4a-b934-4aec-b339-9ae3b14d6d9e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a0c04abc7b2f4fb4ba164a2fbbcfe290'} {{(pid=80244) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 08:30:56.867011 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.notifications [None req-c3158e4a-b934-4aec-b339-9ae3b14d6d9e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a0c04abc7b2f4fb4ba164a2fbbcfe290'} {{(pid=80244) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 08:30:56.891650 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.notifications [None req-c3158e4a-b934-4aec-b339-9ae3b14d6d9e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a0c04abc7b2f4fb4ba164a2fbbcfe290'} {{(pid=80244) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 08:30:56.892643 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.notifications [None req-c3158e4a-b934-4aec-b339-9ae3b14d6d9e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a0c04abc7b2f4fb4ba164a2fbbcfe290'} {{(pid=80244) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 08:30:56.902435 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.notifications [None req-82a5f8ba-3048-4bb4-94c3-b7a047676b1d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7c9ed461f0e5488f84eef4ede64db72f'} {{(pid=80245) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 08:30:56.904929 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.notifications [None req-c3158e4a-b934-4aec-b339-9ae3b14d6d9e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a0c04abc7b2f4fb4ba164a2fbbcfe290'} {{(pid=80244) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 08:30:56.917236 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 947/1880] 10.222.0.47 () {58 vars in 1223 bytes} [Thu Jul 21 08:30:56 2022] DELETE /identity/v3/users/a0c04abc7b2f4fb4ba164a2fbbcfe290 => generated 0 bytes in 246 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 08:30:56.923135 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.notifications [None req-82a5f8ba-3048-4bb4-94c3-b7a047676b1d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7c9ed461f0e5488f84eef4ede64db72f'} {{(pid=80245) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 08:30:56.956231 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.notifications [None req-82a5f8ba-3048-4bb4-94c3-b7a047676b1d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7c9ed461f0e5488f84eef4ede64db72f'} {{(pid=80245) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 08:30:56.977721 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.notifications [None req-82a5f8ba-3048-4bb4-94c3-b7a047676b1d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7c9ed461f0e5488f84eef4ede64db72f'} {{(pid=80245) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 08:30:56.978797 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.notifications [None req-82a5f8ba-3048-4bb4-94c3-b7a047676b1d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7c9ed461f0e5488f84eef4ede64db72f'} {{(pid=80245) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 08:30:56.994506 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: [pid: 80245|app: 0|req: 934/1881] 10.222.0.47 () {58 vars in 1223 bytes} [Thu Jul 21 08:30:56 2022] DELETE /identity/v3/users/7c9ed461f0e5488f84eef4ede64db72f => generated 0 bytes in 227 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 08:30:57.000674 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90f844d8-3a53-48eb-b0f9-bebb06c3ce1c None None] REQUEST_METHOD: `POST` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:30:57.000674 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90f844d8-3a53-48eb-b0f9-bebb06c3ce1c None None] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:30:57.001336 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90f844d8-3a53-48eb-b0f9-bebb06c3ce1c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:30:57.046940 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.auth.core [None req-90f844d8-3a53-48eb-b0f9-bebb06c3ce1c None None] MFA Rules not processed for user `2dce40b8d74c41ec94cc36e6ff05d412`. Rule list: `[]` (Enabled: `True`). {{(pid=80244) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 08:30:57.048212 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.fernet_utils [None req-90f844d8-3a53-48eb-b0f9-bebb06c3ce1c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80244) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 08:30:57.120661 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG dbcounter [-] [80244] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80244) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:30:57.140946 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 948/1882] 10.222.0.47 () {58 vars in 957 bytes} [Thu Jul 21 08:30:56 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 142 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 08:30:57.154077 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82a5f8ba-3048-4bb4-94c3-b7a047676b1d admin admin] Authenticating user token {{(pid=80245) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:30:57.155857 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.fernet_utils [None req-82a5f8ba-3048-4bb4-94c3-b7a047676b1d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80245) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 08:30:57.238029 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82a5f8ba-3048-4bb4-94c3-b7a047676b1d admin admin] Validating token access rules against request {{(pid=80245) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:30:57.243707 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7542d08-8867-4cb8-b719-1805a23ae1aa service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20ca3c3bd3774691b3374cfbfec6e494', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4faef13aa50e4e36b6c265fb1398fa14', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80245) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:30:57.244394 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7542d08-8867-4cb8-b719-1805a23ae1aa service neutron] REQUEST_METHOD: `GET` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:30:57.244675 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7542d08-8867-4cb8-b719-1805a23ae1aa service neutron] SCRIPT_NAME: `/identity` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:30:57.244921 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7542d08-8867-4cb8-b719-1805a23ae1aa service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:30:57.260352 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7542d08-8867-4cb8-b719-1805a23ae1aa service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:30:57.261892 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7542d08-8867-4cb8-b719-1805a23ae1aa service neutron] RBAC: Authorization granted {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:30:57.279252 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: [pid: 80245|app: 0|req: 935/1883] 10.222.0.47 () {60 vars in 1365 bytes} [Thu Jul 21 08:30:57 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 125 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 08:30:57.637361 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90f844d8-3a53-48eb-b0f9-bebb06c3ce1c None None] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:30:57.641238 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.fernet_utils [None req-90f844d8-3a53-48eb-b0f9-bebb06c3ce1c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80244) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 08:30:57.658533 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90f844d8-3a53-48eb-b0f9-bebb06c3ce1c None None] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:30:57.670351 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73e267cd-f1d8-42b3-b621-4686de2176ed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '42d15fc0515945acacbbffcf2356c7a5', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:30:57.671231 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73e267cd-f1d8-42b3-b621-4686de2176ed admin admin] REQUEST_METHOD: `DELETE` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:30:57.671881 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73e267cd-f1d8-42b3-b621-4686de2176ed admin admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:30:57.672050 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73e267cd-f1d8-42b3-b621-4686de2176ed admin admin] PATH_INFO: `/v3/projects/ba9298146fbb492a8a03deb611cec1d8` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:30:57.673271 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73e267cd-f1d8-42b3-b621-4686de2176ed admin admin] RBAC: Authorizing `identity:delete_project(project_id=ba9298146fbb492a8a03deb611cec1d8)` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:30:57.675256 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73e267cd-f1d8-42b3-b621-4686de2176ed admin admin] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:30:57.745260 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.notifications [None req-73e267cd-f1d8-42b3-b621-4686de2176ed admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ba9298146fbb492a8a03deb611cec1d8', 'request_id': 'req-73e267cd-f1d8-42b3-b621-4686de2176ed'} {{(pid=80244) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 08:30:57.757910 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.notifications [None req-73e267cd-f1d8-42b3-b621-4686de2176ed admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ba9298146fbb492a8a03deb611cec1d8', 'request_id': 'req-73e267cd-f1d8-42b3-b621-4686de2176ed'} {{(pid=80244) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 08:30:57.769642 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.notifications [None req-73e267cd-f1d8-42b3-b621-4686de2176ed admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ba9298146fbb492a8a03deb611cec1d8', 'request_id': 'req-73e267cd-f1d8-42b3-b621-4686de2176ed'} {{(pid=80244) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 08:30:57.789041 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.notifications [None req-73e267cd-f1d8-42b3-b621-4686de2176ed admin admin] The token cache is being invalidate because project ba9298146fbb492a8a03deb611cec1d8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80244) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 21 08:30:57.789357 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.notifications [None req-73e267cd-f1d8-42b3-b621-4686de2176ed admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80244) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 08:30:57.799590 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.notifications [None req-73e267cd-f1d8-42b3-b621-4686de2176ed admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80244) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 08:30:57.801845 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 949/1884] 10.222.0.47 () {58 vars in 1232 bytes} [Thu Jul 21 08:30:57 2022] DELETE /identity/v3/projects/ba9298146fbb492a8a03deb611cec1d8 => generated 0 bytes in 165 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 08:30:57.807120 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1850d18a-fcdd-4403-b9f2-cabcf712ed97 None None] REQUEST_METHOD: `POST` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:30:57.807643 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1850d18a-fcdd-4403-b9f2-cabcf712ed97 None None] SCRIPT_NAME: `/identity` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:30:57.807990 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1850d18a-fcdd-4403-b9f2-cabcf712ed97 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:30:57.831198 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.auth.core [None req-1850d18a-fcdd-4403-b9f2-cabcf712ed97 None None] MFA Rules not processed for user `2dce40b8d74c41ec94cc36e6ff05d412`. Rule list: `[]` (Enabled: `True`). {{(pid=80245) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 08:30:57.832588 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.fernet_utils [None req-1850d18a-fcdd-4403-b9f2-cabcf712ed97 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80245) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 08:30:57.902988 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: [pid: 80245|app: 0|req: 936/1885] 10.222.0.47 () {58 vars in 957 bytes} [Thu Jul 21 08:30:57 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 21 08:30:57.911245 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a8b2d3a-665c-4c5c-9ae9-8af456f61e10 None None] REQUEST_METHOD: `POST` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:30:57.911593 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a8b2d3a-665c-4c5c-9ae9-8af456f61e10 None None] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:30:57.911879 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a8b2d3a-665c-4c5c-9ae9-8af456f61e10 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:30:57.930509 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1850d18a-fcdd-4403-b9f2-cabcf712ed97 None None] Authenticating user token {{(pid=80245) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:30:57.935558 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.fernet_utils [None req-1850d18a-fcdd-4403-b9f2-cabcf712ed97 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80245) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 08:30:57.935876 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.auth.core [None req-0a8b2d3a-665c-4c5c-9ae9-8af456f61e10 None None] MFA Rules not processed for user `2dce40b8d74c41ec94cc36e6ff05d412`. Rule list: `[]` (Enabled: `True`). {{(pid=80244) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 08:30:57.937243 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.fernet_utils [None req-0a8b2d3a-665c-4c5c-9ae9-8af456f61e10 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80244) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 08:30:57.946316 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG dbcounter [-] [80245] Writing DB stats keystone:SELECT=87,keystone:INSERT=2,keystone:DELETE=11 {{(pid=80245) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:30:57.947969 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 950/1886] 10.222.0.47 () {58 vars in 957 bytes} [Thu Jul 21 08:30:57 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 08:30:57.951465 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1850d18a-fcdd-4403-b9f2-cabcf712ed97 None None] Validating token access rules against request {{(pid=80245) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:30:57.954914 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a8b2d3a-665c-4c5c-9ae9-8af456f61e10 None None] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:30:57.963240 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39beef09-77e3-4634-bf50-9eb4157a234f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '42d15fc0515945acacbbffcf2356c7a5', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80245) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:30:57.965535 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39beef09-77e3-4634-bf50-9eb4157a234f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:30:57.966042 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39beef09-77e3-4634-bf50-9eb4157a234f admin admin] SCRIPT_NAME: `/identity` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:30:57.966488 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39beef09-77e3-4634-bf50-9eb4157a234f admin admin] PATH_INFO: `/v3/projects/f1bc6cc5583649de8d9149eaa4317ae3` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:30:57.966882 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a8b2d3a-665c-4c5c-9ae9-8af456f61e10 None None] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:30:57.967785 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39beef09-77e3-4634-bf50-9eb4157a234f admin admin] RBAC: Authorizing `identity:delete_project(project_id=f1bc6cc5583649de8d9149eaa4317ae3)` {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:30:57.969873 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39beef09-77e3-4634-bf50-9eb4157a234f admin admin] RBAC: Authorization granted {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:30:57.972164 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05b03622-5634-42b8-99ac-8970c6978f4d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '42d15fc0515945acacbbffcf2356c7a5', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:30:57.972830 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05b03622-5634-42b8-99ac-8970c6978f4d admin admin] REQUEST_METHOD: `GET` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:30:57.973110 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05b03622-5634-42b8-99ac-8970c6978f4d admin admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:30:57.973381 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05b03622-5634-42b8-99ac-8970c6978f4d admin admin] PATH_INFO: `/v3/domains` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:30:57.974110 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05b03622-5634-42b8-99ac-8970c6978f4d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80244) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 08:30:57.974414 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05b03622-5634-42b8-99ac-8970c6978f4d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:30:57.976053 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05b03622-5634-42b8-99ac-8970c6978f4d admin admin] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:30:57.991374 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 951/1887] 10.222.0.47 () {58 vars in 1152 bytes} [Thu Jul 21 08:30:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 08:30:57.992756 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05b03622-5634-42b8-99ac-8970c6978f4d admin admin] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:30:58.004062 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05b03622-5634-42b8-99ac-8970c6978f4d admin admin] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:30:58.009140 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba28afb4-779f-4c50-ab5e-06649d78bc03 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '42d15fc0515945acacbbffcf2356c7a5', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:30:58.009803 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba28afb4-779f-4c50-ab5e-06649d78bc03 admin admin] REQUEST_METHOD: `POST` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:30:58.010088 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba28afb4-779f-4c50-ab5e-06649d78bc03 admin admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:30:58.010463 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba28afb4-779f-4c50-ab5e-06649d78bc03 admin admin] PATH_INFO: `/v3/projects` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:30:58.011063 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba28afb4-779f-4c50-ab5e-06649d78bc03 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:30:58.013075 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba28afb4-779f-4c50-ab5e-06649d78bc03 admin admin] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:30:58.020286 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.notifications [None req-39beef09-77e3-4634-bf50-9eb4157a234f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f1bc6cc5583649de8d9149eaa4317ae3', 'request_id': 'req-39beef09-77e3-4634-bf50-9eb4157a234f'} {{(pid=80245) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 08:30:58.026442 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.notifications [None req-39beef09-77e3-4634-bf50-9eb4157a234f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f1bc6cc5583649de8d9149eaa4317ae3', 'request_id': 'req-39beef09-77e3-4634-bf50-9eb4157a234f'} {{(pid=80245) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 08:30:58.033262 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.notifications [None req-39beef09-77e3-4634-bf50-9eb4157a234f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f1bc6cc5583649de8d9149eaa4317ae3', 'request_id': 'req-39beef09-77e3-4634-bf50-9eb4157a234f'} {{(pid=80245) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 08:30:58.041561 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 952/1888] 10.222.0.47 () {60 vars in 1152 bytes} [Thu Jul 21 08:30:57 2022] POST /identity/v3/projects => generated 401 bytes in 49 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 21 08:30:58.046319 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.notifications [None req-39beef09-77e3-4634-bf50-9eb4157a234f admin admin] The token cache is being invalidate because project f1bc6cc5583649de8d9149eaa4317ae3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80245) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 21 08:30:58.046615 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.notifications [None req-39beef09-77e3-4634-bf50-9eb4157a234f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80245) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 08:30:58.047074 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba28afb4-779f-4c50-ab5e-06649d78bc03 admin admin] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:30:58.047279 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.notifications [None req-39beef09-77e3-4634-bf50-9eb4157a234f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80245) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 08:30:58.048827 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.fernet_utils [None req-ba28afb4-779f-4c50-ab5e-06649d78bc03 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80244) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 08:30:58.049124 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: [pid: 80245|app: 0|req: 937/1889] 10.222.0.47 () {58 vars in 1232 bytes} [Thu Jul 21 08:30:57 2022] DELETE /identity/v3/projects/f1bc6cc5583649de8d9149eaa4317ae3 => generated 0 bytes in 119 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 21 08:30:58.052841 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39beef09-77e3-4634-bf50-9eb4157a234f admin admin] Authenticating user token {{(pid=80245) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:30:58.054187 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.fernet_utils [None req-39beef09-77e3-4634-bf50-9eb4157a234f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80245) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 08:30:58.122060 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba28afb4-779f-4c50-ab5e-06649d78bc03 admin admin] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:30:58.131604 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-924d5768-b4f1-40d2-ac67-5d490f304d23 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '42d15fc0515945acacbbffcf2356c7a5', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:30:58.132673 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-924d5768-b4f1-40d2-ac67-5d490f304d23 admin admin] REQUEST_METHOD: `POST` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:30:58.133167 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-924d5768-b4f1-40d2-ac67-5d490f304d23 admin admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:30:58.133637 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-924d5768-b4f1-40d2-ac67-5d490f304d23 admin admin] PATH_INFO: `/v3/users` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:30:58.134746 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-924d5768-b4f1-40d2-ac67-5d490f304d23 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:30:58.137442 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-924d5768-b4f1-40d2-ac67-5d490f304d23 admin admin] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:30:58.145909 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39beef09-77e3-4634-bf50-9eb4157a234f admin admin] Validating token access rules against request {{(pid=80245) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:30:58.150979 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b31064a-baee-446a-9cf3-0aca94a28d15 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '42d15fc0515945acacbbffcf2356c7a5', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80245) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:30:58.152885 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b31064a-baee-446a-9cf3-0aca94a28d15 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:30:58.153295 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b31064a-baee-446a-9cf3-0aca94a28d15 admin admin] SCRIPT_NAME: `/identity` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:30:58.154136 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b31064a-baee-446a-9cf3-0aca94a28d15 admin admin] PATH_INFO: `/v3/users/4426ac2210594b1b910cd76aa2183df2` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:30:58.155485 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b31064a-baee-446a-9cf3-0aca94a28d15 admin admin] RBAC: Authorizing `identity:delete_user(user_id=4426ac2210594b1b910cd76aa2183df2)` {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:30:58.159474 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b31064a-baee-446a-9cf3-0aca94a28d15 admin admin] RBAC: Authorization granted {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:30:58.170866 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 953/1890] 10.222.0.47 () {60 vars in 1143 bytes} [Thu Jul 21 08:30:58 2022] POST /identity/v3/users => generated 301 bytes in 124 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 21 08:30:58.171944 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-924d5768-b4f1-40d2-ac67-5d490f304d23 admin admin] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:30:58.187986 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-924d5768-b4f1-40d2-ac67-5d490f304d23 admin admin] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:30:58.193686 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-465ed150-d82f-4603-9cd1-c0eb5c858323 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '42d15fc0515945acacbbffcf2356c7a5', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:30:58.193686 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-465ed150-d82f-4603-9cd1-c0eb5c858323 admin admin] REQUEST_METHOD: `GET` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:30:58.193686 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-465ed150-d82f-4603-9cd1-c0eb5c858323 admin admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:30:58.193686 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-465ed150-d82f-4603-9cd1-c0eb5c858323 admin admin] PATH_INFO: `/v3/roles` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:30:58.195084 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-465ed150-d82f-4603-9cd1-c0eb5c858323 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:30:58.197437 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-465ed150-d82f-4603-9cd1-c0eb5c858323 admin admin] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:30:58.210873 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 954/1891] 10.222.0.47 () {58 vars in 1121 bytes} [Thu Jul 21 08:30:58 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 08:30:58.215629 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-465ed150-d82f-4603-9cd1-c0eb5c858323 admin admin] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:30:58.229644 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-465ed150-d82f-4603-9cd1-c0eb5c858323 admin admin] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:30:58.235602 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.notifications [None req-8b31064a-baee-446a-9cf3-0aca94a28d15 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4426ac2210594b1b910cd76aa2183df2'} {{(pid=80245) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 08:30:58.235999 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0bcf80d-2cc3-4abc-a096-703710e6ec36 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '42d15fc0515945acacbbffcf2356c7a5', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:30:58.236678 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0bcf80d-2cc3-4abc-a096-703710e6ec36 admin admin] REQUEST_METHOD: `GET` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:30:58.236974 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0bcf80d-2cc3-4abc-a096-703710e6ec36 admin admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:30:58.237450 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0bcf80d-2cc3-4abc-a096-703710e6ec36 admin admin] PATH_INFO: `/v3/roles` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:30:58.240940 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0bcf80d-2cc3-4abc-a096-703710e6ec36 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:30:58.244851 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0bcf80d-2cc3-4abc-a096-703710e6ec36 admin admin] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:30:58.251837 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.notifications [None req-8b31064a-baee-446a-9cf3-0aca94a28d15 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4426ac2210594b1b910cd76aa2183df2'} {{(pid=80245) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 08:30:58.254618 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG dbcounter [-] [80244] Writing DB stats keystone:SELECT=88,keystone:DELETE=7,keystone:INSERT=5 {{(pid=80244) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:30:58.271522 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 955/1892] 10.222.0.47 () {58 vars in 1121 bytes} [Thu Jul 21 08:30:58 2022] GET /identity/v3/roles => generated 1420 bytes in 57 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 21 08:30:58.272736 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.notifications [None req-8b31064a-baee-446a-9cf3-0aca94a28d15 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4426ac2210594b1b910cd76aa2183df2'} {{(pid=80245) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 08:30:58.273837 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0bcf80d-2cc3-4abc-a096-703710e6ec36 admin admin] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:30:58.284141 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.notifications [None req-8b31064a-baee-446a-9cf3-0aca94a28d15 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4426ac2210594b1b910cd76aa2183df2'} {{(pid=80245) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 08:30:58.284870 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.notifications [None req-8b31064a-baee-446a-9cf3-0aca94a28d15 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4426ac2210594b1b910cd76aa2183df2'} {{(pid=80245) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 08:30:58.287649 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0bcf80d-2cc3-4abc-a096-703710e6ec36 admin admin] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:30:58.293026 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffe343f9-e1c1-45dd-adb9-3231086721f2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '42d15fc0515945acacbbffcf2356c7a5', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:30:58.293916 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffe343f9-e1c1-45dd-adb9-3231086721f2 admin admin] REQUEST_METHOD: `PUT` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:30:58.295033 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffe343f9-e1c1-45dd-adb9-3231086721f2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:30:58.295315 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffe343f9-e1c1-45dd-adb9-3231086721f2 admin admin] PATH_INFO: `/v3/projects/8b3c7dc4d5c1430fb25afe235c0de99e/users/ceb37aaecc4d44399cc1ec0e384cd4bb/roles/3a074738221e4601b5205715ba87d240` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:30:58.297342 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: [pid: 80245|app: 0|req: 938/1893] 10.222.0.47 () {58 vars in 1223 bytes} [Thu Jul 21 08:30:58 2022] DELETE /identity/v3/users/4426ac2210594b1b910cd76aa2183df2 => generated 0 bytes in 245 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 21 08:30:58.313826 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffe343f9-e1c1-45dd-adb9-3231086721f2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8b3c7dc4d5c1430fb25afe235c0de99e, user_id=ceb37aaecc4d44399cc1ec0e384cd4bb, role_id=3a074738221e4601b5205715ba87d240)` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:30:58.316684 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffe343f9-e1c1-45dd-adb9-3231086721f2 admin admin] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:30:58.343643 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 956/1894] 10.222.0.47 () {60 vars in 1482 bytes} [Thu Jul 21 08:30:58 2022] PUT /identity/v3/projects/8b3c7dc4d5c1430fb25afe235c0de99e/users/ceb37aaecc4d44399cc1ec0e384cd4bb/roles/3a074738221e4601b5205715ba87d240 => generated 0 bytes in 62 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 08:30:58.345454 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07511d5d-0423-412e-879c-d0e22fc3b760 None None] REQUEST_METHOD: `POST` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:30:58.345747 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07511d5d-0423-412e-879c-d0e22fc3b760 None None] SCRIPT_NAME: `/identity` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:30:58.346030 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07511d5d-0423-412e-879c-d0e22fc3b760 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:30:58.366618 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG dbcounter [-] [80245] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=80245) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:30:58.393152 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.auth.core [None req-07511d5d-0423-412e-879c-d0e22fc3b760 None None] MFA Rules not processed for user `ceb37aaecc4d44399cc1ec0e384cd4bb`. Rule list: `[]` (Enabled: `True`). {{(pid=80245) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 08:30:58.395820 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.fernet_utils [None req-07511d5d-0423-412e-879c-d0e22fc3b760 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80245) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 08:30:58.512524 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: [pid: 80245|app: 0|req: 939/1895] 10.222.0.47 () {58 vars in 957 bytes} [Thu Jul 21 08:30:58 2022] POST /identity/v3/auth/tokens => generated 3268 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 08:30:58.518553 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffe343f9-e1c1-45dd-adb9-3231086721f2 admin admin] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:30:58.520913 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.fernet_utils [None req-ffe343f9-e1c1-45dd-adb9-3231086721f2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80244) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 08:30:58.620325 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffe343f9-e1c1-45dd-adb9-3231086721f2 admin admin] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:30:58.627042 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eae50f61-45dd-4468-8d58-8f00628f1072 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '42d15fc0515945acacbbffcf2356c7a5', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:30:58.627809 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eae50f61-45dd-4468-8d58-8f00628f1072 admin admin] REQUEST_METHOD: `POST` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:30:58.628102 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eae50f61-45dd-4468-8d58-8f00628f1072 admin admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:30:58.628345 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eae50f61-45dd-4468-8d58-8f00628f1072 admin admin] PATH_INFO: `/v3/projects` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:30:58.629047 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eae50f61-45dd-4468-8d58-8f00628f1072 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:30:58.632524 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eae50f61-45dd-4468-8d58-8f00628f1072 admin admin] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:30:58.682044 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 957/1896] 10.222.0.47 () {60 vars in 1152 bytes} [Thu Jul 21 08:30:58 2022] POST /identity/v3/projects => generated 401 bytes in 162 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 08:30:58.683513 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07511d5d-0423-412e-879c-d0e22fc3b760 None None] Authenticating user token {{(pid=80245) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:30:58.697258 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07511d5d-0423-412e-879c-d0e22fc3b760 None None] Validating token access rules against request {{(pid=80245) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:30:58.732846 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eae50f61-45dd-4468-8d58-8f00628f1072 admin admin] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:30:58.735339 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.fernet_utils [None req-eae50f61-45dd-4468-8d58-8f00628f1072 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80244) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 08:30:58.772410 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2ab7ec2-bcf3-413c-9171-b2400040990a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '42d15fc0515945acacbbffcf2356c7a5', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80245) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:30:58.773471 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2ab7ec2-bcf3-413c-9171-b2400040990a admin admin] REQUEST_METHOD: `POST` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:30:58.773757 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2ab7ec2-bcf3-413c-9171-b2400040990a admin admin] SCRIPT_NAME: `/identity` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:30:58.774033 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2ab7ec2-bcf3-413c-9171-b2400040990a admin admin] PATH_INFO: `/v3/users` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:30:58.774863 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2ab7ec2-bcf3-413c-9171-b2400040990a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:30:58.777711 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2ab7ec2-bcf3-413c-9171-b2400040990a admin admin] RBAC: Authorization granted {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:30:58.799749 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: [pid: 80245|app: 0|req: 940/1897] 10.222.0.47 () {60 vars in 1143 bytes} [Thu Jul 21 08:30:58 2022] POST /identity/v3/users => generated 307 bytes in 117 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 08:30:58.803533 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2ab7ec2-bcf3-413c-9171-b2400040990a admin admin] Authenticating user token {{(pid=80245) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:30:58.813606 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2ab7ec2-bcf3-413c-9171-b2400040990a admin admin] Validating token access rules against request {{(pid=80245) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:30:58.814250 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eae50f61-45dd-4468-8d58-8f00628f1072 admin admin] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:30:58.818220 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43a8305f-a3bd-4254-808a-d8db0c4459d3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '42d15fc0515945acacbbffcf2356c7a5', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80245) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:30:58.818878 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43a8305f-a3bd-4254-808a-d8db0c4459d3 admin admin] REQUEST_METHOD: `GET` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:30:58.818982 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-607397c6-0d72-42d8-a702-128d0e5f4ffd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '42d15fc0515945acacbbffcf2356c7a5', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:30:58.819160 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43a8305f-a3bd-4254-808a-d8db0c4459d3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:30:58.819488 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43a8305f-a3bd-4254-808a-d8db0c4459d3 admin admin] PATH_INFO: `/v3/roles` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:30:58.820045 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-607397c6-0d72-42d8-a702-128d0e5f4ffd admin admin] REQUEST_METHOD: `DELETE` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:30:58.820206 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43a8305f-a3bd-4254-808a-d8db0c4459d3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:30:58.820557 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-607397c6-0d72-42d8-a702-128d0e5f4ffd admin admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:30:58.820917 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-607397c6-0d72-42d8-a702-128d0e5f4ffd admin admin] PATH_INFO: `/v3/projects/ea6fdf0407e4492a9080992904579562` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:30:58.821841 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43a8305f-a3bd-4254-808a-d8db0c4459d3 admin admin] RBAC: Authorization granted {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:30:58.822231 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-607397c6-0d72-42d8-a702-128d0e5f4ffd admin admin] RBAC: Authorizing `identity:delete_project(project_id=ea6fdf0407e4492a9080992904579562)` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:30:58.824231 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-607397c6-0d72-42d8-a702-128d0e5f4ffd admin admin] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:30:58.835285 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: [pid: 80245|app: 0|req: 941/1898] 10.222.0.47 () {58 vars in 1121 bytes} [Thu Jul 21 08:30:58 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 08:30:58.836876 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43a8305f-a3bd-4254-808a-d8db0c4459d3 admin admin] Authenticating user token {{(pid=80245) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:30:58.849352 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43a8305f-a3bd-4254-808a-d8db0c4459d3 admin admin] Validating token access rules against request {{(pid=80245) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:30:58.854724 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-360512d9-96ab-424b-b853-74ab42c5fb86 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '42d15fc0515945acacbbffcf2356c7a5', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80245) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:30:58.855896 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-360512d9-96ab-424b-b853-74ab42c5fb86 admin admin] REQUEST_METHOD: `PUT` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:30:58.856359 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-360512d9-96ab-424b-b853-74ab42c5fb86 admin admin] SCRIPT_NAME: `/identity` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:30:58.856724 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-360512d9-96ab-424b-b853-74ab42c5fb86 admin admin] PATH_INFO: `/v3/projects/b31f7a328f9b46728a64a50c1c6da9de/users/a6bb5dd4e940419ab1efb69d34bdcd0c/roles/e29870f6e9e2411a9db0d5bf45f32ca5` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:30:58.879588 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-360512d9-96ab-424b-b853-74ab42c5fb86 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b31f7a328f9b46728a64a50c1c6da9de, user_id=a6bb5dd4e940419ab1efb69d34bdcd0c, role_id=e29870f6e9e2411a9db0d5bf45f32ca5)` {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:30:58.879982 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-360512d9-96ab-424b-b853-74ab42c5fb86 admin admin] RBAC: Authorization granted {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:30:58.882049 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.notifications [None req-607397c6-0d72-42d8-a702-128d0e5f4ffd admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ea6fdf0407e4492a9080992904579562', 'request_id': 'req-607397c6-0d72-42d8-a702-128d0e5f4ffd'} {{(pid=80244) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 08:30:58.891698 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.notifications [None req-607397c6-0d72-42d8-a702-128d0e5f4ffd admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ea6fdf0407e4492a9080992904579562', 'request_id': 'req-607397c6-0d72-42d8-a702-128d0e5f4ffd'} {{(pid=80244) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 08:30:58.898408 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.notifications [None req-607397c6-0d72-42d8-a702-128d0e5f4ffd admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ea6fdf0407e4492a9080992904579562', 'request_id': 'req-607397c6-0d72-42d8-a702-128d0e5f4ffd'} {{(pid=80244) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 08:30:58.899588 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: [pid: 80245|app: 0|req: 942/1899] 10.222.0.47 () {60 vars in 1482 bytes} [Thu Jul 21 08:30:58 2022] PUT /identity/v3/projects/b31f7a328f9b46728a64a50c1c6da9de/users/a6bb5dd4e940419ab1efb69d34bdcd0c/roles/e29870f6e9e2411a9db0d5bf45f32ca5 => generated 0 bytes in 63 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 21 08:30:58.903605 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG dbcounter [-] [80244] Writing DB stats keystone:SELECT=91,keystone:INSERT=3,keystone:DELETE=6 {{(pid=80244) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:30:58.904317 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-360512d9-96ab-424b-b853-74ab42c5fb86 admin admin] Authenticating user token {{(pid=80245) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:30:58.917880 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.notifications [None req-607397c6-0d72-42d8-a702-128d0e5f4ffd admin admin] The token cache is being invalidate because project ea6fdf0407e4492a9080992904579562 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80244) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 21 08:30:58.918311 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.notifications [None req-607397c6-0d72-42d8-a702-128d0e5f4ffd admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80244) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 08:30:58.918923 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.notifications [None req-607397c6-0d72-42d8-a702-128d0e5f4ffd admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80244) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 08:30:58.921655 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 958/1900] 10.222.0.47 () {58 vars in 1232 bytes} [Thu Jul 21 08:30:58 2022] DELETE /identity/v3/projects/ea6fdf0407e4492a9080992904579562 => generated 0 bytes in 190 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 08:30:58.925590 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-607397c6-0d72-42d8-a702-128d0e5f4ffd admin admin] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:30:58.927783 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.fernet_utils [None req-607397c6-0d72-42d8-a702-128d0e5f4ffd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80244) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 08:30:58.967401 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG dbcounter [-] [80245] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80245) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:30:58.980832 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-360512d9-96ab-424b-b853-74ab42c5fb86 admin admin] Validating token access rules against request {{(pid=80245) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:30:58.986813 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c2533af-e8a5-480a-9148-db6be2dfb67f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '42d15fc0515945acacbbffcf2356c7a5', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80245) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:30:58.987722 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c2533af-e8a5-480a-9148-db6be2dfb67f admin admin] REQUEST_METHOD: `GET` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:30:58.988110 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c2533af-e8a5-480a-9148-db6be2dfb67f admin admin] SCRIPT_NAME: `/identity` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:30:58.989167 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c2533af-e8a5-480a-9148-db6be2dfb67f admin admin] PATH_INFO: `/v3/roles` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:30:58.989771 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c2533af-e8a5-480a-9148-db6be2dfb67f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:30:58.991568 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c2533af-e8a5-480a-9148-db6be2dfb67f admin admin] RBAC: Authorization granted {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:30:59.007276 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: [pid: 80245|app: 0|req: 943/1901] 10.222.0.47 () {58 vars in 1121 bytes} [Thu Jul 21 08:30:58 2022] GET /identity/v3/roles => generated 1420 bytes in 104 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 08:30:59.009274 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-607397c6-0d72-42d8-a702-128d0e5f4ffd admin admin] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:30:59.012073 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c2533af-e8a5-480a-9148-db6be2dfb67f admin admin] Authenticating user token {{(pid=80245) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:30:59.015344 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.fernet_utils [None req-5c2533af-e8a5-480a-9148-db6be2dfb67f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80245) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 08:30:59.017853 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aecca92-e406-4867-b2dd-3592d72423d2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '42d15fc0515945acacbbffcf2356c7a5', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:30:59.019508 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aecca92-e406-4867-b2dd-3592d72423d2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:30:59.019957 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aecca92-e406-4867-b2dd-3592d72423d2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:30:59.020238 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aecca92-e406-4867-b2dd-3592d72423d2 admin admin] PATH_INFO: `/v3/users/f554b9336b00415c82aa32414c516d78` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:30:59.022077 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6aecca92-e406-4867-b2dd-3592d72423d2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f554b9336b00415c82aa32414c516d78)` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:30:59.023590 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6aecca92-e406-4867-b2dd-3592d72423d2 admin admin] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:30:59.031148 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c2533af-e8a5-480a-9148-db6be2dfb67f admin admin] Validating token access rules against request {{(pid=80245) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:30:59.036857 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92002091-e585-4c27-85f0-c17689da2598 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '42d15fc0515945acacbbffcf2356c7a5', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80245) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:30:59.037465 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92002091-e585-4c27-85f0-c17689da2598 admin admin] REQUEST_METHOD: `PUT` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:30:59.037754 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92002091-e585-4c27-85f0-c17689da2598 admin admin] SCRIPT_NAME: `/identity` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:30:59.038035 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92002091-e585-4c27-85f0-c17689da2598 admin admin] PATH_INFO: `/v3/projects/b31f7a328f9b46728a64a50c1c6da9de/users/a6bb5dd4e940419ab1efb69d34bdcd0c/roles/e29870f6e9e2411a9db0d5bf45f32ca5` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:30:59.040339 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92002091-e585-4c27-85f0-c17689da2598 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b31f7a328f9b46728a64a50c1c6da9de, user_id=a6bb5dd4e940419ab1efb69d34bdcd0c, role_id=e29870f6e9e2411a9db0d5bf45f32ca5)` {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:30:59.043268 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92002091-e585-4c27-85f0-c17689da2598 admin admin] RBAC: Authorization granted {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:30:59.056491 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: [pid: 80245|app: 0|req: 944/1902] 10.222.0.47 () {60 vars in 1482 bytes} [Thu Jul 21 08:30:59 2022] PUT /identity/v3/projects/b31f7a328f9b46728a64a50c1c6da9de/users/a6bb5dd4e940419ab1efb69d34bdcd0c/roles/e29870f6e9e2411a9db0d5bf45f32ca5 => generated 0 bytes in 45 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 21 08:30:59.069360 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ed91593-60d7-45f6-8664-8ace6b38f762 None None] REQUEST_METHOD: `POST` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:30:59.069677 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ed91593-60d7-45f6-8664-8ace6b38f762 None None] SCRIPT_NAME: `/identity` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:30:59.069961 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ed91593-60d7-45f6-8664-8ace6b38f762 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:30:59.080160 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.notifications [None req-6aecca92-e406-4867-b2dd-3592d72423d2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f554b9336b00415c82aa32414c516d78'} {{(pid=80244) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 08:30:59.089008 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.notifications [None req-6aecca92-e406-4867-b2dd-3592d72423d2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f554b9336b00415c82aa32414c516d78'} {{(pid=80244) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 08:30:59.098216 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.auth.core [None req-3ed91593-60d7-45f6-8664-8ace6b38f762 None None] MFA Rules not processed for user `a6bb5dd4e940419ab1efb69d34bdcd0c`. Rule list: `[]` (Enabled: `True`). {{(pid=80245) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 08:30:59.099460 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.fernet_utils [None req-3ed91593-60d7-45f6-8664-8ace6b38f762 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80245) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 08:30:59.101696 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.notifications [None req-6aecca92-e406-4867-b2dd-3592d72423d2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f554b9336b00415c82aa32414c516d78'} {{(pid=80244) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 08:30:59.102349 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.notifications [None req-6aecca92-e406-4867-b2dd-3592d72423d2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f554b9336b00415c82aa32414c516d78'} {{(pid=80244) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 08:30:59.109776 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.notifications [None req-6aecca92-e406-4867-b2dd-3592d72423d2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f554b9336b00415c82aa32414c516d78'} {{(pid=80244) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 08:30:59.116050 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 959/1903] 10.222.0.47 () {58 vars in 1223 bytes} [Thu Jul 21 08:30:58 2022] DELETE /identity/v3/users/f554b9336b00415c82aa32414c516d78 => generated 0 bytes in 191 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 08:30:59.173036 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: [pid: 80245|app: 0|req: 945/1904] 10.222.0.47 () {58 vars in 957 bytes} [Thu Jul 21 08:30:59 2022] POST /identity/v3/auth/tokens => generated 3335 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 21 08:30:59.185376 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aecca92-e406-4867-b2dd-3592d72423d2 admin admin] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:30:59.187226 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.fernet_utils [None req-6aecca92-e406-4867-b2dd-3592d72423d2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80244) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 08:30:59.256364 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aecca92-e406-4867-b2dd-3592d72423d2 admin admin] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:30:59.260061 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7d0e3d1-09d5-4405-9878-16dd55897858 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4f40410b7674687ab5362ac982248ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4faef13aa50e4e36b6c265fb1398fa14', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:30:59.260783 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7d0e3d1-09d5-4405-9878-16dd55897858 service cinder] REQUEST_METHOD: `GET` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:30:59.261006 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7d0e3d1-09d5-4405-9878-16dd55897858 service cinder] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:30:59.261352 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7d0e3d1-09d5-4405-9878-16dd55897858 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:30:59.290762 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG dbcounter [-] [80244] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=80244) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:30:59.350566 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7d0e3d1-09d5-4405-9878-16dd55897858 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:30:59.353388 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7d0e3d1-09d5-4405-9878-16dd55897858 service cinder] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:30:59.369698 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 960/1905] 10.222.0.47 () {60 vars in 1365 bytes} [Thu Jul 21 08:30:59 2022] GET /identity/v3/auth/tokens => generated 3335 bytes in 185 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 08:30:59.424981 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ed91593-60d7-45f6-8664-8ace6b38f762 None None] Authenticating user token {{(pid=80245) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:30:59.426677 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.fernet_utils [None req-3ed91593-60d7-45f6-8664-8ace6b38f762 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80245) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 08:30:59.469484 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7d0e3d1-09d5-4405-9878-16dd55897858 service cinder] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:30:59.480405 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7d0e3d1-09d5-4405-9878-16dd55897858 service cinder] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:30:59.484801 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8191f74-ad3f-4cbb-a1cf-faeeb09bcd66 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4f40410b7674687ab5362ac982248ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4faef13aa50e4e36b6c265fb1398fa14', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:30:59.485754 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8191f74-ad3f-4cbb-a1cf-faeeb09bcd66 service cinder] REQUEST_METHOD: `GET` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:30:59.486088 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8191f74-ad3f-4cbb-a1cf-faeeb09bcd66 service cinder] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:30:59.486369 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8191f74-ad3f-4cbb-a1cf-faeeb09bcd66 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:30:59.496360 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.fernet_utils [None req-d8191f74-ad3f-4cbb-a1cf-faeeb09bcd66 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80244) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 08:30:59.535697 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ed91593-60d7-45f6-8664-8ace6b38f762 None None] Validating token access rules against request {{(pid=80245) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:30:59.561933 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8191f74-ad3f-4cbb-a1cf-faeeb09bcd66 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:30:59.563400 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8191f74-ad3f-4cbb-a1cf-faeeb09bcd66 service cinder] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:30:59.572528 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 961/1906] 10.222.0.47 () {60 vars in 1365 bytes} [Thu Jul 21 08:30:59 2022] GET /identity/v3/auth/tokens => generated 3268 bytes in 104 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 08:30:59.593197 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02334582-344f-4ed3-aadb-54385578c6f5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '42d15fc0515945acacbbffcf2356c7a5', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80245) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:30:59.594167 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02334582-344f-4ed3-aadb-54385578c6f5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:30:59.594359 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02334582-344f-4ed3-aadb-54385578c6f5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:30:59.594637 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02334582-344f-4ed3-aadb-54385578c6f5 admin admin] PATH_INFO: `/v3/projects/b0c086d149ca495783890b4200ccfdc2` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:30:59.595871 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02334582-344f-4ed3-aadb-54385578c6f5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b0c086d149ca495783890b4200ccfdc2)` {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:30:59.597789 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02334582-344f-4ed3-aadb-54385578c6f5 admin admin] RBAC: Authorization granted {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:30:59.607462 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG dbcounter [-] [80245] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80245) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:30:59.659231 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.notifications [None req-02334582-344f-4ed3-aadb-54385578c6f5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b0c086d149ca495783890b4200ccfdc2', 'request_id': 'req-02334582-344f-4ed3-aadb-54385578c6f5'} {{(pid=80245) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 08:30:59.666207 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.notifications [None req-02334582-344f-4ed3-aadb-54385578c6f5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b0c086d149ca495783890b4200ccfdc2', 'request_id': 'req-02334582-344f-4ed3-aadb-54385578c6f5'} {{(pid=80245) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 08:30:59.674585 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.notifications [None req-02334582-344f-4ed3-aadb-54385578c6f5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b0c086d149ca495783890b4200ccfdc2', 'request_id': 'req-02334582-344f-4ed3-aadb-54385578c6f5'} {{(pid=80245) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 08:30:59.692149 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.notifications [None req-02334582-344f-4ed3-aadb-54385578c6f5 admin admin] The token cache is being invalidate because project b0c086d149ca495783890b4200ccfdc2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80245) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 21 08:30:59.692857 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.notifications [None req-02334582-344f-4ed3-aadb-54385578c6f5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80245) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 08:30:59.693930 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.notifications [None req-02334582-344f-4ed3-aadb-54385578c6f5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80245) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 08:30:59.695753 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: [pid: 80245|app: 0|req: 946/1907] 10.222.0.47 () {58 vars in 1232 bytes} [Thu Jul 21 08:30:59 2022] DELETE /identity/v3/projects/b0c086d149ca495783890b4200ccfdc2 => generated 0 bytes in 272 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 08:30:59.701994 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df67b45d-df6a-4fd4-8fab-43fbcc701e43 None None] REQUEST_METHOD: `POST` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:30:59.702484 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df67b45d-df6a-4fd4-8fab-43fbcc701e43 None None] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:30:59.702966 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df67b45d-df6a-4fd4-8fab-43fbcc701e43 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:30:59.732434 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.auth.core [None req-df67b45d-df6a-4fd4-8fab-43fbcc701e43 None None] MFA Rules not processed for user `2dce40b8d74c41ec94cc36e6ff05d412`. Rule list: `[]` (Enabled: `True`). {{(pid=80244) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 08:30:59.733675 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.fernet_utils [None req-df67b45d-df6a-4fd4-8fab-43fbcc701e43 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80244) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 08:30:59.805671 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 962/1908] 10.222.0.47 () {58 vars in 957 bytes} [Thu Jul 21 08:30:59 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 21 08:30:59.814815 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d7b2e89-41e2-4f2c-ac75-17935a87f5bb None None] REQUEST_METHOD: `POST` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:30:59.815352 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d7b2e89-41e2-4f2c-ac75-17935a87f5bb None None] SCRIPT_NAME: `/identity` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:30:59.815463 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d7b2e89-41e2-4f2c-ac75-17935a87f5bb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:30:59.838769 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.auth.core [None req-0d7b2e89-41e2-4f2c-ac75-17935a87f5bb None None] MFA Rules not processed for user `2dce40b8d74c41ec94cc36e6ff05d412`. Rule list: `[]` (Enabled: `True`). {{(pid=80245) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 08:30:59.840050 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.fernet_utils [None req-0d7b2e89-41e2-4f2c-ac75-17935a87f5bb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80245) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 08:30:59.849049 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: [pid: 80245|app: 0|req: 947/1909] 10.222.0.47 () {58 vars in 957 bytes} [Thu Jul 21 08:30:59 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 08:30:59.853322 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df67b45d-df6a-4fd4-8fab-43fbcc701e43 None None] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:30:59.864327 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df67b45d-df6a-4fd4-8fab-43fbcc701e43 None None] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:30:59.870334 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27e01779-ae54-47c0-bb30-2b68120dd813 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '42d15fc0515945acacbbffcf2356c7a5', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:30:59.871018 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27e01779-ae54-47c0-bb30-2b68120dd813 admin admin] REQUEST_METHOD: `GET` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:30:59.871396 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27e01779-ae54-47c0-bb30-2b68120dd813 admin admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:30:59.871789 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27e01779-ae54-47c0-bb30-2b68120dd813 admin admin] PATH_INFO: `/v3/domains` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:30:59.872603 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27e01779-ae54-47c0-bb30-2b68120dd813 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80244) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 08:30:59.872918 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27e01779-ae54-47c0-bb30-2b68120dd813 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:30:59.874878 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27e01779-ae54-47c0-bb30-2b68120dd813 admin admin] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:30:59.891509 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 963/1910] 10.222.0.47 () {58 vars in 1152 bytes} [Thu Jul 21 08:30:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 08:30:59.895100 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d7b2e89-41e2-4f2c-ac75-17935a87f5bb None None] Authenticating user token {{(pid=80245) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:30:59.906043 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d7b2e89-41e2-4f2c-ac75-17935a87f5bb None None] Validating token access rules against request {{(pid=80245) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:30:59.916722 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55f051ed-82c5-4bf7-888b-8e2a837445f6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '42d15fc0515945acacbbffcf2356c7a5', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80245) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:30:59.916722 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55f051ed-82c5-4bf7-888b-8e2a837445f6 admin admin] REQUEST_METHOD: `POST` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:30:59.916722 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55f051ed-82c5-4bf7-888b-8e2a837445f6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:30:59.916722 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55f051ed-82c5-4bf7-888b-8e2a837445f6 admin admin] PATH_INFO: `/v3/projects` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:30:59.917031 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55f051ed-82c5-4bf7-888b-8e2a837445f6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:30:59.917590 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55f051ed-82c5-4bf7-888b-8e2a837445f6 admin admin] RBAC: Authorization granted {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:30:59.947666 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: [pid: 80245|app: 0|req: 948/1911] 10.222.0.47 () {60 vars in 1152 bytes} [Thu Jul 21 08:30:59 2022] POST /identity/v3/projects => generated 385 bytes in 50 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 21 08:30:59.948982 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27e01779-ae54-47c0-bb30-2b68120dd813 admin admin] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:30:59.960756 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27e01779-ae54-47c0-bb30-2b68120dd813 admin admin] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:31:00.005407 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG dbcounter [-] [80244] Writing DB stats keystone:SELECT=100 {{(pid=80244) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:31:00.020059 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4305c3a-784a-441b-8ad8-927ab7ab1d11 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '42d15fc0515945acacbbffcf2356c7a5', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:31:00.020743 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4305c3a-784a-441b-8ad8-927ab7ab1d11 admin admin] REQUEST_METHOD: `POST` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:31:00.021031 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4305c3a-784a-441b-8ad8-927ab7ab1d11 admin admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:31:00.021307 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4305c3a-784a-441b-8ad8-927ab7ab1d11 admin admin] PATH_INFO: `/v3/users` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:31:00.022019 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4305c3a-784a-441b-8ad8-927ab7ab1d11 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:31:00.023994 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4305c3a-784a-441b-8ad8-927ab7ab1d11 admin admin] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:31:00.054684 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 964/1912] 10.222.0.47 () {60 vars in 1143 bytes} [Thu Jul 21 08:30:59 2022] POST /identity/v3/users => generated 293 bytes in 103 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 08:31:00.056165 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55f051ed-82c5-4bf7-888b-8e2a837445f6 admin admin] Authenticating user token {{(pid=80245) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:31:00.066244 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55f051ed-82c5-4bf7-888b-8e2a837445f6 admin admin] Validating token access rules against request {{(pid=80245) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:31:00.070933 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9193d406-d8d8-47c1-9a26-fa0945535389 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '42d15fc0515945acacbbffcf2356c7a5', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80245) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:31:00.071620 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9193d406-d8d8-47c1-9a26-fa0945535389 admin admin] REQUEST_METHOD: `GET` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:31:00.071903 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9193d406-d8d8-47c1-9a26-fa0945535389 admin admin] SCRIPT_NAME: `/identity` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:31:00.072174 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9193d406-d8d8-47c1-9a26-fa0945535389 admin admin] PATH_INFO: `/v3/roles` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:31:00.072889 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9193d406-d8d8-47c1-9a26-fa0945535389 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:31:00.074497 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9193d406-d8d8-47c1-9a26-fa0945535389 admin admin] RBAC: Authorization granted {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:31:00.085802 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: [pid: 80245|app: 0|req: 949/1913] 10.222.0.47 () {58 vars in 1121 bytes} [Thu Jul 21 08:31:00 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 08:31:00.089911 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4305c3a-784a-441b-8ad8-927ab7ab1d11 admin admin] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:31:00.100505 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4305c3a-784a-441b-8ad8-927ab7ab1d11 admin admin] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:31:00.105438 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65917466-91af-4a68-a676-7138cf91d874 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '42d15fc0515945acacbbffcf2356c7a5', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:31:00.106114 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65917466-91af-4a68-a676-7138cf91d874 admin admin] REQUEST_METHOD: `GET` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:31:00.106393 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65917466-91af-4a68-a676-7138cf91d874 admin admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:31:00.106712 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65917466-91af-4a68-a676-7138cf91d874 admin admin] PATH_INFO: `/v3/roles` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:31:00.107473 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65917466-91af-4a68-a676-7138cf91d874 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:31:00.109078 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65917466-91af-4a68-a676-7138cf91d874 admin admin] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:31:00.119784 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 965/1914] 10.222.0.47 () {58 vars in 1121 bytes} [Thu Jul 21 08:31:00 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 08:31:00.124169 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9193d406-d8d8-47c1-9a26-fa0945535389 admin admin] Authenticating user token {{(pid=80245) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:31:00.134666 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9193d406-d8d8-47c1-9a26-fa0945535389 admin admin] Validating token access rules against request {{(pid=80245) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:31:00.140587 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20e2145b-933d-4226-b9a3-fbe802b3cee7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '42d15fc0515945acacbbffcf2356c7a5', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80245) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:31:00.141499 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20e2145b-933d-4226-b9a3-fbe802b3cee7 admin admin] REQUEST_METHOD: `PUT` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:31:00.142552 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20e2145b-933d-4226-b9a3-fbe802b3cee7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:31:00.142552 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20e2145b-933d-4226-b9a3-fbe802b3cee7 admin admin] PATH_INFO: `/v3/projects/ddc31a2c572c41efa7b0e6661c39c7c5/users/9eb3e65a38bd4f52a2c015a190b52dd6/roles/3a074738221e4601b5205715ba87d240` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:31:00.160300 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20e2145b-933d-4226-b9a3-fbe802b3cee7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ddc31a2c572c41efa7b0e6661c39c7c5, user_id=9eb3e65a38bd4f52a2c015a190b52dd6, role_id=3a074738221e4601b5205715ba87d240)` {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:31:00.163697 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20e2145b-933d-4226-b9a3-fbe802b3cee7 admin admin] RBAC: Authorization granted {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:31:00.187520 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: [pid: 80245|app: 0|req: 950/1915] 10.222.0.47 () {60 vars in 1482 bytes} [Thu Jul 21 08:31:00 2022] PUT /identity/v3/projects/ddc31a2c572c41efa7b0e6661c39c7c5/users/9eb3e65a38bd4f52a2c015a190b52dd6/roles/3a074738221e4601b5205715ba87d240 => generated 0 bytes in 57 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 21 08:31:00.192666 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1fe9de2-2099-4474-96aa-aa3a23f13b3b None None] REQUEST_METHOD: `POST` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:31:00.192963 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1fe9de2-2099-4474-96aa-aa3a23f13b3b None None] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:31:00.193250 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1fe9de2-2099-4474-96aa-aa3a23f13b3b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:31:00.223087 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.auth.core [None req-a1fe9de2-2099-4474-96aa-aa3a23f13b3b None None] MFA Rules not processed for user `9eb3e65a38bd4f52a2c015a190b52dd6`. Rule list: `[]` (Enabled: `True`). {{(pid=80244) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 08:31:00.224548 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.fernet_utils [None req-a1fe9de2-2099-4474-96aa-aa3a23f13b3b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80244) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 08:31:00.308870 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 966/1916] 10.222.0.47 () {58 vars in 957 bytes} [Thu Jul 21 08:31:00 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 08:31:00.320922 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20e2145b-933d-4226-b9a3-fbe802b3cee7 admin admin] Authenticating user token {{(pid=80245) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:31:00.322751 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.fernet_utils [None req-20e2145b-933d-4226-b9a3-fbe802b3cee7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80245) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 08:31:00.402590 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20e2145b-933d-4226-b9a3-fbe802b3cee7 admin admin] Validating token access rules against request {{(pid=80245) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:31:00.406457 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94e27fa6-d138-4461-a754-0618e6b3c56e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4f40410b7674687ab5362ac982248ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4faef13aa50e4e36b6c265fb1398fa14', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80245) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:31:00.407113 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94e27fa6-d138-4461-a754-0618e6b3c56e service cinder] REQUEST_METHOD: `GET` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:31:00.407489 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94e27fa6-d138-4461-a754-0618e6b3c56e service cinder] SCRIPT_NAME: `/identity` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:31:00.407767 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94e27fa6-d138-4461-a754-0618e6b3c56e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:31:00.416559 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94e27fa6-d138-4461-a754-0618e6b3c56e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:31:00.418606 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94e27fa6-d138-4461-a754-0618e6b3c56e service cinder] RBAC: Authorization granted {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:31:00.428015 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: [pid: 80245|app: 0|req: 951/1917] 10.222.0.47 () {60 vars in 1365 bytes} [Thu Jul 21 08:31:00 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 108 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 08:31:03.692368 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ce7553f-7831-4193-a619-ed71853a4bb6 None None] REQUEST_METHOD: `GET` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:31:03.692895 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ce7553f-7831-4193-a619-ed71853a4bb6 None None] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:31:03.693209 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ce7553f-7831-4193-a619-ed71853a4bb6 None None] PATH_INFO: `/` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:31:03.695234 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 967/1918] 10.222.0.47 () {54 vars in 921 bytes} [Thu Jul 21 08:31:03 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 08:31:03.707548 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51d3f299-87a1-4dff-97f5-da14502dfa48 None None] REQUEST_METHOD: `POST` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:31:03.708063 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51d3f299-87a1-4dff-97f5-da14502dfa48 None None] SCRIPT_NAME: `/identity` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:31:03.708267 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51d3f299-87a1-4dff-97f5-da14502dfa48 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:31:03.736106 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.auth.core [None req-51d3f299-87a1-4dff-97f5-da14502dfa48 None None] MFA Rules not processed for user `ac2b5b99a36442148fe743269054e991`. Rule list: `[]` (Enabled: `True`). {{(pid=80245) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 08:31:03.737703 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.fernet_utils [None req-51d3f299-87a1-4dff-97f5-da14502dfa48 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80245) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 08:31:03.789471 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG dbcounter [-] [80245] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80245) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:31:03.836040 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: [pid: 80245|app: 0|req: 952/1919] 10.222.0.47 () {58 vars in 1018 bytes} [Thu Jul 21 08:31:03 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 08:31:03.849000 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ce7553f-7831-4193-a619-ed71853a4bb6 None None] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:31:03.850882 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.fernet_utils [None req-3ce7553f-7831-4193-a619-ed71853a4bb6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80244) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 08:31:03.870383 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ce7553f-7831-4193-a619-ed71853a4bb6 None None] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:31:03.876447 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7c2ef74-592f-4193-863d-aff4bf897aca service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2b5b99a36442148fe743269054e991', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4faef13aa50e4e36b6c265fb1398fa14', 'project_domain_id': 'default', 'roles': ['member', 'service', 'ResellerAdmin', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:31:03.878582 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7c2ef74-592f-4193-863d-aff4bf897aca service nova] REQUEST_METHOD: `GET` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:31:03.878582 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7c2ef74-592f-4193-863d-aff4bf897aca service nova] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:31:03.878582 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7c2ef74-592f-4193-863d-aff4bf897aca service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:31:03.892785 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7c2ef74-592f-4193-863d-aff4bf897aca service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:31:03.894451 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7c2ef74-592f-4193-863d-aff4bf897aca service nova] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:31:03.907676 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 968/1920] 10.222.0.47 () {60 vars in 1365 bytes} [Thu Jul 21 08:31:03 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 59 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 08:31:05.175448 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51d3f299-87a1-4dff-97f5-da14502dfa48 None None] Authenticating user token {{(pid=80245) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:31:05.239283 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51d3f299-87a1-4dff-97f5-da14502dfa48 None None] Validating token access rules against request {{(pid=80245) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:31:05.244243 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2049327-63e5-435b-bb5c-623f5a6a333f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '42d15fc0515945acacbbffcf2356c7a5', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80245) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:31:05.245136 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2049327-63e5-435b-bb5c-623f5a6a333f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:31:05.245417 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2049327-63e5-435b-bb5c-623f5a6a333f admin admin] SCRIPT_NAME: `/identity` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:31:05.245687 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2049327-63e5-435b-bb5c-623f5a6a333f admin admin] PATH_INFO: `/v3/users/9eb3e65a38bd4f52a2c015a190b52dd6` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:31:05.246909 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2049327-63e5-435b-bb5c-623f5a6a333f admin admin] RBAC: Authorizing `identity:delete_user(user_id=9eb3e65a38bd4f52a2c015a190b52dd6)` {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:31:05.248910 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2049327-63e5-435b-bb5c-623f5a6a333f admin admin] RBAC: Authorization granted {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:31:05.295221 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.notifications [None req-e2049327-63e5-435b-bb5c-623f5a6a333f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9eb3e65a38bd4f52a2c015a190b52dd6'} {{(pid=80245) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 08:31:05.304579 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.notifications [None req-e2049327-63e5-435b-bb5c-623f5a6a333f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9eb3e65a38bd4f52a2c015a190b52dd6'} {{(pid=80245) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 08:31:05.316970 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.notifications [None req-e2049327-63e5-435b-bb5c-623f5a6a333f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9eb3e65a38bd4f52a2c015a190b52dd6'} {{(pid=80245) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 08:31:05.324018 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.notifications [None req-e2049327-63e5-435b-bb5c-623f5a6a333f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9eb3e65a38bd4f52a2c015a190b52dd6'} {{(pid=80245) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 08:31:05.324708 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.notifications [None req-e2049327-63e5-435b-bb5c-623f5a6a333f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9eb3e65a38bd4f52a2c015a190b52dd6'} {{(pid=80245) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 08:31:05.331223 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: [pid: 80245|app: 0|req: 953/1921] 10.222.0.47 () {58 vars in 1223 bytes} [Thu Jul 21 08:31:05 2022] DELETE /identity/v3/users/9eb3e65a38bd4f52a2c015a190b52dd6 => generated 0 bytes in 156 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 08:31:05.335820 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8c7001e-3725-4225-80af-e969b68a0551 None None] REQUEST_METHOD: `POST` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:31:05.336282 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8c7001e-3725-4225-80af-e969b68a0551 None None] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:31:05.336390 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8c7001e-3725-4225-80af-e969b68a0551 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:31:05.355302 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.auth.core [None req-d8c7001e-3725-4225-80af-e969b68a0551 None None] MFA Rules not processed for user `2dce40b8d74c41ec94cc36e6ff05d412`. Rule list: `[]` (Enabled: `True`). {{(pid=80244) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 08:31:05.357227 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.fernet_utils [None req-d8c7001e-3725-4225-80af-e969b68a0551 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80244) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 08:31:05.428949 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 969/1922] 10.222.0.47 () {58 vars in 957 bytes} [Thu Jul 21 08:31:05 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 08:31:05.440239 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2049327-63e5-435b-bb5c-623f5a6a333f admin admin] Authenticating user token {{(pid=80245) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:31:05.441877 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.fernet_utils [None req-e2049327-63e5-435b-bb5c-623f5a6a333f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80245) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 08:31:05.502060 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG dbcounter [-] [80245] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80245) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:31:05.508356 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2049327-63e5-435b-bb5c-623f5a6a333f admin admin] Validating token access rules against request {{(pid=80245) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:31:05.513906 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dd5b131-edbb-4d39-a06c-8bf43fd4c29c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20ca3c3bd3774691b3374cfbfec6e494', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4faef13aa50e4e36b6c265fb1398fa14', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80245) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:31:05.514587 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dd5b131-edbb-4d39-a06c-8bf43fd4c29c service neutron] REQUEST_METHOD: `GET` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:31:05.514888 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dd5b131-edbb-4d39-a06c-8bf43fd4c29c service neutron] SCRIPT_NAME: `/identity` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:31:05.515142 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dd5b131-edbb-4d39-a06c-8bf43fd4c29c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:31:05.524982 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dd5b131-edbb-4d39-a06c-8bf43fd4c29c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:31:05.526703 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dd5b131-edbb-4d39-a06c-8bf43fd4c29c service neutron] RBAC: Authorization granted {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:31:05.538571 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: [pid: 80245|app: 0|req: 954/1923] 10.222.0.47 () {60 vars in 1365 bytes} [Thu Jul 21 08:31:05 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 21 08:31:05.912359 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8c7001e-3725-4225-80af-e969b68a0551 None None] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:31:05.914105 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.fernet_utils [None req-d8c7001e-3725-4225-80af-e969b68a0551 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80244) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 08:31:05.937795 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8c7001e-3725-4225-80af-e969b68a0551 None None] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:31:05.951895 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed0bd2b5-ec7d-49b4-a6de-d053468e1e7d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '42d15fc0515945acacbbffcf2356c7a5', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:31:05.952795 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed0bd2b5-ec7d-49b4-a6de-d053468e1e7d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:31:05.953079 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed0bd2b5-ec7d-49b4-a6de-d053468e1e7d admin admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:31:05.953371 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed0bd2b5-ec7d-49b4-a6de-d053468e1e7d admin admin] PATH_INFO: `/v3/projects/ddc31a2c572c41efa7b0e6661c39c7c5` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:31:05.954591 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed0bd2b5-ec7d-49b4-a6de-d053468e1e7d admin admin] RBAC: Authorizing `identity:delete_project(project_id=ddc31a2c572c41efa7b0e6661c39c7c5)` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:31:05.956556 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed0bd2b5-ec7d-49b4-a6de-d053468e1e7d admin admin] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:31:06.020868 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG dbcounter [-] [80244] Writing DB stats keystone:SELECT=94,keystone:INSERT=3,keystone:DELETE=3 {{(pid=80244) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:31:06.038466 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.notifications [None req-ed0bd2b5-ec7d-49b4-a6de-d053468e1e7d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ddc31a2c572c41efa7b0e6661c39c7c5', 'request_id': 'req-ed0bd2b5-ec7d-49b4-a6de-d053468e1e7d'} {{(pid=80244) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 08:31:06.052509 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.notifications [None req-ed0bd2b5-ec7d-49b4-a6de-d053468e1e7d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ddc31a2c572c41efa7b0e6661c39c7c5', 'request_id': 'req-ed0bd2b5-ec7d-49b4-a6de-d053468e1e7d'} {{(pid=80244) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 08:31:06.063682 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.notifications [None req-ed0bd2b5-ec7d-49b4-a6de-d053468e1e7d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ddc31a2c572c41efa7b0e6661c39c7c5', 'request_id': 'req-ed0bd2b5-ec7d-49b4-a6de-d053468e1e7d'} {{(pid=80244) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 08:31:06.080304 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.notifications [None req-ed0bd2b5-ec7d-49b4-a6de-d053468e1e7d admin admin] The token cache is being invalidate because project ddc31a2c572c41efa7b0e6661c39c7c5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80244) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 21 08:31:06.080831 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.notifications [None req-ed0bd2b5-ec7d-49b4-a6de-d053468e1e7d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80244) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 08:31:06.081895 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.notifications [None req-ed0bd2b5-ec7d-49b4-a6de-d053468e1e7d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80244) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 08:31:06.084561 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 970/1924] 10.222.0.47 () {58 vars in 1232 bytes} [Thu Jul 21 08:31:05 2022] DELETE /identity/v3/projects/ddc31a2c572c41efa7b0e6661c39c7c5 => generated 0 bytes in 173 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 08:31:06.089581 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25f34756-efd3-4673-87de-5c53659c5286 None None] REQUEST_METHOD: `POST` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:31:06.090244 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25f34756-efd3-4673-87de-5c53659c5286 None None] SCRIPT_NAME: `/identity` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:31:06.090796 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25f34756-efd3-4673-87de-5c53659c5286 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:31:06.115108 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.auth.core [None req-25f34756-efd3-4673-87de-5c53659c5286 None None] MFA Rules not processed for user `2dce40b8d74c41ec94cc36e6ff05d412`. Rule list: `[]` (Enabled: `True`). {{(pid=80245) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 08:31:06.116354 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.fernet_utils [None req-25f34756-efd3-4673-87de-5c53659c5286 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80245) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 08:31:06.182581 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: [pid: 80245|app: 0|req: 955/1925] 10.222.0.47 () {58 vars in 957 bytes} [Thu Jul 21 08:31:06 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 08:31:06.190816 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d618ec9c-7b4b-48ea-98fa-a95bd167d743 None None] REQUEST_METHOD: `POST` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:31:06.190816 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d618ec9c-7b4b-48ea-98fa-a95bd167d743 None None] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:31:06.191030 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d618ec9c-7b4b-48ea-98fa-a95bd167d743 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:31:06.210587 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.auth.core [None req-d618ec9c-7b4b-48ea-98fa-a95bd167d743 None None] MFA Rules not processed for user `2dce40b8d74c41ec94cc36e6ff05d412`. Rule list: `[]` (Enabled: `True`). {{(pid=80244) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 08:31:06.211892 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.fernet_utils [None req-d618ec9c-7b4b-48ea-98fa-a95bd167d743 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80244) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 08:31:06.221257 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 971/1926] 10.222.0.47 () {58 vars in 957 bytes} [Thu Jul 21 08:31:06 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 08:31:06.225600 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25f34756-efd3-4673-87de-5c53659c5286 None None] Authenticating user token {{(pid=80245) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:31:06.235784 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25f34756-efd3-4673-87de-5c53659c5286 None None] Validating token access rules against request {{(pid=80245) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:31:06.242718 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3640e100-9e9a-4b7c-b2fa-bae1d7fda393 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '42d15fc0515945acacbbffcf2356c7a5', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80245) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:31:06.243439 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3640e100-9e9a-4b7c-b2fa-bae1d7fda393 admin admin] REQUEST_METHOD: `GET` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:31:06.243730 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3640e100-9e9a-4b7c-b2fa-bae1d7fda393 admin admin] SCRIPT_NAME: `/identity` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:31:06.244007 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3640e100-9e9a-4b7c-b2fa-bae1d7fda393 admin admin] PATH_INFO: `/v3/domains` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:31:06.245237 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3640e100-9e9a-4b7c-b2fa-bae1d7fda393 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80245) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 08:31:06.245237 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3640e100-9e9a-4b7c-b2fa-bae1d7fda393 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:31:06.246717 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3640e100-9e9a-4b7c-b2fa-bae1d7fda393 admin admin] RBAC: Authorization granted {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:31:06.262383 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: [pid: 80245|app: 0|req: 956/1927] 10.222.0.47 () {58 vars in 1152 bytes} [Thu Jul 21 08:31:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 08:31:06.264823 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d618ec9c-7b4b-48ea-98fa-a95bd167d743 None None] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:31:06.274363 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d618ec9c-7b4b-48ea-98fa-a95bd167d743 None None] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:31:06.279771 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a8ec4c9-33dd-4c59-818e-5dda1d0f4032 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '42d15fc0515945acacbbffcf2356c7a5', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:31:06.280438 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a8ec4c9-33dd-4c59-818e-5dda1d0f4032 admin admin] REQUEST_METHOD: `POST` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:31:06.280717 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a8ec4c9-33dd-4c59-818e-5dda1d0f4032 admin admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:31:06.280989 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a8ec4c9-33dd-4c59-818e-5dda1d0f4032 admin admin] PATH_INFO: `/v3/projects` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:31:06.281693 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a8ec4c9-33dd-4c59-818e-5dda1d0f4032 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:31:06.283620 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a8ec4c9-33dd-4c59-818e-5dda1d0f4032 admin admin] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:31:06.309102 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 972/1928] 10.222.0.47 () {60 vars in 1152 bytes} [Thu Jul 21 08:31:06 2022] POST /identity/v3/projects => generated 371 bytes in 45 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 08:31:06.314357 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3640e100-9e9a-4b7c-b2fa-bae1d7fda393 admin admin] Authenticating user token {{(pid=80245) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:31:06.324942 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3640e100-9e9a-4b7c-b2fa-bae1d7fda393 admin admin] Validating token access rules against request {{(pid=80245) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:31:06.383073 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5c4e4b7-1c1f-4277-b8d2-259563ef82ce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '42d15fc0515945acacbbffcf2356c7a5', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80245) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:31:06.383770 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5c4e4b7-1c1f-4277-b8d2-259563ef82ce admin admin] REQUEST_METHOD: `POST` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:31:06.384053 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5c4e4b7-1c1f-4277-b8d2-259563ef82ce admin admin] SCRIPT_NAME: `/identity` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:31:06.384329 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5c4e4b7-1c1f-4277-b8d2-259563ef82ce admin admin] PATH_INFO: `/v3/users` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:31:06.385009 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5c4e4b7-1c1f-4277-b8d2-259563ef82ce admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:31:06.386917 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5c4e4b7-1c1f-4277-b8d2-259563ef82ce admin admin] RBAC: Authorization granted {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:31:06.407699 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: [pid: 80245|app: 0|req: 957/1929] 10.222.0.47 () {60 vars in 1143 bytes} [Thu Jul 21 08:31:06 2022] POST /identity/v3/users => generated 286 bytes in 94 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 08:31:06.411730 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a8ec4c9-33dd-4c59-818e-5dda1d0f4032 admin admin] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:31:06.421964 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a8ec4c9-33dd-4c59-818e-5dda1d0f4032 admin admin] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:31:06.426430 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9ada96f-f11d-4c77-9589-61f9d6203b11 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '42d15fc0515945acacbbffcf2356c7a5', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:31:06.427088 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9ada96f-f11d-4c77-9589-61f9d6203b11 admin admin] REQUEST_METHOD: `GET` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:31:06.427423 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9ada96f-f11d-4c77-9589-61f9d6203b11 admin admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:31:06.427699 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9ada96f-f11d-4c77-9589-61f9d6203b11 admin admin] PATH_INFO: `/v3/roles` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:31:06.428419 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9ada96f-f11d-4c77-9589-61f9d6203b11 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:31:06.430020 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9ada96f-f11d-4c77-9589-61f9d6203b11 admin admin] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:31:06.441745 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 973/1930] 10.222.0.47 () {58 vars in 1121 bytes} [Thu Jul 21 08:31:06 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 08:31:06.445767 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5c4e4b7-1c1f-4277-b8d2-259563ef82ce admin admin] Authenticating user token {{(pid=80245) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:31:06.456608 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5c4e4b7-1c1f-4277-b8d2-259563ef82ce admin admin] Validating token access rules against request {{(pid=80245) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:31:06.461273 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cb42ff0-99d4-4b9b-8e10-376866b97331 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '42d15fc0515945acacbbffcf2356c7a5', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80245) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:31:06.462193 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cb42ff0-99d4-4b9b-8e10-376866b97331 admin admin] REQUEST_METHOD: `GET` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:31:06.462483 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cb42ff0-99d4-4b9b-8e10-376866b97331 admin admin] SCRIPT_NAME: `/identity` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:31:06.462798 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cb42ff0-99d4-4b9b-8e10-376866b97331 admin admin] PATH_INFO: `/v3/roles` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:31:06.463557 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cb42ff0-99d4-4b9b-8e10-376866b97331 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:31:06.465154 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cb42ff0-99d4-4b9b-8e10-376866b97331 admin admin] RBAC: Authorization granted {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:31:06.475838 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: [pid: 80245|app: 0|req: 958/1931] 10.222.0.47 () {58 vars in 1121 bytes} [Thu Jul 21 08:31:06 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 08:31:06.479869 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9ada96f-f11d-4c77-9589-61f9d6203b11 admin admin] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:31:06.491343 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9ada96f-f11d-4c77-9589-61f9d6203b11 admin admin] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:31:06.496446 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-424bdab3-fb9b-41b0-9603-2d9e178475ec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '42d15fc0515945acacbbffcf2356c7a5', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:31:06.497940 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-424bdab3-fb9b-41b0-9603-2d9e178475ec admin admin] REQUEST_METHOD: `PUT` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:31:06.498230 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-424bdab3-fb9b-41b0-9603-2d9e178475ec admin admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:31:06.498505 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-424bdab3-fb9b-41b0-9603-2d9e178475ec admin admin] PATH_INFO: `/v3/projects/85be753d7eb74931955e7e4c70c5e4fd/users/b65b9757a1ab45649e0fb369932d76d4/roles/3a074738221e4601b5205715ba87d240` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:31:06.517454 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-424bdab3-fb9b-41b0-9603-2d9e178475ec admin admin] RBAC: Authorizing `identity:create_grant(project_id=85be753d7eb74931955e7e4c70c5e4fd, user_id=b65b9757a1ab45649e0fb369932d76d4, role_id=3a074738221e4601b5205715ba87d240)` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:31:06.520673 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-424bdab3-fb9b-41b0-9603-2d9e178475ec admin admin] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:31:06.535396 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 974/1932] 10.222.0.47 () {60 vars in 1482 bytes} [Thu Jul 21 08:31:06 2022] PUT /identity/v3/projects/85be753d7eb74931955e7e4c70c5e4fd/users/b65b9757a1ab45649e0fb369932d76d4/roles/3a074738221e4601b5205715ba87d240 => generated 0 bytes in 56 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 08:31:06.548778 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c06e0e1-61b0-4bc4-b617-90ec5f76dfd4 None None] REQUEST_METHOD: `POST` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:31:06.549077 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c06e0e1-61b0-4bc4-b617-90ec5f76dfd4 None None] SCRIPT_NAME: `/identity` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:31:06.549369 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c06e0e1-61b0-4bc4-b617-90ec5f76dfd4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:31:06.578291 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.auth.core [None req-0c06e0e1-61b0-4bc4-b617-90ec5f76dfd4 None None] MFA Rules not processed for user `b65b9757a1ab45649e0fb369932d76d4`. Rule list: `[]` (Enabled: `True`). {{(pid=80245) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 08:31:06.579544 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.fernet_utils [None req-0c06e0e1-61b0-4bc4-b617-90ec5f76dfd4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80245) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 08:31:06.624552 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG dbcounter [-] [80245] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80245) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:31:06.647685 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: [pid: 80245|app: 0|req: 959/1933] 10.222.0.47 () {58 vars in 957 bytes} [Thu Jul 21 08:31:06 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 08:31:06.658973 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-424bdab3-fb9b-41b0-9603-2d9e178475ec admin admin] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:31:06.660666 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.fernet_utils [None req-424bdab3-fb9b-41b0-9603-2d9e178475ec admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80244) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 08:31:06.717736 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-424bdab3-fb9b-41b0-9603-2d9e178475ec admin admin] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:31:06.721293 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dafdb1a-35da-4cbf-ac3a-2b655785cfaa service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4f40410b7674687ab5362ac982248ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4faef13aa50e4e36b6c265fb1398fa14', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:31:06.721935 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dafdb1a-35da-4cbf-ac3a-2b655785cfaa service cinder] REQUEST_METHOD: `GET` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:31:06.722218 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dafdb1a-35da-4cbf-ac3a-2b655785cfaa service cinder] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:31:06.722488 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dafdb1a-35da-4cbf-ac3a-2b655785cfaa service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:31:06.730556 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dafdb1a-35da-4cbf-ac3a-2b655785cfaa service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:31:06.732030 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dafdb1a-35da-4cbf-ac3a-2b655785cfaa service cinder] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:31:06.740690 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 975/1934] 10.222.0.47 () {60 vars in 1365 bytes} [Thu Jul 21 08:31:06 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 82 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 08:31:06.765185 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c06e0e1-61b0-4bc4-b617-90ec5f76dfd4 None None] Authenticating user token {{(pid=80245) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:31:06.831774 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c06e0e1-61b0-4bc4-b617-90ec5f76dfd4 None None] Validating token access rules against request {{(pid=80245) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:31:06.836271 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e84c396-1e2e-48ea-89c1-5202cda5416b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '42d15fc0515945acacbbffcf2356c7a5', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80245) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:31:06.837165 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e84c396-1e2e-48ea-89c1-5202cda5416b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:31:06.837447 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e84c396-1e2e-48ea-89c1-5202cda5416b admin admin] SCRIPT_NAME: `/identity` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:31:06.837800 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e84c396-1e2e-48ea-89c1-5202cda5416b admin admin] PATH_INFO: `/v3/users/b65b9757a1ab45649e0fb369932d76d4` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:31:06.838908 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e84c396-1e2e-48ea-89c1-5202cda5416b admin admin] RBAC: Authorizing `identity:delete_user(user_id=b65b9757a1ab45649e0fb369932d76d4)` {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:31:06.841453 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e84c396-1e2e-48ea-89c1-5202cda5416b admin admin] RBAC: Authorization granted {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:31:06.890991 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.notifications [None req-7e84c396-1e2e-48ea-89c1-5202cda5416b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b65b9757a1ab45649e0fb369932d76d4'} {{(pid=80245) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 08:31:06.900232 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.notifications [None req-7e84c396-1e2e-48ea-89c1-5202cda5416b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b65b9757a1ab45649e0fb369932d76d4'} {{(pid=80245) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 08:31:06.913893 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.notifications [None req-7e84c396-1e2e-48ea-89c1-5202cda5416b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b65b9757a1ab45649e0fb369932d76d4'} {{(pid=80245) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 08:31:06.920829 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.notifications [None req-7e84c396-1e2e-48ea-89c1-5202cda5416b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b65b9757a1ab45649e0fb369932d76d4'} {{(pid=80245) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 08:31:06.921565 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.notifications [None req-7e84c396-1e2e-48ea-89c1-5202cda5416b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b65b9757a1ab45649e0fb369932d76d4'} {{(pid=80245) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 08:31:06.927588 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: [pid: 80245|app: 0|req: 960/1935] 10.222.0.47 () {58 vars in 1223 bytes} [Thu Jul 21 08:31:06 2022] DELETE /identity/v3/users/b65b9757a1ab45649e0fb369932d76d4 => generated 0 bytes in 163 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 08:31:06.932284 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18e05f03-c9d6-4066-b087-6d21c53bbb73 None None] REQUEST_METHOD: `POST` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:31:06.932593 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18e05f03-c9d6-4066-b087-6d21c53bbb73 None None] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:31:06.932868 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18e05f03-c9d6-4066-b087-6d21c53bbb73 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:31:06.950813 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.auth.core [None req-18e05f03-c9d6-4066-b087-6d21c53bbb73 None None] MFA Rules not processed for user `2dce40b8d74c41ec94cc36e6ff05d412`. Rule list: `[]` (Enabled: `True`). {{(pid=80244) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 08:31:06.952131 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.fernet_utils [None req-18e05f03-c9d6-4066-b087-6d21c53bbb73 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80244) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 08:31:07.022839 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 976/1936] 10.222.0.47 () {58 vars in 957 bytes} [Thu Jul 21 08:31:06 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 08:31:07.033859 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e84c396-1e2e-48ea-89c1-5202cda5416b admin admin] Authenticating user token {{(pid=80245) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:31:07.035285 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.fernet_utils [None req-7e84c396-1e2e-48ea-89c1-5202cda5416b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80245) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 08:31:07.092593 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e84c396-1e2e-48ea-89c1-5202cda5416b admin admin] Validating token access rules against request {{(pid=80245) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:31:07.096216 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b181e5e-0805-4cce-9e6f-0fae75239fcc service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20ca3c3bd3774691b3374cfbfec6e494', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4faef13aa50e4e36b6c265fb1398fa14', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80245) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:31:07.096885 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b181e5e-0805-4cce-9e6f-0fae75239fcc service neutron] REQUEST_METHOD: `GET` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:31:07.097165 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b181e5e-0805-4cce-9e6f-0fae75239fcc service neutron] SCRIPT_NAME: `/identity` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:31:07.097438 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b181e5e-0805-4cce-9e6f-0fae75239fcc service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:31:07.106370 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b181e5e-0805-4cce-9e6f-0fae75239fcc service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:31:07.107815 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b181e5e-0805-4cce-9e6f-0fae75239fcc service neutron] RBAC: Authorization granted {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:31:07.118372 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: [pid: 80245|app: 0|req: 961/1937] 10.222.0.47 () {60 vars in 1365 bytes} [Thu Jul 21 08:31:07 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 08:31:07.464637 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18e05f03-c9d6-4066-b087-6d21c53bbb73 None None] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:31:07.466415 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.fernet_utils [None req-18e05f03-c9d6-4066-b087-6d21c53bbb73 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80244) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 08:31:07.475693 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG dbcounter [-] [80244] Writing DB stats keystone:SELECT=93,keystone:DELETE=4,keystone:INSERT=3 {{(pid=80244) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:31:07.483885 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18e05f03-c9d6-4066-b087-6d21c53bbb73 None None] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:31:07.491966 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59f4efac-9138-4170-a467-61bdd69cbad2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '42d15fc0515945acacbbffcf2356c7a5', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:31:07.493473 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59f4efac-9138-4170-a467-61bdd69cbad2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:31:07.494010 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59f4efac-9138-4170-a467-61bdd69cbad2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:31:07.494361 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59f4efac-9138-4170-a467-61bdd69cbad2 admin admin] PATH_INFO: `/v3/projects/85be753d7eb74931955e7e4c70c5e4fd` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:31:07.495899 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59f4efac-9138-4170-a467-61bdd69cbad2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=85be753d7eb74931955e7e4c70c5e4fd)` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:31:07.498775 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59f4efac-9138-4170-a467-61bdd69cbad2 admin admin] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:31:07.549003 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.notifications [None req-59f4efac-9138-4170-a467-61bdd69cbad2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '85be753d7eb74931955e7e4c70c5e4fd', 'request_id': 'req-59f4efac-9138-4170-a467-61bdd69cbad2'} {{(pid=80244) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 08:31:07.556419 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.notifications [None req-59f4efac-9138-4170-a467-61bdd69cbad2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '85be753d7eb74931955e7e4c70c5e4fd', 'request_id': 'req-59f4efac-9138-4170-a467-61bdd69cbad2'} {{(pid=80244) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 08:31:07.565280 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.notifications [None req-59f4efac-9138-4170-a467-61bdd69cbad2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '85be753d7eb74931955e7e4c70c5e4fd', 'request_id': 'req-59f4efac-9138-4170-a467-61bdd69cbad2'} {{(pid=80244) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 08:31:07.576161 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.notifications [None req-59f4efac-9138-4170-a467-61bdd69cbad2 admin admin] The token cache is being invalidate because project 85be753d7eb74931955e7e4c70c5e4fd was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80244) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 21 08:31:07.576453 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.notifications [None req-59f4efac-9138-4170-a467-61bdd69cbad2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80244) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 08:31:07.577120 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.notifications [None req-59f4efac-9138-4170-a467-61bdd69cbad2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80244) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 08:31:07.578691 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 977/1938] 10.222.0.47 () {58 vars in 1232 bytes} [Thu Jul 21 08:31:07 2022] DELETE /identity/v3/projects/85be753d7eb74931955e7e4c70c5e4fd => generated 0 bytes in 115 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 08:31:07.583691 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3063f349-6148-416d-b5ff-ae7393281407 None None] REQUEST_METHOD: `POST` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:31:07.584153 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3063f349-6148-416d-b5ff-ae7393281407 None None] SCRIPT_NAME: `/identity` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:31:07.584268 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3063f349-6148-416d-b5ff-ae7393281407 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:31:07.596499 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG dbcounter [-] [80245] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80245) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:31:07.603223 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.auth.core [None req-3063f349-6148-416d-b5ff-ae7393281407 None None] MFA Rules not processed for user `2dce40b8d74c41ec94cc36e6ff05d412`. Rule list: `[]` (Enabled: `True`). {{(pid=80245) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 08:31:07.604998 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.fernet_utils [None req-3063f349-6148-416d-b5ff-ae7393281407 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80245) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 08:31:07.664362 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: [pid: 80245|app: 0|req: 962/1939] 10.222.0.47 () {58 vars in 957 bytes} [Thu Jul 21 08:31:07 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 08:31:07.672636 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aad2f3a-8729-43fb-8178-1cc536e9a30b None None] REQUEST_METHOD: `POST` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:31:07.672923 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aad2f3a-8729-43fb-8178-1cc536e9a30b None None] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:31:07.673198 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aad2f3a-8729-43fb-8178-1cc536e9a30b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:31:07.692283 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.auth.core [None req-2aad2f3a-8729-43fb-8178-1cc536e9a30b None None] MFA Rules not processed for user `2dce40b8d74c41ec94cc36e6ff05d412`. Rule list: `[]` (Enabled: `True`). {{(pid=80244) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 08:31:07.693486 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.fernet_utils [None req-2aad2f3a-8729-43fb-8178-1cc536e9a30b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80244) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 08:31:07.705150 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 978/1940] 10.222.0.47 () {58 vars in 957 bytes} [Thu Jul 21 08:31:07 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 08:31:07.706757 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3063f349-6148-416d-b5ff-ae7393281407 None None] Authenticating user token {{(pid=80245) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:31:07.717489 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3063f349-6148-416d-b5ff-ae7393281407 None None] Validating token access rules against request {{(pid=80245) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:31:07.723903 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb72502b-fa1d-49de-ac53-22bbefeddf6c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '42d15fc0515945acacbbffcf2356c7a5', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80245) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:31:07.724985 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb72502b-fa1d-49de-ac53-22bbefeddf6c admin admin] REQUEST_METHOD: `GET` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:31:07.725106 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb72502b-fa1d-49de-ac53-22bbefeddf6c admin admin] SCRIPT_NAME: `/identity` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:31:07.725458 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb72502b-fa1d-49de-ac53-22bbefeddf6c admin admin] PATH_INFO: `/v3/domains` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:31:07.726333 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb72502b-fa1d-49de-ac53-22bbefeddf6c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80245) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 08:31:07.726711 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb72502b-fa1d-49de-ac53-22bbefeddf6c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:31:07.728501 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb72502b-fa1d-49de-ac53-22bbefeddf6c admin admin] RBAC: Authorization granted {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:31:07.743162 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: [pid: 80245|app: 0|req: 963/1941] 10.222.0.47 () {58 vars in 1152 bytes} [Thu Jul 21 08:31:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 21 08:31:07.748254 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aad2f3a-8729-43fb-8178-1cc536e9a30b None None] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:31:07.759033 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aad2f3a-8729-43fb-8178-1cc536e9a30b None None] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:31:07.769002 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b777fd5-43bf-43d9-9ff6-136f35d7c5b2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '42d15fc0515945acacbbffcf2356c7a5', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:31:07.769695 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b777fd5-43bf-43d9-9ff6-136f35d7c5b2 admin admin] REQUEST_METHOD: `POST` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:31:07.770115 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b777fd5-43bf-43d9-9ff6-136f35d7c5b2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:31:07.770531 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b777fd5-43bf-43d9-9ff6-136f35d7c5b2 admin admin] PATH_INFO: `/v3/projects` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:31:07.771435 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b777fd5-43bf-43d9-9ff6-136f35d7c5b2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:31:07.773548 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b777fd5-43bf-43d9-9ff6-136f35d7c5b2 admin admin] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:31:07.809636 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 979/1942] 10.222.0.47 () {60 vars in 1152 bytes} [Thu Jul 21 08:31:07 2022] POST /identity/v3/projects => generated 381 bytes in 63 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 08:31:07.815519 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb72502b-fa1d-49de-ac53-22bbefeddf6c admin admin] Authenticating user token {{(pid=80245) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:31:07.827484 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb72502b-fa1d-49de-ac53-22bbefeddf6c admin admin] Validating token access rules against request {{(pid=80245) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:31:07.900805 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ea449d3-2eb8-43e9-99f8-ea6d23234334 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '42d15fc0515945acacbbffcf2356c7a5', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80245) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:31:07.901532 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ea449d3-2eb8-43e9-99f8-ea6d23234334 admin admin] REQUEST_METHOD: `POST` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:31:07.901817 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ea449d3-2eb8-43e9-99f8-ea6d23234334 admin admin] SCRIPT_NAME: `/identity` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:31:07.902093 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ea449d3-2eb8-43e9-99f8-ea6d23234334 admin admin] PATH_INFO: `/v3/users` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:31:07.902814 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ea449d3-2eb8-43e9-99f8-ea6d23234334 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:31:07.905283 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ea449d3-2eb8-43e9-99f8-ea6d23234334 admin admin] RBAC: Authorization granted {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:31:07.931248 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: [pid: 80245|app: 0|req: 964/1943] 10.222.0.47 () {60 vars in 1143 bytes} [Thu Jul 21 08:31:07 2022] POST /identity/v3/users => generated 291 bytes in 113 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 08:31:07.932726 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b777fd5-43bf-43d9-9ff6-136f35d7c5b2 admin admin] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:31:07.943684 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b777fd5-43bf-43d9-9ff6-136f35d7c5b2 admin admin] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:31:07.948219 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a777253-0ce8-4d11-b232-f6e698098b51 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '42d15fc0515945acacbbffcf2356c7a5', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:31:07.948892 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a777253-0ce8-4d11-b232-f6e698098b51 admin admin] REQUEST_METHOD: `GET` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:31:07.949180 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a777253-0ce8-4d11-b232-f6e698098b51 admin admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:31:07.949485 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a777253-0ce8-4d11-b232-f6e698098b51 admin admin] PATH_INFO: `/v3/roles` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:31:07.950267 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a777253-0ce8-4d11-b232-f6e698098b51 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:31:07.951956 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a777253-0ce8-4d11-b232-f6e698098b51 admin admin] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:31:07.968734 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 980/1944] 10.222.0.47 () {58 vars in 1121 bytes} [Thu Jul 21 08:31:07 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 08:31:07.972799 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ea449d3-2eb8-43e9-99f8-ea6d23234334 admin admin] Authenticating user token {{(pid=80245) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:31:07.985483 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ea449d3-2eb8-43e9-99f8-ea6d23234334 admin admin] Validating token access rules against request {{(pid=80245) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:31:07.990661 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dd5f65b-43ac-4555-aff6-fab89d58ad97 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '42d15fc0515945acacbbffcf2356c7a5', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80245) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:31:07.991348 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dd5f65b-43ac-4555-aff6-fab89d58ad97 admin admin] REQUEST_METHOD: `GET` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:31:07.991642 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dd5f65b-43ac-4555-aff6-fab89d58ad97 admin admin] SCRIPT_NAME: `/identity` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:31:07.991980 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dd5f65b-43ac-4555-aff6-fab89d58ad97 admin admin] PATH_INFO: `/v3/roles` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:31:07.993051 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dd5f65b-43ac-4555-aff6-fab89d58ad97 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:31:07.995282 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dd5f65b-43ac-4555-aff6-fab89d58ad97 admin admin] RBAC: Authorization granted {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:31:08.006639 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: [pid: 80245|app: 0|req: 965/1945] 10.222.0.47 () {58 vars in 1121 bytes} [Thu Jul 21 08:31:07 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 21 08:31:08.010455 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a777253-0ce8-4d11-b232-f6e698098b51 admin admin] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:31:08.020567 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a777253-0ce8-4d11-b232-f6e698098b51 admin admin] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:31:08.025541 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-242d19c6-c9be-46b7-96d4-76ce3ee456ee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '42d15fc0515945acacbbffcf2356c7a5', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:31:08.026488 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-242d19c6-c9be-46b7-96d4-76ce3ee456ee admin admin] REQUEST_METHOD: `PUT` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:31:08.026763 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-242d19c6-c9be-46b7-96d4-76ce3ee456ee admin admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:31:08.027043 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-242d19c6-c9be-46b7-96d4-76ce3ee456ee admin admin] PATH_INFO: `/v3/projects/419b6c6707544364b82f3f6fb5030032/users/1f914243baf846f5b43a929332bcef6e/roles/3a074738221e4601b5205715ba87d240` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:31:08.044828 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-242d19c6-c9be-46b7-96d4-76ce3ee456ee admin admin] RBAC: Authorizing `identity:create_grant(project_id=419b6c6707544364b82f3f6fb5030032, user_id=1f914243baf846f5b43a929332bcef6e, role_id=3a074738221e4601b5205715ba87d240)` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:31:08.048400 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-242d19c6-c9be-46b7-96d4-76ce3ee456ee admin admin] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:31:08.067000 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 981/1946] 10.222.0.47 () {60 vars in 1482 bytes} [Thu Jul 21 08:31:08 2022] PUT /identity/v3/projects/419b6c6707544364b82f3f6fb5030032/users/1f914243baf846f5b43a929332bcef6e/roles/3a074738221e4601b5205715ba87d240 => generated 0 bytes in 57 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 08:31:08.072064 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe76c1cd-67a3-4aea-929f-a51ce68abf7a None None] REQUEST_METHOD: `POST` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:31:08.072406 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe76c1cd-67a3-4aea-929f-a51ce68abf7a None None] SCRIPT_NAME: `/identity` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:31:08.072693 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe76c1cd-67a3-4aea-929f-a51ce68abf7a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:31:08.092000 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.auth.core [None req-fe76c1cd-67a3-4aea-929f-a51ce68abf7a None None] MFA Rules not processed for user `2dce40b8d74c41ec94cc36e6ff05d412`. Rule list: `[]` (Enabled: `True`). {{(pid=80245) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 08:31:08.093226 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.fernet_utils [None req-fe76c1cd-67a3-4aea-929f-a51ce68abf7a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80245) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 08:31:08.162252 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: [pid: 80245|app: 0|req: 966/1947] 10.222.0.47 () {58 vars in 957 bytes} [Thu Jul 21 08:31:08 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 21 08:31:08.174504 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-242d19c6-c9be-46b7-96d4-76ce3ee456ee admin admin] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:31:08.176570 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.fernet_utils [None req-242d19c6-c9be-46b7-96d4-76ce3ee456ee admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80244) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 08:31:08.283625 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-242d19c6-c9be-46b7-96d4-76ce3ee456ee admin admin] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:31:08.287903 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73dd6f1a-8078-4a9f-af21-e05d555a0167 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20ca3c3bd3774691b3374cfbfec6e494', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4faef13aa50e4e36b6c265fb1398fa14', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:31:08.288582 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73dd6f1a-8078-4a9f-af21-e05d555a0167 service neutron] REQUEST_METHOD: `GET` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:31:08.288944 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73dd6f1a-8078-4a9f-af21-e05d555a0167 service neutron] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:31:08.289291 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73dd6f1a-8078-4a9f-af21-e05d555a0167 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:31:08.299977 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73dd6f1a-8078-4a9f-af21-e05d555a0167 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:31:08.301522 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73dd6f1a-8078-4a9f-af21-e05d555a0167 service neutron] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:31:08.311590 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 982/1948] 10.222.0.47 () {60 vars in 1365 bytes} [Thu Jul 21 08:31:08 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 138 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 08:31:09.109339 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce7960d9-631e-4423-9889-9912921115ab None None] REQUEST_METHOD: `POST` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:31:09.109815 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce7960d9-631e-4423-9889-9912921115ab None None] SCRIPT_NAME: `/identity` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:31:09.109890 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce7960d9-631e-4423-9889-9912921115ab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:31:09.135300 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG dbcounter [-] [80245] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80245) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:31:09.139118 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.auth.core [None req-ce7960d9-631e-4423-9889-9912921115ab None None] MFA Rules not processed for user `1f914243baf846f5b43a929332bcef6e`. Rule list: `[]` (Enabled: `True`). {{(pid=80245) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 08:31:09.142040 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.fernet_utils [None req-ce7960d9-631e-4423-9889-9912921115ab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80245) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 08:31:09.226183 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: [pid: 80245|app: 0|req: 967/1949] 10.222.0.47 () {58 vars in 957 bytes} [Thu Jul 21 08:31:09 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 08:31:09.238312 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73dd6f1a-8078-4a9f-af21-e05d555a0167 service neutron] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:31:09.240100 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.fernet_utils [None req-73dd6f1a-8078-4a9f-af21-e05d555a0167 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80244) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 08:31:09.275778 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG dbcounter [-] [80244] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80244) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:31:09.323428 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73dd6f1a-8078-4a9f-af21-e05d555a0167 service neutron] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:31:09.327821 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eecb6fbc-f1d7-4af8-be39-2baae5241b62 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4f40410b7674687ab5362ac982248ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4faef13aa50e4e36b6c265fb1398fa14', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:31:09.328574 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eecb6fbc-f1d7-4af8-be39-2baae5241b62 service cinder] REQUEST_METHOD: `GET` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:31:09.328855 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eecb6fbc-f1d7-4af8-be39-2baae5241b62 service cinder] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:31:09.329133 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eecb6fbc-f1d7-4af8-be39-2baae5241b62 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:31:09.339524 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eecb6fbc-f1d7-4af8-be39-2baae5241b62 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:31:09.341684 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eecb6fbc-f1d7-4af8-be39-2baae5241b62 service cinder] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:31:09.355079 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 983/1950] 10.222.0.47 () {60 vars in 1365 bytes} [Thu Jul 21 08:31:09 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 117 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 08:31:11.895290 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69c882cb-203c-46a5-9360-3730ab0213fe None None] REQUEST_METHOD: `POST` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:31:11.895749 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69c882cb-203c-46a5-9360-3730ab0213fe None None] SCRIPT_NAME: `/identity` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:31:11.895978 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69c882cb-203c-46a5-9360-3730ab0213fe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:31:11.914854 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.auth.core [None req-69c882cb-203c-46a5-9360-3730ab0213fe None None] MFA Rules not processed for user `1f914243baf846f5b43a929332bcef6e`. Rule list: `[]` (Enabled: `True`). {{(pid=80245) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 08:31:11.916104 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.fernet_utils [None req-69c882cb-203c-46a5-9360-3730ab0213fe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80245) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 08:31:11.923871 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: [pid: 80245|app: 0|req: 968/1951] 10.222.0.47 () {58 vars in 957 bytes} [Thu Jul 21 08:31:11 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 08:31:12.752291 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eecb6fbc-f1d7-4af8-be39-2baae5241b62 service cinder] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:31:12.753965 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.fernet_utils [None req-eecb6fbc-f1d7-4af8-be39-2baae5241b62 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80244) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 08:31:12.791460 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eecb6fbc-f1d7-4af8-be39-2baae5241b62 service cinder] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:31:12.804827 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfba880b-b99e-4c02-b8cc-19cc14dbee65 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '42d15fc0515945acacbbffcf2356c7a5', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:31:12.805930 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfba880b-b99e-4c02-b8cc-19cc14dbee65 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:31:12.808287 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfba880b-b99e-4c02-b8cc-19cc14dbee65 admin admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:31:12.808287 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfba880b-b99e-4c02-b8cc-19cc14dbee65 admin admin] PATH_INFO: `/v3/users/ceb37aaecc4d44399cc1ec0e384cd4bb` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:31:12.810509 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfba880b-b99e-4c02-b8cc-19cc14dbee65 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ceb37aaecc4d44399cc1ec0e384cd4bb)` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:31:12.816018 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfba880b-b99e-4c02-b8cc-19cc14dbee65 admin admin] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:31:12.932363 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69c882cb-203c-46a5-9360-3730ab0213fe None None] Authenticating user token {{(pid=80245) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:31:12.934998 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.fernet_utils [None req-69c882cb-203c-46a5-9360-3730ab0213fe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80245) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 08:31:12.940721 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.notifications [None req-dfba880b-b99e-4c02-b8cc-19cc14dbee65 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ceb37aaecc4d44399cc1ec0e384cd4bb'} {{(pid=80244) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 08:31:12.954305 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.notifications [None req-dfba880b-b99e-4c02-b8cc-19cc14dbee65 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ceb37aaecc4d44399cc1ec0e384cd4bb'} {{(pid=80244) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 08:31:12.973768 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69c882cb-203c-46a5-9360-3730ab0213fe None None] Validating token access rules against request {{(pid=80245) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:31:12.975437 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.notifications [None req-dfba880b-b99e-4c02-b8cc-19cc14dbee65 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ceb37aaecc4d44399cc1ec0e384cd4bb'} {{(pid=80244) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 08:31:12.976760 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.notifications [None req-dfba880b-b99e-4c02-b8cc-19cc14dbee65 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ceb37aaecc4d44399cc1ec0e384cd4bb'} {{(pid=80244) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 08:31:12.980646 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07aaf54e-b87d-45bd-b971-bd89f9a650f3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '42d15fc0515945acacbbffcf2356c7a5', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80245) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:31:12.982259 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07aaf54e-b87d-45bd-b971-bd89f9a650f3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:31:12.982793 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07aaf54e-b87d-45bd-b971-bd89f9a650f3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:31:12.983398 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07aaf54e-b87d-45bd-b971-bd89f9a650f3 admin admin] PATH_INFO: `/v3/users/ce63a95cc2aa4ff6844cf0a1b49c648d` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:31:12.985644 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07aaf54e-b87d-45bd-b971-bd89f9a650f3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ce63a95cc2aa4ff6844cf0a1b49c648d)` {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:31:12.986440 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.notifications [None req-dfba880b-b99e-4c02-b8cc-19cc14dbee65 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ceb37aaecc4d44399cc1ec0e384cd4bb'} {{(pid=80244) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 08:31:12.992214 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07aaf54e-b87d-45bd-b971-bd89f9a650f3 admin admin] RBAC: Authorization granted {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:31:13.022429 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 984/1952] 10.222.0.47 () {58 vars in 1223 bytes} [Thu Jul 21 08:31:12 2022] DELETE /identity/v3/users/ceb37aaecc4d44399cc1ec0e384cd4bb => generated 0 bytes in 267 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 08:31:13.025328 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86b03be8-4e86-4ea1-bc65-2e6c64771e04 None None] REQUEST_METHOD: `POST` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:31:13.030324 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86b03be8-4e86-4ea1-bc65-2e6c64771e04 None None] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:31:13.030655 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86b03be8-4e86-4ea1-bc65-2e6c64771e04 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:31:13.060637 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.auth.core [None req-86b03be8-4e86-4ea1-bc65-2e6c64771e04 None None] MFA Rules not processed for user `2dce40b8d74c41ec94cc36e6ff05d412`. Rule list: `[]` (Enabled: `True`). {{(pid=80244) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 08:31:13.062123 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.fernet_utils [None req-86b03be8-4e86-4ea1-bc65-2e6c64771e04 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80244) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 08:31:13.102087 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.notifications [None req-07aaf54e-b87d-45bd-b971-bd89f9a650f3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ce63a95cc2aa4ff6844cf0a1b49c648d'} {{(pid=80245) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 08:31:13.117409 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.notifications [None req-07aaf54e-b87d-45bd-b971-bd89f9a650f3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ce63a95cc2aa4ff6844cf0a1b49c648d'} {{(pid=80245) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 08:31:13.138641 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.notifications [None req-07aaf54e-b87d-45bd-b971-bd89f9a650f3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ce63a95cc2aa4ff6844cf0a1b49c648d'} {{(pid=80245) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 08:31:13.154375 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.notifications [None req-07aaf54e-b87d-45bd-b971-bd89f9a650f3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ce63a95cc2aa4ff6844cf0a1b49c648d'} {{(pid=80245) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 08:31:13.156527 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.notifications [None req-07aaf54e-b87d-45bd-b971-bd89f9a650f3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ce63a95cc2aa4ff6844cf0a1b49c648d'} {{(pid=80245) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 08:31:13.181298 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: [pid: 80245|app: 0|req: 969/1953] 10.222.0.47 () {58 vars in 1223 bytes} [Thu Jul 21 08:31:12 2022] DELETE /identity/v3/users/ce63a95cc2aa4ff6844cf0a1b49c648d => generated 0 bytes in 250 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 21 08:31:13.187404 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a37b83d4-040e-4151-b8cb-d11c43abea99 None None] REQUEST_METHOD: `POST` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:31:13.187767 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a37b83d4-040e-4151-b8cb-d11c43abea99 None None] SCRIPT_NAME: `/identity` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:31:13.188365 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a37b83d4-040e-4151-b8cb-d11c43abea99 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:31:13.192975 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 985/1954] 10.222.0.47 () {58 vars in 957 bytes} [Thu Jul 21 08:31:13 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 08:31:13.201086 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86b03be8-4e86-4ea1-bc65-2e6c64771e04 None None] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:31:13.203411 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.fernet_utils [None req-86b03be8-4e86-4ea1-bc65-2e6c64771e04 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80244) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 08:31:13.217993 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.auth.core [None req-a37b83d4-040e-4151-b8cb-d11c43abea99 None None] MFA Rules not processed for user `2dce40b8d74c41ec94cc36e6ff05d412`. Rule list: `[]` (Enabled: `True`). {{(pid=80245) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 08:31:13.219610 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.fernet_utils [None req-a37b83d4-040e-4151-b8cb-d11c43abea99 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80245) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 08:31:13.355051 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG dbcounter [-] [80244] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80244) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:31:13.384568 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86b03be8-4e86-4ea1-bc65-2e6c64771e04 None None] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:31:13.389811 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: [pid: 80245|app: 0|req: 970/1955] 10.222.0.47 () {58 vars in 957 bytes} [Thu Jul 21 08:31:13 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 203 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 21 08:31:13.400055 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a37b83d4-040e-4151-b8cb-d11c43abea99 None None] Authenticating user token {{(pid=80245) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:31:13.421595 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG dbcounter [-] [80245] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80245) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:31:13.509331 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3859c725-6526-4c50-a094-76c9b519152d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20ca3c3bd3774691b3374cfbfec6e494', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4faef13aa50e4e36b6c265fb1398fa14', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:31:13.510080 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3859c725-6526-4c50-a094-76c9b519152d service neutron] REQUEST_METHOD: `GET` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:31:13.510364 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3859c725-6526-4c50-a094-76c9b519152d service neutron] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:31:13.510640 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3859c725-6526-4c50-a094-76c9b519152d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:31:13.518950 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a37b83d4-040e-4151-b8cb-d11c43abea99 None None] Validating token access rules against request {{(pid=80245) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:31:13.522561 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3859c725-6526-4c50-a094-76c9b519152d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:31:13.523059 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-633dd9f0-dcf5-4541-87e9-8662d8b1eb6a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20ca3c3bd3774691b3374cfbfec6e494', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4faef13aa50e4e36b6c265fb1398fa14', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80245) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:31:13.524481 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3859c725-6526-4c50-a094-76c9b519152d service neutron] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:31:13.524631 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-633dd9f0-dcf5-4541-87e9-8662d8b1eb6a service neutron] REQUEST_METHOD: `GET` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:31:13.525022 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-633dd9f0-dcf5-4541-87e9-8662d8b1eb6a service neutron] SCRIPT_NAME: `/identity` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:31:13.525498 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-633dd9f0-dcf5-4541-87e9-8662d8b1eb6a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:31:13.537403 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 986/1956] 10.222.0.47 () {60 vars in 1365 bytes} [Thu Jul 21 08:31:13 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 337 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 08:31:13.539897 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-633dd9f0-dcf5-4541-87e9-8662d8b1eb6a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:31:13.541386 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-633dd9f0-dcf5-4541-87e9-8662d8b1eb6a service neutron] RBAC: Authorization granted {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:31:13.552895 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: [pid: 80245|app: 0|req: 971/1957] 10.222.0.47 () {60 vars in 1365 bytes} [Thu Jul 21 08:31:13 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 154 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 08:31:14.005174 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3859c725-6526-4c50-a094-76c9b519152d service neutron] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:31:14.006626 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.fernet_utils [None req-3859c725-6526-4c50-a094-76c9b519152d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80244) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 08:31:14.034441 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3859c725-6526-4c50-a094-76c9b519152d service neutron] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:31:14.035513 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-633dd9f0-dcf5-4541-87e9-8662d8b1eb6a service neutron] Authenticating user token {{(pid=80245) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:31:14.037413 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.fernet_utils [None req-633dd9f0-dcf5-4541-87e9-8662d8b1eb6a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80245) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 08:31:14.045461 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09867ab4-7504-43b5-a6dd-643b5ddfcc6d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '42d15fc0515945acacbbffcf2356c7a5', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:31:14.046449 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09867ab4-7504-43b5-a6dd-643b5ddfcc6d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:31:14.052056 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09867ab4-7504-43b5-a6dd-643b5ddfcc6d admin admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:31:14.052313 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09867ab4-7504-43b5-a6dd-643b5ddfcc6d admin admin] PATH_INFO: `/v3/projects/8b3c7dc4d5c1430fb25afe235c0de99e` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:31:14.053797 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09867ab4-7504-43b5-a6dd-643b5ddfcc6d admin admin] RBAC: Authorizing `identity:delete_project(project_id=8b3c7dc4d5c1430fb25afe235c0de99e)` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:31:14.054746 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-633dd9f0-dcf5-4541-87e9-8662d8b1eb6a service neutron] Validating token access rules against request {{(pid=80245) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:31:14.055949 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09867ab4-7504-43b5-a6dd-643b5ddfcc6d admin admin] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:31:14.078449 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b97d1026-0c36-4aab-82fe-133b399642c0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '42d15fc0515945acacbbffcf2356c7a5', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80245) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:31:14.081677 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b97d1026-0c36-4aab-82fe-133b399642c0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:31:14.081967 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b97d1026-0c36-4aab-82fe-133b399642c0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:31:14.082255 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b97d1026-0c36-4aab-82fe-133b399642c0 admin admin] PATH_INFO: `/v3/projects/6401b0739da443cda479f51a5a87e522` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:31:14.086648 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b97d1026-0c36-4aab-82fe-133b399642c0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=6401b0739da443cda479f51a5a87e522)` {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:31:14.088984 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b97d1026-0c36-4aab-82fe-133b399642c0 admin admin] RBAC: Authorization granted {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:31:14.144223 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.notifications [None req-09867ab4-7504-43b5-a6dd-643b5ddfcc6d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8b3c7dc4d5c1430fb25afe235c0de99e', 'request_id': 'req-09867ab4-7504-43b5-a6dd-643b5ddfcc6d'} {{(pid=80244) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 08:31:14.154763 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.notifications [None req-09867ab4-7504-43b5-a6dd-643b5ddfcc6d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8b3c7dc4d5c1430fb25afe235c0de99e', 'request_id': 'req-09867ab4-7504-43b5-a6dd-643b5ddfcc6d'} {{(pid=80244) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 08:31:14.160964 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.notifications [None req-b97d1026-0c36-4aab-82fe-133b399642c0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6401b0739da443cda479f51a5a87e522', 'request_id': 'req-b97d1026-0c36-4aab-82fe-133b399642c0'} {{(pid=80245) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 08:31:14.170997 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.notifications [None req-b97d1026-0c36-4aab-82fe-133b399642c0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6401b0739da443cda479f51a5a87e522', 'request_id': 'req-b97d1026-0c36-4aab-82fe-133b399642c0'} {{(pid=80245) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 08:31:14.171386 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.notifications [None req-09867ab4-7504-43b5-a6dd-643b5ddfcc6d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8b3c7dc4d5c1430fb25afe235c0de99e', 'request_id': 'req-09867ab4-7504-43b5-a6dd-643b5ddfcc6d'} {{(pid=80244) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 08:31:14.178463 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.notifications [None req-b97d1026-0c36-4aab-82fe-133b399642c0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6401b0739da443cda479f51a5a87e522', 'request_id': 'req-b97d1026-0c36-4aab-82fe-133b399642c0'} {{(pid=80245) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 08:31:14.189406 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.notifications [None req-09867ab4-7504-43b5-a6dd-643b5ddfcc6d admin admin] The token cache is being invalidate because project 8b3c7dc4d5c1430fb25afe235c0de99e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80244) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 21 08:31:14.189778 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.notifications [None req-09867ab4-7504-43b5-a6dd-643b5ddfcc6d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80244) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 08:31:14.190643 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.notifications [None req-09867ab4-7504-43b5-a6dd-643b5ddfcc6d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80244) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 08:31:14.192736 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 987/1958] 10.222.0.47 () {58 vars in 1232 bytes} [Thu Jul 21 08:31:14 2022] DELETE /identity/v3/projects/8b3c7dc4d5c1430fb25afe235c0de99e => generated 0 bytes in 188 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 08:31:14.196122 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09867ab4-7504-43b5-a6dd-643b5ddfcc6d admin admin] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:31:14.198140 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.notifications [None req-b97d1026-0c36-4aab-82fe-133b399642c0 admin admin] The token cache is being invalidate because project 6401b0739da443cda479f51a5a87e522 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80245) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 21 08:31:14.198432 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.notifications [None req-b97d1026-0c36-4aab-82fe-133b399642c0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80245) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 08:31:14.199183 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.notifications [None req-b97d1026-0c36-4aab-82fe-133b399642c0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80245) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 08:31:14.199475 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.fernet_utils [None req-09867ab4-7504-43b5-a6dd-643b5ddfcc6d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80244) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 08:31:14.202967 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: [pid: 80245|app: 0|req: 972/1959] 10.222.0.47 () {58 vars in 1232 bytes} [Thu Jul 21 08:31:14 2022] DELETE /identity/v3/projects/6401b0739da443cda479f51a5a87e522 => generated 0 bytes in 168 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 08:31:14.208871 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b97d1026-0c36-4aab-82fe-133b399642c0 admin admin] Authenticating user token {{(pid=80245) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:31:14.210427 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.fernet_utils [None req-b97d1026-0c36-4aab-82fe-133b399642c0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80245) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 08:31:14.312380 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b97d1026-0c36-4aab-82fe-133b399642c0 admin admin] Validating token access rules against request {{(pid=80245) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:31:14.315913 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09867ab4-7504-43b5-a6dd-643b5ddfcc6d admin admin] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:31:14.320863 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ef8c759-886b-4e66-8eb5-5e3dae2a0c83 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '42d15fc0515945acacbbffcf2356c7a5', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:31:14.321005 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b342233-652c-4437-b944-19fe58fd21aa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '42d15fc0515945acacbbffcf2356c7a5', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80245) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:31:14.321788 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ef8c759-886b-4e66-8eb5-5e3dae2a0c83 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:31:14.321900 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b342233-652c-4437-b944-19fe58fd21aa admin admin] REQUEST_METHOD: `DELETE` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:31:14.322073 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ef8c759-886b-4e66-8eb5-5e3dae2a0c83 admin admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:31:14.322170 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b342233-652c-4437-b944-19fe58fd21aa admin admin] SCRIPT_NAME: `/identity` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:31:14.322351 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ef8c759-886b-4e66-8eb5-5e3dae2a0c83 admin admin] PATH_INFO: `/v3/users/a6bb5dd4e940419ab1efb69d34bdcd0c` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:31:14.322449 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b342233-652c-4437-b944-19fe58fd21aa admin admin] PATH_INFO: `/v3/users/2311fea5afca4496a5f2bcdb3cb7d6d5` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:31:14.323645 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b342233-652c-4437-b944-19fe58fd21aa admin admin] RBAC: Authorizing `identity:delete_user(user_id=2311fea5afca4496a5f2bcdb3cb7d6d5)` {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:31:14.323892 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ef8c759-886b-4e66-8eb5-5e3dae2a0c83 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a6bb5dd4e940419ab1efb69d34bdcd0c)` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:31:14.325771 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b342233-652c-4437-b944-19fe58fd21aa admin admin] RBAC: Authorization granted {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:31:14.325938 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ef8c759-886b-4e66-8eb5-5e3dae2a0c83 admin admin] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:31:14.359393 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG dbcounter [-] [80244] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=80244) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:31:14.376362 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG dbcounter [-] [80245] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=80245) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:31:14.383862 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.notifications [None req-8ef8c759-886b-4e66-8eb5-5e3dae2a0c83 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a6bb5dd4e940419ab1efb69d34bdcd0c'} {{(pid=80244) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 08:31:14.392415 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.notifications [None req-5b342233-652c-4437-b944-19fe58fd21aa admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2311fea5afca4496a5f2bcdb3cb7d6d5'} {{(pid=80245) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 08:31:14.392598 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.notifications [None req-8ef8c759-886b-4e66-8eb5-5e3dae2a0c83 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a6bb5dd4e940419ab1efb69d34bdcd0c'} {{(pid=80244) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 08:31:14.401467 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.notifications [None req-5b342233-652c-4437-b944-19fe58fd21aa admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2311fea5afca4496a5f2bcdb3cb7d6d5'} {{(pid=80245) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 08:31:14.413149 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.notifications [None req-8ef8c759-886b-4e66-8eb5-5e3dae2a0c83 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a6bb5dd4e940419ab1efb69d34bdcd0c'} {{(pid=80244) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 08:31:14.414366 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.notifications [None req-8ef8c759-886b-4e66-8eb5-5e3dae2a0c83 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a6bb5dd4e940419ab1efb69d34bdcd0c'} {{(pid=80244) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 08:31:14.415395 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.notifications [None req-5b342233-652c-4437-b944-19fe58fd21aa admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2311fea5afca4496a5f2bcdb3cb7d6d5'} {{(pid=80245) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 08:31:14.422678 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.notifications [None req-8ef8c759-886b-4e66-8eb5-5e3dae2a0c83 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a6bb5dd4e940419ab1efb69d34bdcd0c'} {{(pid=80244) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 08:31:14.423443 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.notifications [None req-5b342233-652c-4437-b944-19fe58fd21aa admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2311fea5afca4496a5f2bcdb3cb7d6d5'} {{(pid=80245) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 08:31:14.424146 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.notifications [None req-5b342233-652c-4437-b944-19fe58fd21aa admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2311fea5afca4496a5f2bcdb3cb7d6d5'} {{(pid=80245) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 08:31:14.428795 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 988/1960] 10.222.0.47 () {58 vars in 1223 bytes} [Thu Jul 21 08:31:14 2022] DELETE /identity/v3/users/a6bb5dd4e940419ab1efb69d34bdcd0c => generated 0 bytes in 233 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 08:31:14.432527 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ef8c759-886b-4e66-8eb5-5e3dae2a0c83 admin admin] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:31:14.432871 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: [pid: 80245|app: 0|req: 973/1961] 10.222.0.47 () {58 vars in 1223 bytes} [Thu Jul 21 08:31:14 2022] DELETE /identity/v3/users/2311fea5afca4496a5f2bcdb3cb7d6d5 => generated 0 bytes in 224 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 21 08:31:14.435001 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.fernet_utils [None req-8ef8c759-886b-4e66-8eb5-5e3dae2a0c83 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80244) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 08:31:14.534684 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ef8c759-886b-4e66-8eb5-5e3dae2a0c83 admin admin] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:31:14.629346 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70115730-6baa-4d9c-a316-b271b22a31d3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '42d15fc0515945acacbbffcf2356c7a5', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:31:14.630770 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70115730-6baa-4d9c-a316-b271b22a31d3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:31:14.631447 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70115730-6baa-4d9c-a316-b271b22a31d3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:31:14.631979 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70115730-6baa-4d9c-a316-b271b22a31d3 admin admin] PATH_INFO: `/v3/users/ffe1270415354f1caff259ad7bc7b956` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:31:14.633429 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70115730-6baa-4d9c-a316-b271b22a31d3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ffe1270415354f1caff259ad7bc7b956)` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:31:14.636334 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70115730-6baa-4d9c-a316-b271b22a31d3 admin admin] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:31:14.701987 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.notifications [None req-70115730-6baa-4d9c-a316-b271b22a31d3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ffe1270415354f1caff259ad7bc7b956'} {{(pid=80244) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 08:31:14.707254 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG dbcounter [-] [80244] Writing DB stats keystone:DELETE=14,keystone:SELECT=84,keystone:INSERT=2 {{(pid=80244) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:31:14.714248 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.notifications [None req-70115730-6baa-4d9c-a316-b271b22a31d3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ffe1270415354f1caff259ad7bc7b956'} {{(pid=80244) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 08:31:14.757866 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.notifications [None req-70115730-6baa-4d9c-a316-b271b22a31d3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ffe1270415354f1caff259ad7bc7b956'} {{(pid=80244) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 08:31:14.759819 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.notifications [None req-70115730-6baa-4d9c-a316-b271b22a31d3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ffe1270415354f1caff259ad7bc7b956'} {{(pid=80244) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 08:31:14.767990 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.notifications [None req-70115730-6baa-4d9c-a316-b271b22a31d3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ffe1270415354f1caff259ad7bc7b956'} {{(pid=80244) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 08:31:14.777223 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 989/1962] 10.222.0.47 () {58 vars in 1223 bytes} [Thu Jul 21 08:31:14 2022] DELETE /identity/v3/users/ffe1270415354f1caff259ad7bc7b956 => generated 0 bytes in 345 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 08:31:14.781834 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2579378c-c51b-472f-8da9-d0a4aca17ce9 None None] REQUEST_METHOD: `POST` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:31:14.782134 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2579378c-c51b-472f-8da9-d0a4aca17ce9 None None] SCRIPT_NAME: `/identity` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:31:14.782411 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2579378c-c51b-472f-8da9-d0a4aca17ce9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:31:14.806850 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.auth.core [None req-2579378c-c51b-472f-8da9-d0a4aca17ce9 None None] MFA Rules not processed for user `2dce40b8d74c41ec94cc36e6ff05d412`. Rule list: `[]` (Enabled: `True`). {{(pid=80245) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 08:31:14.808103 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.fernet_utils [None req-2579378c-c51b-472f-8da9-d0a4aca17ce9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80245) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 08:31:14.880259 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70115730-6baa-4d9c-a316-b271b22a31d3 admin admin] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:31:14.881203 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.fernet_utils [None req-70115730-6baa-4d9c-a316-b271b22a31d3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80244) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 08:31:14.898511 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: [pid: 80245|app: 0|req: 974/1963] 10.222.0.47 () {58 vars in 957 bytes} [Thu Jul 21 08:31:14 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 21 08:31:14.898511 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2579378c-c51b-472f-8da9-d0a4aca17ce9 None None] Authenticating user token {{(pid=80245) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:31:14.899419 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.fernet_utils [None req-2579378c-c51b-472f-8da9-d0a4aca17ce9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80245) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 08:31:14.915785 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2579378c-c51b-472f-8da9-d0a4aca17ce9 None None] Validating token access rules against request {{(pid=80245) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:31:14.920545 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc4d2e5d-d5a0-43c2-acc0-adf3a67e81d1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '42d15fc0515945acacbbffcf2356c7a5', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80245) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:31:14.921509 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc4d2e5d-d5a0-43c2-acc0-adf3a67e81d1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:31:14.922017 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc4d2e5d-d5a0-43c2-acc0-adf3a67e81d1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:31:14.922385 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc4d2e5d-d5a0-43c2-acc0-adf3a67e81d1 admin admin] PATH_INFO: `/v3/projects/b31f7a328f9b46728a64a50c1c6da9de` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:31:14.923701 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc4d2e5d-d5a0-43c2-acc0-adf3a67e81d1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b31f7a328f9b46728a64a50c1c6da9de)` {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:31:14.925631 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc4d2e5d-d5a0-43c2-acc0-adf3a67e81d1 admin admin] RBAC: Authorization granted {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:31:14.984541 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.notifications [None req-bc4d2e5d-d5a0-43c2-acc0-adf3a67e81d1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b31f7a328f9b46728a64a50c1c6da9de', 'request_id': 'req-bc4d2e5d-d5a0-43c2-acc0-adf3a67e81d1'} {{(pid=80245) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 08:31:14.993884 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70115730-6baa-4d9c-a316-b271b22a31d3 admin admin] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:31:14.994045 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.notifications [None req-bc4d2e5d-d5a0-43c2-acc0-adf3a67e81d1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b31f7a328f9b46728a64a50c1c6da9de', 'request_id': 'req-bc4d2e5d-d5a0-43c2-acc0-adf3a67e81d1'} {{(pid=80245) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 08:31:15.000629 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.notifications [None req-bc4d2e5d-d5a0-43c2-acc0-adf3a67e81d1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b31f7a328f9b46728a64a50c1c6da9de', 'request_id': 'req-bc4d2e5d-d5a0-43c2-acc0-adf3a67e81d1'} {{(pid=80245) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 08:31:15.014456 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.notifications [None req-bc4d2e5d-d5a0-43c2-acc0-adf3a67e81d1 admin admin] The token cache is being invalidate because project b31f7a328f9b46728a64a50c1c6da9de was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80245) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 21 08:31:15.014753 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.notifications [None req-bc4d2e5d-d5a0-43c2-acc0-adf3a67e81d1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80245) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 08:31:15.015987 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.notifications [None req-bc4d2e5d-d5a0-43c2-acc0-adf3a67e81d1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80245) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 08:31:15.017260 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: [pid: 80245|app: 0|req: 975/1964] 10.222.0.47 () {58 vars in 1232 bytes} [Thu Jul 21 08:31:14 2022] DELETE /identity/v3/projects/b31f7a328f9b46728a64a50c1c6da9de => generated 0 bytes in 121 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 08:31:15.018254 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc4d2e5d-d5a0-43c2-acc0-adf3a67e81d1 admin admin] Authenticating user token {{(pid=80245) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:31:15.019649 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.fernet_utils [None req-bc4d2e5d-d5a0-43c2-acc0-adf3a67e81d1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80245) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 08:31:15.056709 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e716909f-c3e6-43fb-b86a-a142641ca195 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '42d15fc0515945acacbbffcf2356c7a5', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:31:15.057575 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e716909f-c3e6-43fb-b86a-a142641ca195 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:31:15.057852 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e716909f-c3e6-43fb-b86a-a142641ca195 admin admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:31:15.058118 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e716909f-c3e6-43fb-b86a-a142641ca195 admin admin] PATH_INFO: `/v3/projects/fa64e380f53a4d999af86be40d4fa83e` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:31:15.059361 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e716909f-c3e6-43fb-b86a-a142641ca195 admin admin] RBAC: Authorizing `identity:delete_project(project_id=fa64e380f53a4d999af86be40d4fa83e)` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:31:15.061306 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e716909f-c3e6-43fb-b86a-a142641ca195 admin admin] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:31:15.062942 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG dbcounter [-] [80245] Writing DB stats keystone:SELECT=86,keystone:DELETE=12,keystone:INSERT=2 {{(pid=80245) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:31:15.082345 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc4d2e5d-d5a0-43c2-acc0-adf3a67e81d1 admin admin] Validating token access rules against request {{(pid=80245) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:31:15.086261 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77273d4f-6bd3-46d8-bdf5-45ebe3947f57 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20ca3c3bd3774691b3374cfbfec6e494', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4faef13aa50e4e36b6c265fb1398fa14', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80245) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:31:15.086933 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77273d4f-6bd3-46d8-bdf5-45ebe3947f57 service neutron] REQUEST_METHOD: `GET` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:31:15.087226 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77273d4f-6bd3-46d8-bdf5-45ebe3947f57 service neutron] SCRIPT_NAME: `/identity` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:31:15.087592 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77273d4f-6bd3-46d8-bdf5-45ebe3947f57 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:31:15.090268 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.fernet_utils [None req-77273d4f-6bd3-46d8-bdf5-45ebe3947f57 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80245) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 08:31:15.104022 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77273d4f-6bd3-46d8-bdf5-45ebe3947f57 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:31:15.105469 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77273d4f-6bd3-46d8-bdf5-45ebe3947f57 service neutron] RBAC: Authorization granted {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:31:15.112714 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.notifications [None req-e716909f-c3e6-43fb-b86a-a142641ca195 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'fa64e380f53a4d999af86be40d4fa83e', 'request_id': 'req-e716909f-c3e6-43fb-b86a-a142641ca195'} {{(pid=80244) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 08:31:15.115110 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: [pid: 80245|app: 0|req: 976/1965] 10.222.0.47 () {60 vars in 1365 bytes} [Thu Jul 21 08:31:15 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 08:31:15.116429 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e35dbdc-ef8c-47dd-80d2-a8d9a4fcad02 None None] REQUEST_METHOD: `POST` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:31:15.116719 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e35dbdc-ef8c-47dd-80d2-a8d9a4fcad02 None None] SCRIPT_NAME: `/identity` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:31:15.117000 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e35dbdc-ef8c-47dd-80d2-a8d9a4fcad02 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:31:15.121945 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.notifications [None req-e716909f-c3e6-43fb-b86a-a142641ca195 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'fa64e380f53a4d999af86be40d4fa83e', 'request_id': 'req-e716909f-c3e6-43fb-b86a-a142641ca195'} {{(pid=80244) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 08:31:15.131074 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.notifications [None req-e716909f-c3e6-43fb-b86a-a142641ca195 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'fa64e380f53a4d999af86be40d4fa83e', 'request_id': 'req-e716909f-c3e6-43fb-b86a-a142641ca195'} {{(pid=80244) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 08:31:15.139478 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.auth.core [None req-9e35dbdc-ef8c-47dd-80d2-a8d9a4fcad02 None None] MFA Rules not processed for user `2dce40b8d74c41ec94cc36e6ff05d412`. Rule list: `[]` (Enabled: `True`). {{(pid=80245) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 08:31:15.140928 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.fernet_utils [None req-9e35dbdc-ef8c-47dd-80d2-a8d9a4fcad02 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80245) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 08:31:15.144170 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.notifications [None req-e716909f-c3e6-43fb-b86a-a142641ca195 admin admin] The token cache is being invalidate because project fa64e380f53a4d999af86be40d4fa83e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80244) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 21 08:31:15.144460 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.notifications [None req-e716909f-c3e6-43fb-b86a-a142641ca195 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80244) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 08:31:15.145129 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.notifications [None req-e716909f-c3e6-43fb-b86a-a142641ca195 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80244) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 08:31:15.147114 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 990/1966] 10.222.0.47 () {58 vars in 1232 bytes} [Thu Jul 21 08:31:14 2022] DELETE /identity/v3/projects/fa64e380f53a4d999af86be40d4fa83e => generated 0 bytes in 269 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 21 08:31:15.152552 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a50de1c7-1ec3-4fa8-accb-bf70dbf2f3d5 None None] REQUEST_METHOD: `POST` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:31:15.153472 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a50de1c7-1ec3-4fa8-accb-bf70dbf2f3d5 None None] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:31:15.153896 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a50de1c7-1ec3-4fa8-accb-bf70dbf2f3d5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:31:15.172873 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG dbcounter [-] [80244] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=80244) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:31:15.179802 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.auth.core [None req-a50de1c7-1ec3-4fa8-accb-bf70dbf2f3d5 None None] MFA Rules not processed for user `2dce40b8d74c41ec94cc36e6ff05d412`. Rule list: `[]` (Enabled: `True`). {{(pid=80244) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 08:31:15.181357 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.fernet_utils [None req-a50de1c7-1ec3-4fa8-accb-bf70dbf2f3d5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80244) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 08:31:15.216771 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: [pid: 80245|app: 0|req: 977/1967] 10.222.0.47 () {58 vars in 957 bytes} [Thu Jul 21 08:31:15 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 08:31:15.224878 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c149b40c-d4cc-4d7d-bcb6-bb95e76926cd None None] REQUEST_METHOD: `POST` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:31:15.225361 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c149b40c-d4cc-4d7d-bcb6-bb95e76926cd None None] SCRIPT_NAME: `/identity` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:31:15.225705 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c149b40c-d4cc-4d7d-bcb6-bb95e76926cd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:31:15.244585 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.auth.core [None req-c149b40c-d4cc-4d7d-bcb6-bb95e76926cd None None] MFA Rules not processed for user `2dce40b8d74c41ec94cc36e6ff05d412`. Rule list: `[]` (Enabled: `True`). {{(pid=80245) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 08:31:15.246113 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.fernet_utils [None req-c149b40c-d4cc-4d7d-bcb6-bb95e76926cd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80245) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 08:31:15.264141 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: [pid: 80245|app: 0|req: 978/1968] 10.222.0.47 () {58 vars in 957 bytes} [Thu Jul 21 08:31:15 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 40 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 21 08:31:15.269360 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c149b40c-d4cc-4d7d-bcb6-bb95e76926cd None None] Authenticating user token {{(pid=80245) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:31:15.290322 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 991/1969] 10.222.0.47 () {58 vars in 957 bytes} [Thu Jul 21 08:31:15 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 08:31:15.291367 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c149b40c-d4cc-4d7d-bcb6-bb95e76926cd None None] Validating token access rules against request {{(pid=80245) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:31:15.291367 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8499db5d-b629-4112-9dd3-ed871b906161 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '42d15fc0515945acacbbffcf2356c7a5', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80245) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:31:15.291367 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8499db5d-b629-4112-9dd3-ed871b906161 admin admin] REQUEST_METHOD: `GET` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:31:15.291367 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8499db5d-b629-4112-9dd3-ed871b906161 admin admin] SCRIPT_NAME: `/identity` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:31:15.291824 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8499db5d-b629-4112-9dd3-ed871b906161 admin admin] PATH_INFO: `/v3/domains` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:31:15.291968 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cf1652a-95b2-469c-b087-688423e55686 None None] REQUEST_METHOD: `POST` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:31:15.292305 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cf1652a-95b2-469c-b087-688423e55686 None None] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:31:15.292446 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cf1652a-95b2-469c-b087-688423e55686 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:31:15.302889 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8499db5d-b629-4112-9dd3-ed871b906161 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80245) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 08:31:15.303222 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8499db5d-b629-4112-9dd3-ed871b906161 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:31:15.308797 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8499db5d-b629-4112-9dd3-ed871b906161 admin admin] RBAC: Authorization granted {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:31:15.322635 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: [pid: 80245|app: 0|req: 979/1970] 10.222.0.47 () {58 vars in 1152 bytes} [Thu Jul 21 08:31:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 54 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 21 08:31:15.328130 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8499db5d-b629-4112-9dd3-ed871b906161 admin admin] Authenticating user token {{(pid=80245) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:31:15.332096 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.auth.core [None req-0cf1652a-95b2-469c-b087-688423e55686 None None] MFA Rules not processed for user `2dce40b8d74c41ec94cc36e6ff05d412`. Rule list: `[]` (Enabled: `True`). {{(pid=80244) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 08:31:15.335373 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.fernet_utils [None req-0cf1652a-95b2-469c-b087-688423e55686 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80244) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 08:31:15.342554 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8499db5d-b629-4112-9dd3-ed871b906161 admin admin] Validating token access rules against request {{(pid=80245) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:31:15.347525 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d1aa9ed-ad58-4944-96e8-aa3f3d365651 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '42d15fc0515945acacbbffcf2356c7a5', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80245) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:31:15.348353 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d1aa9ed-ad58-4944-96e8-aa3f3d365651 admin admin] REQUEST_METHOD: `POST` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:31:15.348641 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d1aa9ed-ad58-4944-96e8-aa3f3d365651 admin admin] SCRIPT_NAME: `/identity` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:31:15.348920 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d1aa9ed-ad58-4944-96e8-aa3f3d365651 admin admin] PATH_INFO: `/v3/projects` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:31:15.349768 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d1aa9ed-ad58-4944-96e8-aa3f3d365651 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:31:15.351960 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d1aa9ed-ad58-4944-96e8-aa3f3d365651 admin admin] RBAC: Authorization granted {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:31:15.353249 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 992/1971] 10.222.0.47 () {58 vars in 957 bytes} [Thu Jul 21 08:31:15 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 62 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 21 08:31:15.359028 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cf1652a-95b2-469c-b087-688423e55686 None None] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:31:15.375852 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cf1652a-95b2-469c-b087-688423e55686 None None] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:31:15.383393 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe958df8-c3a4-441a-b79a-2c7c4eba4cde admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '42d15fc0515945acacbbffcf2356c7a5', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:31:15.384153 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe958df8-c3a4-441a-b79a-2c7c4eba4cde admin admin] REQUEST_METHOD: `GET` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:31:15.384442 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe958df8-c3a4-441a-b79a-2c7c4eba4cde admin admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:31:15.384720 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe958df8-c3a4-441a-b79a-2c7c4eba4cde admin admin] PATH_INFO: `/v3/domains` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:31:15.385470 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe958df8-c3a4-441a-b79a-2c7c4eba4cde admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80244) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 08:31:15.385780 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe958df8-c3a4-441a-b79a-2c7c4eba4cde admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:31:15.388114 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe958df8-c3a4-441a-b79a-2c7c4eba4cde admin admin] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:31:15.403544 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: [pid: 80245|app: 0|req: 980/1972] 10.222.0.47 () {60 vars in 1152 bytes} [Thu Jul 21 08:31:15 2022] POST /identity/v3/projects => generated 381 bytes in 76 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 08:31:15.407484 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d1aa9ed-ad58-4944-96e8-aa3f3d365651 admin admin] Authenticating user token {{(pid=80245) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:31:15.408408 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 993/1973] 10.222.0.47 () {58 vars in 1152 bytes} [Thu Jul 21 08:31:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 50 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 21 08:31:15.412269 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe958df8-c3a4-441a-b79a-2c7c4eba4cde admin admin] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:31:15.428275 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe958df8-c3a4-441a-b79a-2c7c4eba4cde admin admin] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:31:15.529841 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d1aa9ed-ad58-4944-96e8-aa3f3d365651 admin admin] Validating token access rules against request {{(pid=80245) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:31:15.536179 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7348a646-16d1-4e12-94aa-6249ea670e54 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '42d15fc0515945acacbbffcf2356c7a5', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80245) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:31:15.537189 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7348a646-16d1-4e12-94aa-6249ea670e54 admin admin] REQUEST_METHOD: `POST` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:31:15.537495 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7348a646-16d1-4e12-94aa-6249ea670e54 admin admin] SCRIPT_NAME: `/identity` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:31:15.537801 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7348a646-16d1-4e12-94aa-6249ea670e54 admin admin] PATH_INFO: `/v3/users` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:31:15.540713 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7348a646-16d1-4e12-94aa-6249ea670e54 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:31:15.540862 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7348a646-16d1-4e12-94aa-6249ea670e54 admin admin] RBAC: Authorization granted {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:31:15.547434 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f1cd2a9-7843-4c57-8cbd-111171a35ec4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '42d15fc0515945acacbbffcf2356c7a5', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:31:15.548886 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f1cd2a9-7843-4c57-8cbd-111171a35ec4 admin admin] REQUEST_METHOD: `POST` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:31:15.549463 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f1cd2a9-7843-4c57-8cbd-111171a35ec4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:31:15.549750 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f1cd2a9-7843-4c57-8cbd-111171a35ec4 admin admin] PATH_INFO: `/v3/projects` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:31:15.550560 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f1cd2a9-7843-4c57-8cbd-111171a35ec4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:31:15.552650 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f1cd2a9-7843-4c57-8cbd-111171a35ec4 admin admin] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:31:15.563522 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: [pid: 80245|app: 0|req: 981/1974] 10.222.0.47 () {60 vars in 1143 bytes} [Thu Jul 21 08:31:15 2022] POST /identity/v3/users => generated 291 bytes in 157 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 08:31:15.568733 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7348a646-16d1-4e12-94aa-6249ea670e54 admin admin] Authenticating user token {{(pid=80245) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:31:15.584236 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7348a646-16d1-4e12-94aa-6249ea670e54 admin admin] Validating token access rules against request {{(pid=80245) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:31:15.588010 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 994/1975] 10.222.0.47 () {60 vars in 1152 bytes} [Thu Jul 21 08:31:15 2022] POST /identity/v3/projects => generated 383 bytes in 176 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 21 08:31:15.589214 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f1cd2a9-7843-4c57-8cbd-111171a35ec4 admin admin] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:31:15.589534 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdee024e-b09d-40db-b293-4a30ae79e4d4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '42d15fc0515945acacbbffcf2356c7a5', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80245) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:31:15.590255 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdee024e-b09d-40db-b293-4a30ae79e4d4 admin admin] REQUEST_METHOD: `GET` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:31:15.590653 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdee024e-b09d-40db-b293-4a30ae79e4d4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:31:15.591234 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdee024e-b09d-40db-b293-4a30ae79e4d4 admin admin] PATH_INFO: `/v3/roles` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:31:15.592011 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdee024e-b09d-40db-b293-4a30ae79e4d4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:31:15.592296 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.fernet_utils [None req-3f1cd2a9-7843-4c57-8cbd-111171a35ec4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80244) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 08:31:15.593659 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdee024e-b09d-40db-b293-4a30ae79e4d4 admin admin] RBAC: Authorization granted {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:31:15.604915 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: [pid: 80245|app: 0|req: 982/1976] 10.222.0.47 () {58 vars in 1121 bytes} [Thu Jul 21 08:31:15 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 21 08:31:15.609046 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdee024e-b09d-40db-b293-4a30ae79e4d4 admin admin] Authenticating user token {{(pid=80245) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:31:15.634776 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdee024e-b09d-40db-b293-4a30ae79e4d4 admin admin] Validating token access rules against request {{(pid=80245) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:31:15.648974 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG dbcounter [-] [80245] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80245) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:31:15.686337 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f1cd2a9-7843-4c57-8cbd-111171a35ec4 admin admin] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:31:15.691194 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8684c50-8601-4c47-bd6d-59d39e6f19c4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '42d15fc0515945acacbbffcf2356c7a5', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:31:15.692230 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8684c50-8601-4c47-bd6d-59d39e6f19c4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:31:15.692515 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8684c50-8601-4c47-bd6d-59d39e6f19c4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:31:15.692818 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8684c50-8601-4c47-bd6d-59d39e6f19c4 admin admin] PATH_INFO: `/v3/projects/d0c25b6392a740d9b324c8a780a08830` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:31:15.694015 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8684c50-8601-4c47-bd6d-59d39e6f19c4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d0c25b6392a740d9b324c8a780a08830)` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:31:15.696070 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8684c50-8601-4c47-bd6d-59d39e6f19c4 admin admin] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:31:15.698058 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86965e49-2700-47ab-8f67-a23ec7ac6efc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '42d15fc0515945acacbbffcf2356c7a5', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80245) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:31:15.698772 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86965e49-2700-47ab-8f67-a23ec7ac6efc admin admin] REQUEST_METHOD: `POST` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:31:15.699060 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86965e49-2700-47ab-8f67-a23ec7ac6efc admin admin] SCRIPT_NAME: `/identity` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:31:15.699366 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86965e49-2700-47ab-8f67-a23ec7ac6efc admin admin] PATH_INFO: `/v3/users` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:31:15.700082 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86965e49-2700-47ab-8f67-a23ec7ac6efc admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:31:15.702018 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86965e49-2700-47ab-8f67-a23ec7ac6efc admin admin] RBAC: Authorization granted {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:31:15.712414 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG dbcounter [-] [80244] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80244) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:31:15.725773 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: [pid: 80245|app: 0|req: 983/1977] 10.222.0.47 () {60 vars in 1143 bytes} [Thu Jul 21 08:31:15 2022] POST /identity/v3/users => generated 292 bytes in 117 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 08:31:15.725773 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86965e49-2700-47ab-8f67-a23ec7ac6efc admin admin] Authenticating user token {{(pid=80245) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:31:15.734163 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86965e49-2700-47ab-8f67-a23ec7ac6efc admin admin] Validating token access rules against request {{(pid=80245) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:31:15.748063 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.notifications [None req-f8684c50-8601-4c47-bd6d-59d39e6f19c4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd0c25b6392a740d9b324c8a780a08830', 'request_id': 'req-f8684c50-8601-4c47-bd6d-59d39e6f19c4'} {{(pid=80244) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 08:31:15.756503 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.notifications [None req-f8684c50-8601-4c47-bd6d-59d39e6f19c4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd0c25b6392a740d9b324c8a780a08830', 'request_id': 'req-f8684c50-8601-4c47-bd6d-59d39e6f19c4'} {{(pid=80244) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 08:31:15.763502 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.notifications [None req-f8684c50-8601-4c47-bd6d-59d39e6f19c4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd0c25b6392a740d9b324c8a780a08830', 'request_id': 'req-f8684c50-8601-4c47-bd6d-59d39e6f19c4'} {{(pid=80244) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 08:31:15.775469 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.notifications [None req-f8684c50-8601-4c47-bd6d-59d39e6f19c4 admin admin] The token cache is being invalidate because project d0c25b6392a740d9b324c8a780a08830 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80244) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 21 08:31:15.775967 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.notifications [None req-f8684c50-8601-4c47-bd6d-59d39e6f19c4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80244) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 08:31:15.776650 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.notifications [None req-f8684c50-8601-4c47-bd6d-59d39e6f19c4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80244) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 08:31:15.778257 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 995/1978] 10.222.0.47 () {58 vars in 1232 bytes} [Thu Jul 21 08:31:15 2022] DELETE /identity/v3/projects/d0c25b6392a740d9b324c8a780a08830 => generated 0 bytes in 189 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 08:31:15.781510 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8684c50-8601-4c47-bd6d-59d39e6f19c4 admin admin] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:31:15.782411 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.fernet_utils [None req-f8684c50-8601-4c47-bd6d-59d39e6f19c4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80244) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 08:31:15.801212 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd2b3151-2384-4609-a08a-97cdcc6536fe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '42d15fc0515945acacbbffcf2356c7a5', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80245) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:31:15.801895 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd2b3151-2384-4609-a08a-97cdcc6536fe admin admin] REQUEST_METHOD: `GET` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:31:15.802177 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd2b3151-2384-4609-a08a-97cdcc6536fe admin admin] SCRIPT_NAME: `/identity` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:31:15.802451 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd2b3151-2384-4609-a08a-97cdcc6536fe admin admin] PATH_INFO: `/v3/roles` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:31:15.803206 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd2b3151-2384-4609-a08a-97cdcc6536fe admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:31:15.804956 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd2b3151-2384-4609-a08a-97cdcc6536fe admin admin] RBAC: Authorization granted {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:31:15.820936 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: [pid: 80245|app: 0|req: 984/1979] 10.222.0.47 () {58 vars in 1121 bytes} [Thu Jul 21 08:31:15 2022] GET /identity/v3/roles => generated 1420 bytes in 96 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 08:31:15.820936 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd2b3151-2384-4609-a08a-97cdcc6536fe admin admin] Authenticating user token {{(pid=80245) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:31:15.822766 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.fernet_utils [None req-cd2b3151-2384-4609-a08a-97cdcc6536fe admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80245) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 08:31:15.842367 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd2b3151-2384-4609-a08a-97cdcc6536fe admin admin] Validating token access rules against request {{(pid=80245) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:31:15.849506 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58e0feca-4c32-40e0-b6cf-c030ccb7f30a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '42d15fc0515945acacbbffcf2356c7a5', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80245) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:31:15.850502 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58e0feca-4c32-40e0-b6cf-c030ccb7f30a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:31:15.850792 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58e0feca-4c32-40e0-b6cf-c030ccb7f30a admin admin] SCRIPT_NAME: `/identity` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:31:15.851107 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58e0feca-4c32-40e0-b6cf-c030ccb7f30a admin admin] PATH_INFO: `/v3/users/39a796af3051487f8d97c207896e7e97` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:31:15.852490 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58e0feca-4c32-40e0-b6cf-c030ccb7f30a admin admin] RBAC: Authorizing `identity:delete_user(user_id=39a796af3051487f8d97c207896e7e97)` {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:31:15.854891 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58e0feca-4c32-40e0-b6cf-c030ccb7f30a admin admin] RBAC: Authorization granted {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:31:15.860765 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8684c50-8601-4c47-bd6d-59d39e6f19c4 admin admin] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:31:15.866570 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1461acbe-b651-4a97-bcdd-ba1218d590e2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '42d15fc0515945acacbbffcf2356c7a5', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:31:15.867469 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1461acbe-b651-4a97-bcdd-ba1218d590e2 admin admin] REQUEST_METHOD: `GET` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:31:15.867761 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1461acbe-b651-4a97-bcdd-ba1218d590e2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:31:15.868040 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1461acbe-b651-4a97-bcdd-ba1218d590e2 admin admin] PATH_INFO: `/v3/roles` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:31:15.868786 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1461acbe-b651-4a97-bcdd-ba1218d590e2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:31:15.870728 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1461acbe-b651-4a97-bcdd-ba1218d590e2 admin admin] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:31:15.887106 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 996/1980] 10.222.0.47 () {58 vars in 1121 bytes} [Thu Jul 21 08:31:15 2022] GET /identity/v3/roles => generated 1420 bytes in 106 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 08:31:15.887106 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1461acbe-b651-4a97-bcdd-ba1218d590e2 admin admin] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:31:15.887676 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.fernet_utils [None req-1461acbe-b651-4a97-bcdd-ba1218d590e2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80244) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 08:31:15.907202 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1461acbe-b651-4a97-bcdd-ba1218d590e2 admin admin] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:31:15.915731 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26baf0ae-cc97-4e4f-b1fe-735697a9c0f3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '42d15fc0515945acacbbffcf2356c7a5', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:31:15.917101 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26baf0ae-cc97-4e4f-b1fe-735697a9c0f3 admin admin] REQUEST_METHOD: `PUT` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:31:15.917397 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26baf0ae-cc97-4e4f-b1fe-735697a9c0f3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:31:15.917678 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26baf0ae-cc97-4e4f-b1fe-735697a9c0f3 admin admin] PATH_INFO: `/v3/projects/08a358f65b42429d92b327574b6f4f7c/users/afe5dd01ff9c46b18a6835cc582faa32/roles/3a074738221e4601b5205715ba87d240` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:31:15.940131 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.notifications [None req-58e0feca-4c32-40e0-b6cf-c030ccb7f30a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '39a796af3051487f8d97c207896e7e97'} {{(pid=80245) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 08:31:15.940757 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26baf0ae-cc97-4e4f-b1fe-735697a9c0f3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=08a358f65b42429d92b327574b6f4f7c, user_id=afe5dd01ff9c46b18a6835cc582faa32, role_id=3a074738221e4601b5205715ba87d240)` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:31:15.941856 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.notifications [None req-58e0feca-4c32-40e0-b6cf-c030ccb7f30a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '39a796af3051487f8d97c207896e7e97'} {{(pid=80245) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 08:31:15.944332 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26baf0ae-cc97-4e4f-b1fe-735697a9c0f3 admin admin] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:31:15.956732 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.notifications [None req-58e0feca-4c32-40e0-b6cf-c030ccb7f30a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '39a796af3051487f8d97c207896e7e97'} {{(pid=80245) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 08:31:15.969161 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 997/1981] 10.222.0.47 () {60 vars in 1482 bytes} [Thu Jul 21 08:31:15 2022] PUT /identity/v3/projects/08a358f65b42429d92b327574b6f4f7c/users/afe5dd01ff9c46b18a6835cc582faa32/roles/3a074738221e4601b5205715ba87d240 => generated 0 bytes in 75 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 08:31:15.969161 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26baf0ae-cc97-4e4f-b1fe-735697a9c0f3 admin admin] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:31:15.972511 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.notifications [None req-58e0feca-4c32-40e0-b6cf-c030ccb7f30a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '39a796af3051487f8d97c207896e7e97'} {{(pid=80245) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 08:31:15.973229 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.notifications [None req-58e0feca-4c32-40e0-b6cf-c030ccb7f30a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '39a796af3051487f8d97c207896e7e97'} {{(pid=80245) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 08:31:15.980576 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: [pid: 80245|app: 0|req: 985/1982] 10.222.0.47 () {58 vars in 1223 bytes} [Thu Jul 21 08:31:15 2022] DELETE /identity/v3/users/39a796af3051487f8d97c207896e7e97 => generated 0 bytes in 160 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 08:31:15.983995 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94cd75e0-214b-48c8-8906-c265c26195bf None None] REQUEST_METHOD: `POST` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:31:15.987373 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94cd75e0-214b-48c8-8906-c265c26195bf None None] SCRIPT_NAME: `/identity` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:31:15.987840 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94cd75e0-214b-48c8-8906-c265c26195bf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:31:16.029350 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.auth.core [None req-94cd75e0-214b-48c8-8906-c265c26195bf None None] MFA Rules not processed for user `afe5dd01ff9c46b18a6835cc582faa32`. Rule list: `[]` (Enabled: `True`). {{(pid=80245) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 08:31:16.031237 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.fernet_utils [None req-94cd75e0-214b-48c8-8906-c265c26195bf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80245) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 08:31:16.035806 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG dbcounter [-] [80245] Writing DB stats keystone:SELECT=86,keystone:INSERT=4,keystone:DELETE=10 {{(pid=80245) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:31:16.051212 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26baf0ae-cc97-4e4f-b1fe-735697a9c0f3 admin admin] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:31:16.075031 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG dbcounter [-] [80244] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80244) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:31:16.110037 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: [pid: 80245|app: 0|req: 986/1983] 10.222.0.47 () {58 vars in 957 bytes} [Thu Jul 21 08:31:15 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 08:31:16.114682 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94cd75e0-214b-48c8-8906-c265c26195bf None None] Authenticating user token {{(pid=80245) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:31:16.116893 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.fernet_utils [None req-94cd75e0-214b-48c8-8906-c265c26195bf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80245) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 08:31:16.127708 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2783455-c3d7-409f-a027-847186582b03 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '42d15fc0515945acacbbffcf2356c7a5', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:31:16.128629 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2783455-c3d7-409f-a027-847186582b03 admin admin] REQUEST_METHOD: `GET` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:31:16.129014 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2783455-c3d7-409f-a027-847186582b03 admin admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:31:16.129846 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2783455-c3d7-409f-a027-847186582b03 admin admin] PATH_INFO: `/v3/roles` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:31:16.130683 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2783455-c3d7-409f-a027-847186582b03 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:31:16.132452 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2783455-c3d7-409f-a027-847186582b03 admin admin] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:31:16.146405 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 998/1984] 10.222.0.47 () {58 vars in 1121 bytes} [Thu Jul 21 08:31:15 2022] GET /identity/v3/roles => generated 1420 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 08:31:16.147736 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2783455-c3d7-409f-a027-847186582b03 admin admin] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:31:16.149682 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.fernet_utils [None req-d2783455-c3d7-409f-a027-847186582b03 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80244) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 08:31:16.163128 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2783455-c3d7-409f-a027-847186582b03 admin admin] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:31:16.169063 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27a2d6c5-3a8d-4934-8297-2e5f48e2e4c9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '42d15fc0515945acacbbffcf2356c7a5', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:31:16.170092 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27a2d6c5-3a8d-4934-8297-2e5f48e2e4c9 admin admin] REQUEST_METHOD: `PUT` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:31:16.170508 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27a2d6c5-3a8d-4934-8297-2e5f48e2e4c9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:31:16.170905 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27a2d6c5-3a8d-4934-8297-2e5f48e2e4c9 admin admin] PATH_INFO: `/v3/projects/02978068e4fd4beca2e126b8d06d2b7c/users/848c949f57a34c84bb40c86cd4729d11/roles/3a074738221e4601b5205715ba87d240` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:31:16.188837 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27a2d6c5-3a8d-4934-8297-2e5f48e2e4c9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=02978068e4fd4beca2e126b8d06d2b7c, user_id=848c949f57a34c84bb40c86cd4729d11, role_id=3a074738221e4601b5205715ba87d240)` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:31:16.191997 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27a2d6c5-3a8d-4934-8297-2e5f48e2e4c9 admin admin] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:31:16.199098 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94cd75e0-214b-48c8-8906-c265c26195bf None None] Validating token access rules against request {{(pid=80245) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:31:16.205798 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c63954e5-032f-4e12-8073-bbad025ca47b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '42d15fc0515945acacbbffcf2356c7a5', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80245) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:31:16.206613 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c63954e5-032f-4e12-8073-bbad025ca47b admin admin] REQUEST_METHOD: `POST` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:31:16.207150 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c63954e5-032f-4e12-8073-bbad025ca47b admin admin] SCRIPT_NAME: `/identity` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:31:16.207662 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 999/1985] 10.222.0.47 () {60 vars in 1482 bytes} [Thu Jul 21 08:31:16 2022] PUT /identity/v3/projects/02978068e4fd4beca2e126b8d06d2b7c/users/848c949f57a34c84bb40c86cd4729d11/roles/3a074738221e4601b5205715ba87d240 => generated 0 bytes in 60 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 08:31:16.207793 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c63954e5-032f-4e12-8073-bbad025ca47b admin admin] PATH_INFO: `/v3/projects` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:31:16.208677 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c63954e5-032f-4e12-8073-bbad025ca47b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:31:16.212054 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c63954e5-032f-4e12-8073-bbad025ca47b admin admin] RBAC: Authorization granted {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:31:16.218375 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-917611f3-2cf9-4460-bb8d-37c0fea46b67 None None] REQUEST_METHOD: `POST` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:31:16.218766 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-917611f3-2cf9-4460-bb8d-37c0fea46b67 None None] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:31:16.219314 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-917611f3-2cf9-4460-bb8d-37c0fea46b67 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:31:16.247005 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: [pid: 80245|app: 0|req: 987/1986] 10.222.0.47 () {60 vars in 1152 bytes} [Thu Jul 21 08:31:16 2022] POST /identity/v3/projects => generated 383 bytes in 133 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 21 08:31:16.251361 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c63954e5-032f-4e12-8073-bbad025ca47b admin admin] Authenticating user token {{(pid=80245) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:31:16.254509 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.auth.core [None req-917611f3-2cf9-4460-bb8d-37c0fea46b67 None None] MFA Rules not processed for user `848c949f57a34c84bb40c86cd4729d11`. Rule list: `[]` (Enabled: `True`). {{(pid=80244) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 08:31:16.255755 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.fernet_utils [None req-917611f3-2cf9-4460-bb8d-37c0fea46b67 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80244) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 08:31:16.331577 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c63954e5-032f-4e12-8073-bbad025ca47b admin admin] Validating token access rules against request {{(pid=80245) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:31:16.340934 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 1000/1987] 10.222.0.47 () {58 vars in 957 bytes} [Thu Jul 21 08:31:16 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 21 08:31:16.341206 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d613b10-9c9a-48c4-a506-2ede09d9db85 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '42d15fc0515945acacbbffcf2356c7a5', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80245) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:31:16.341314 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-917611f3-2cf9-4460-bb8d-37c0fea46b67 None None] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:31:16.341460 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d613b10-9c9a-48c4-a506-2ede09d9db85 admin admin] REQUEST_METHOD: `POST` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:31:16.341460 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d613b10-9c9a-48c4-a506-2ede09d9db85 admin admin] SCRIPT_NAME: `/identity` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:31:16.341460 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d613b10-9c9a-48c4-a506-2ede09d9db85 admin admin] PATH_INFO: `/v3/users` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:31:16.341637 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.fernet_utils [None req-917611f3-2cf9-4460-bb8d-37c0fea46b67 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80244) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 08:31:16.341717 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d613b10-9c9a-48c4-a506-2ede09d9db85 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:31:16.342255 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d613b10-9c9a-48c4-a506-2ede09d9db85 admin admin] RBAC: Authorization granted {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:31:16.358575 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-917611f3-2cf9-4460-bb8d-37c0fea46b67 None None] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:31:16.363126 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ad1495e-dfcf-4154-9ff9-ccf2c920dae5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '42d15fc0515945acacbbffcf2356c7a5', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:31:16.364030 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ad1495e-dfcf-4154-9ff9-ccf2c920dae5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:31:16.364315 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ad1495e-dfcf-4154-9ff9-ccf2c920dae5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:31:16.364594 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ad1495e-dfcf-4154-9ff9-ccf2c920dae5 admin admin] PATH_INFO: `/v3/projects/ca42afd27a3a43fea040e9bad44c8ba9` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:31:16.365785 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ad1495e-dfcf-4154-9ff9-ccf2c920dae5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ca42afd27a3a43fea040e9bad44c8ba9)` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:31:16.367484 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: [pid: 80245|app: 0|req: 988/1988] 10.222.0.47 () {60 vars in 1143 bytes} [Thu Jul 21 08:31:16 2022] POST /identity/v3/users => generated 298 bytes in 117 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 21 08:31:16.367740 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ad1495e-dfcf-4154-9ff9-ccf2c920dae5 admin admin] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:31:16.368870 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d613b10-9c9a-48c4-a506-2ede09d9db85 admin admin] Authenticating user token {{(pid=80245) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:31:16.379479 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d613b10-9c9a-48c4-a506-2ede09d9db85 admin admin] Validating token access rules against request {{(pid=80245) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:31:16.384274 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd885161-3e04-49a1-9674-8bad9f1ec249 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '42d15fc0515945acacbbffcf2356c7a5', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80245) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:31:16.384938 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd885161-3e04-49a1-9674-8bad9f1ec249 admin admin] REQUEST_METHOD: `POST` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:31:16.385277 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd885161-3e04-49a1-9674-8bad9f1ec249 admin admin] SCRIPT_NAME: `/identity` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:31:16.385721 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd885161-3e04-49a1-9674-8bad9f1ec249 admin admin] PATH_INFO: `/v3/projects` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:31:16.386494 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd885161-3e04-49a1-9674-8bad9f1ec249 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:31:16.389024 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd885161-3e04-49a1-9674-8bad9f1ec249 admin admin] RBAC: Authorization granted {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:31:16.411948 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG dbcounter [-] [80245] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80245) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:31:16.419303 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.notifications [None req-4ad1495e-dfcf-4154-9ff9-ccf2c920dae5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ca42afd27a3a43fea040e9bad44c8ba9', 'request_id': 'req-4ad1495e-dfcf-4154-9ff9-ccf2c920dae5'} {{(pid=80244) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 08:31:16.421455 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: [pid: 80245|app: 0|req: 989/1989] 10.222.0.47 () {60 vars in 1152 bytes} [Thu Jul 21 08:31:16 2022] POST /identity/v3/projects => generated 383 bytes in 53 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 08:31:16.422571 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd885161-3e04-49a1-9674-8bad9f1ec249 admin admin] Authenticating user token {{(pid=80245) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:31:16.428772 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.notifications [None req-4ad1495e-dfcf-4154-9ff9-ccf2c920dae5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ca42afd27a3a43fea040e9bad44c8ba9', 'request_id': 'req-4ad1495e-dfcf-4154-9ff9-ccf2c920dae5'} {{(pid=80244) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 08:31:16.435644 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.notifications [None req-4ad1495e-dfcf-4154-9ff9-ccf2c920dae5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ca42afd27a3a43fea040e9bad44c8ba9', 'request_id': 'req-4ad1495e-dfcf-4154-9ff9-ccf2c920dae5'} {{(pid=80244) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 08:31:16.446363 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.notifications [None req-4ad1495e-dfcf-4154-9ff9-ccf2c920dae5 admin admin] The token cache is being invalidate because project ca42afd27a3a43fea040e9bad44c8ba9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80244) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 21 08:31:16.446653 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.notifications [None req-4ad1495e-dfcf-4154-9ff9-ccf2c920dae5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80244) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 08:31:16.447317 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.notifications [None req-4ad1495e-dfcf-4154-9ff9-ccf2c920dae5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80244) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 08:31:16.448919 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 1001/1990] 10.222.0.47 () {58 vars in 1232 bytes} [Thu Jul 21 08:31:16 2022] DELETE /identity/v3/projects/ca42afd27a3a43fea040e9bad44c8ba9 => generated 0 bytes in 111 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 08:31:16.450186 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ad1495e-dfcf-4154-9ff9-ccf2c920dae5 admin admin] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:31:16.454351 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.fernet_utils [None req-4ad1495e-dfcf-4154-9ff9-ccf2c920dae5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80244) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 08:31:16.466058 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG dbcounter [-] [80244] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=80244) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:31:16.527171 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd885161-3e04-49a1-9674-8bad9f1ec249 admin admin] Validating token access rules against request {{(pid=80245) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:31:16.535068 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5c97d56-1052-4781-a952-7d1940d096f4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '42d15fc0515945acacbbffcf2356c7a5', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80245) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:31:16.548366 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5c97d56-1052-4781-a952-7d1940d096f4 admin admin] REQUEST_METHOD: `GET` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:31:16.549643 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ad1495e-dfcf-4154-9ff9-ccf2c920dae5 admin admin] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:31:16.549979 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5c97d56-1052-4781-a952-7d1940d096f4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:31:16.550530 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5c97d56-1052-4781-a952-7d1940d096f4 admin admin] PATH_INFO: `/v3/roles` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:31:16.551625 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5c97d56-1052-4781-a952-7d1940d096f4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:31:16.553479 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5c97d56-1052-4781-a952-7d1940d096f4 admin admin] RBAC: Authorization granted {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:31:16.556282 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73f445e0-890a-46d2-b99d-5bd74dff5d17 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '42d15fc0515945acacbbffcf2356c7a5', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:31:16.557189 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73f445e0-890a-46d2-b99d-5bd74dff5d17 admin admin] REQUEST_METHOD: `POST` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:31:16.557604 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73f445e0-890a-46d2-b99d-5bd74dff5d17 admin admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:31:16.558027 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73f445e0-890a-46d2-b99d-5bd74dff5d17 admin admin] PATH_INFO: `/v3/users` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:31:16.558805 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73f445e0-890a-46d2-b99d-5bd74dff5d17 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:31:16.561020 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73f445e0-890a-46d2-b99d-5bd74dff5d17 admin admin] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:31:16.569262 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: [pid: 80245|app: 0|req: 990/1991] 10.222.0.47 () {58 vars in 1121 bytes} [Thu Jul 21 08:31:16 2022] GET /identity/v3/roles => generated 1420 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 08:31:16.569262 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf3691ce-f40a-48ee-a816-f39d9d1adade None None] REQUEST_METHOD: `POST` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:31:16.569262 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf3691ce-f40a-48ee-a816-f39d9d1adade None None] SCRIPT_NAME: `/identity` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:31:16.569262 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf3691ce-f40a-48ee-a816-f39d9d1adade None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:31:16.590128 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 1002/1992] 10.222.0.47 () {60 vars in 1143 bytes} [Thu Jul 21 08:31:16 2022] POST /identity/v3/users => generated 298 bytes in 140 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 08:31:16.591685 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73f445e0-890a-46d2-b99d-5bd74dff5d17 admin admin] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:31:16.594587 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.fernet_utils [None req-73f445e0-890a-46d2-b99d-5bd74dff5d17 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80244) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 08:31:16.598154 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.auth.core [None req-cf3691ce-f40a-48ee-a816-f39d9d1adade None None] MFA Rules not processed for user `2dce40b8d74c41ec94cc36e6ff05d412`. Rule list: `[]` (Enabled: `True`). {{(pid=80245) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 08:31:16.599432 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.fernet_utils [None req-cf3691ce-f40a-48ee-a816-f39d9d1adade None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80245) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 08:31:16.610448 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73f445e0-890a-46d2-b99d-5bd74dff5d17 admin admin] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:31:16.611588 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: [pid: 80245|app: 0|req: 991/1993] 10.222.0.47 () {58 vars in 957 bytes} [Thu Jul 21 08:31:16 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 45 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 08:31:16.616912 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d4f1632-521e-4bde-a0d9-004ca86fe163 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '42d15fc0515945acacbbffcf2356c7a5', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:31:16.618430 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf3691ce-f40a-48ee-a816-f39d9d1adade None None] Authenticating user token {{(pid=80245) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:31:16.618739 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d4f1632-521e-4bde-a0d9-004ca86fe163 admin admin] REQUEST_METHOD: `PUT` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:31:16.619316 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d4f1632-521e-4bde-a0d9-004ca86fe163 admin admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:31:16.619934 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d4f1632-521e-4bde-a0d9-004ca86fe163 admin admin] PATH_INFO: `/v3/projects/89c38fb288a142cca8a4a79887f20425/users/cd807929c2a44571b264657cf1778ef1/roles/e29870f6e9e2411a9db0d5bf45f32ca5` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:31:16.633155 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf3691ce-f40a-48ee-a816-f39d9d1adade None None] Validating token access rules against request {{(pid=80245) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:31:16.648845 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92079459-48c2-4200-9e1d-50da64a6e8d3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '42d15fc0515945acacbbffcf2356c7a5', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80245) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:31:16.649607 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92079459-48c2-4200-9e1d-50da64a6e8d3 admin admin] REQUEST_METHOD: `GET` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:31:16.650042 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92079459-48c2-4200-9e1d-50da64a6e8d3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:31:16.650433 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92079459-48c2-4200-9e1d-50da64a6e8d3 admin admin] PATH_INFO: `/v3/roles` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:31:16.651184 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92079459-48c2-4200-9e1d-50da64a6e8d3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:31:16.654226 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92079459-48c2-4200-9e1d-50da64a6e8d3 admin admin] RBAC: Authorization granted {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:31:16.655956 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d4f1632-521e-4bde-a0d9-004ca86fe163 admin admin] RBAC: Authorizing `identity:create_grant(project_id=89c38fb288a142cca8a4a79887f20425, user_id=cd807929c2a44571b264657cf1778ef1, role_id=e29870f6e9e2411a9db0d5bf45f32ca5)` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:31:16.661041 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d4f1632-521e-4bde-a0d9-004ca86fe163 admin admin] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:31:16.682823 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: [pid: 80245|app: 0|req: 992/1994] 10.222.0.47 () {58 vars in 1121 bytes} [Thu Jul 21 08:31:16 2022] GET /identity/v3/roles => generated 1420 bytes in 70 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 08:31:16.685646 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44eac700-a4b1-4577-b510-03f5053ed6fd None None] REQUEST_METHOD: `POST` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:31:16.685960 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44eac700-a4b1-4577-b510-03f5053ed6fd None None] SCRIPT_NAME: `/identity` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:31:16.686632 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44eac700-a4b1-4577-b510-03f5053ed6fd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:31:16.691978 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 1003/1995] 10.222.0.47 () {60 vars in 1482 bytes} [Thu Jul 21 08:31:16 2022] PUT /identity/v3/projects/89c38fb288a142cca8a4a79887f20425/users/cd807929c2a44571b264657cf1778ef1/roles/e29870f6e9e2411a9db0d5bf45f32ca5 => generated 0 bytes in 101 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 08:31:16.692619 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d4f1632-521e-4bde-a0d9-004ca86fe163 admin admin] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:31:16.731662 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.auth.core [None req-44eac700-a4b1-4577-b510-03f5053ed6fd None None] MFA Rules not processed for user `2dce40b8d74c41ec94cc36e6ff05d412`. Rule list: `[]` (Enabled: `True`). {{(pid=80245) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 08:31:16.732897 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.fernet_utils [None req-44eac700-a4b1-4577-b510-03f5053ed6fd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80245) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 08:31:16.819443 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d4f1632-521e-4bde-a0d9-004ca86fe163 admin admin] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:31:16.827830 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7820dc90-1e08-4c6e-b83b-b8ec630c2058 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '42d15fc0515945acacbbffcf2356c7a5', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:31:16.829192 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7820dc90-1e08-4c6e-b83b-b8ec630c2058 admin admin] REQUEST_METHOD: `PUT` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:31:16.829928 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7820dc90-1e08-4c6e-b83b-b8ec630c2058 admin admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:31:16.830212 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7820dc90-1e08-4c6e-b83b-b8ec630c2058 admin admin] PATH_INFO: `/v3/projects/abad576e5d224349988a7fe6d25ff6df/users/a3f0f8badbc44664a2d0c572aac9f021/roles/e29870f6e9e2411a9db0d5bf45f32ca5` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:31:16.846575 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: [pid: 80245|app: 0|req: 993/1996] 10.222.0.47 () {58 vars in 957 bytes} [Thu Jul 21 08:31:16 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 08:31:16.847653 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44eac700-a4b1-4577-b510-03f5053ed6fd None None] Authenticating user token {{(pid=80245) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:31:16.859519 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7820dc90-1e08-4c6e-b83b-b8ec630c2058 admin admin] RBAC: Authorizing `identity:create_grant(project_id=abad576e5d224349988a7fe6d25ff6df, user_id=a3f0f8badbc44664a2d0c572aac9f021, role_id=e29870f6e9e2411a9db0d5bf45f32ca5)` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:31:16.860224 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44eac700-a4b1-4577-b510-03f5053ed6fd None None] Validating token access rules against request {{(pid=80245) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:31:16.862478 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7820dc90-1e08-4c6e-b83b-b8ec630c2058 admin admin] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:31:16.866058 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5d4da63-c22b-48ed-a78d-0902b920e572 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '42d15fc0515945acacbbffcf2356c7a5', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80245) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:31:16.866953 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5d4da63-c22b-48ed-a78d-0902b920e572 admin admin] REQUEST_METHOD: `GET` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:31:16.867561 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5d4da63-c22b-48ed-a78d-0902b920e572 admin admin] SCRIPT_NAME: `/identity` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:31:16.867672 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5d4da63-c22b-48ed-a78d-0902b920e572 admin admin] PATH_INFO: `/v3/roles` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:31:16.869089 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5d4da63-c22b-48ed-a78d-0902b920e572 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:31:16.871357 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5d4da63-c22b-48ed-a78d-0902b920e572 admin admin] RBAC: Authorization granted {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:31:16.885096 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: [pid: 80245|app: 0|req: 994/1997] 10.222.0.47 () {58 vars in 1121 bytes} [Thu Jul 21 08:31:16 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 08:31:16.889185 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5d4da63-c22b-48ed-a78d-0902b920e572 admin admin] Authenticating user token {{(pid=80245) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:31:16.894259 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 1004/1998] 10.222.0.47 () {60 vars in 1482 bytes} [Thu Jul 21 08:31:16 2022] PUT /identity/v3/projects/abad576e5d224349988a7fe6d25ff6df/users/a3f0f8badbc44664a2d0c572aac9f021/roles/e29870f6e9e2411a9db0d5bf45f32ca5 => generated 0 bytes in 198 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 08:31:16.895502 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7820dc90-1e08-4c6e-b83b-b8ec630c2058 admin admin] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:31:16.899829 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5d4da63-c22b-48ed-a78d-0902b920e572 admin admin] Validating token access rules against request {{(pid=80245) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:31:17.003403 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG dbcounter [-] [80244] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80244) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:31:17.003551 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7181cc87-fdb2-4bad-aa38-6c446601421c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '42d15fc0515945acacbbffcf2356c7a5', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80245) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:31:17.004936 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7181cc87-fdb2-4bad-aa38-6c446601421c admin admin] REQUEST_METHOD: `GET` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:31:17.005220 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7181cc87-fdb2-4bad-aa38-6c446601421c admin admin] SCRIPT_NAME: `/identity` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:31:17.005496 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7181cc87-fdb2-4bad-aa38-6c446601421c admin admin] PATH_INFO: `/v3/domains` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:31:17.006516 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7181cc87-fdb2-4bad-aa38-6c446601421c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80245) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 08:31:17.006831 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7181cc87-fdb2-4bad-aa38-6c446601421c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:31:17.009333 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7181cc87-fdb2-4bad-aa38-6c446601421c admin admin] RBAC: Authorization granted {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:31:17.019796 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG dbcounter [-] [80245] Writing DB stats keystone:SELECT=100 {{(pid=80245) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:31:17.020973 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7820dc90-1e08-4c6e-b83b-b8ec630c2058 admin admin] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:31:17.027805 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: [pid: 80245|app: 0|req: 995/1999] 10.222.0.47 () {58 vars in 1152 bytes} [Thu Jul 21 08:31:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 08:31:17.029211 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7181cc87-fdb2-4bad-aa38-6c446601421c admin admin] Authenticating user token {{(pid=80245) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:31:17.032570 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-379e56ca-c3c6-4ecc-9fbf-31f96d700a61 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '42d15fc0515945acacbbffcf2356c7a5', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:31:17.033788 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-379e56ca-c3c6-4ecc-9fbf-31f96d700a61 admin admin] REQUEST_METHOD: `PUT` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:31:17.033909 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-379e56ca-c3c6-4ecc-9fbf-31f96d700a61 admin admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:31:17.034227 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-379e56ca-c3c6-4ecc-9fbf-31f96d700a61 admin admin] PATH_INFO: `/v3/projects/89c38fb288a142cca8a4a79887f20425/users/cd807929c2a44571b264657cf1778ef1/roles/e29870f6e9e2411a9db0d5bf45f32ca5` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:31:17.036662 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-379e56ca-c3c6-4ecc-9fbf-31f96d700a61 admin admin] RBAC: Authorizing `identity:create_grant(project_id=89c38fb288a142cca8a4a79887f20425, user_id=cd807929c2a44571b264657cf1778ef1, role_id=e29870f6e9e2411a9db0d5bf45f32ca5)` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:31:17.039610 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-379e56ca-c3c6-4ecc-9fbf-31f96d700a61 admin admin] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:31:17.045863 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7181cc87-fdb2-4bad-aa38-6c446601421c admin admin] Validating token access rules against request {{(pid=80245) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:31:17.051622 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 1005/2000] 10.222.0.47 () {60 vars in 1482 bytes} [Thu Jul 21 08:31:16 2022] PUT /identity/v3/projects/89c38fb288a142cca8a4a79887f20425/users/cd807929c2a44571b264657cf1778ef1/roles/e29870f6e9e2411a9db0d5bf45f32ca5 => generated 0 bytes in 157 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 08:31:17.061849 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-379e56ca-c3c6-4ecc-9fbf-31f96d700a61 admin admin] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:31:17.117373 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ee64758-5c40-445d-ae29-3876b7b3d784 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '42d15fc0515945acacbbffcf2356c7a5', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80245) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:31:17.118208 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ee64758-5c40-445d-ae29-3876b7b3d784 admin admin] REQUEST_METHOD: `GET` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:31:17.118510 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ee64758-5c40-445d-ae29-3876b7b3d784 admin admin] SCRIPT_NAME: `/identity` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:31:17.118790 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ee64758-5c40-445d-ae29-3876b7b3d784 admin admin] PATH_INFO: `/v3/roles` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:31:17.119573 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ee64758-5c40-445d-ae29-3876b7b3d784 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:31:17.121179 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ee64758-5c40-445d-ae29-3876b7b3d784 admin admin] RBAC: Authorization granted {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:31:17.136060 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: [pid: 80245|app: 0|req: 996/2001] 10.222.0.47 () {58 vars in 1121 bytes} [Thu Jul 21 08:31:17 2022] GET /identity/v3/roles => generated 1420 bytes in 105 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 08:31:17.136060 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e614632f-5565-46e1-b836-b3d9d2dc9d40 None None] REQUEST_METHOD: `POST` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:31:17.136060 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e614632f-5565-46e1-b836-b3d9d2dc9d40 None None] SCRIPT_NAME: `/identity` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:31:17.136060 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e614632f-5565-46e1-b836-b3d9d2dc9d40 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:31:17.147249 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-379e56ca-c3c6-4ecc-9fbf-31f96d700a61 admin admin] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:31:17.153211 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0b2c572-c3ec-445f-9073-17d6b40f3018 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '42d15fc0515945acacbbffcf2356c7a5', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:31:17.153900 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0b2c572-c3ec-445f-9073-17d6b40f3018 admin admin] REQUEST_METHOD: `POST` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:31:17.154189 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0b2c572-c3ec-445f-9073-17d6b40f3018 admin admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:31:17.154473 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0b2c572-c3ec-445f-9073-17d6b40f3018 admin admin] PATH_INFO: `/v3/projects` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:31:17.155176 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0b2c572-c3ec-445f-9073-17d6b40f3018 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:31:17.157588 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0b2c572-c3ec-445f-9073-17d6b40f3018 admin admin] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:31:17.165546 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.auth.core [None req-e614632f-5565-46e1-b836-b3d9d2dc9d40 None None] MFA Rules not processed for user `cd807929c2a44571b264657cf1778ef1`. Rule list: `[]` (Enabled: `True`). {{(pid=80245) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 08:31:17.166860 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.fernet_utils [None req-e614632f-5565-46e1-b836-b3d9d2dc9d40 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80245) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 08:31:17.189091 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 1006/2002] 10.222.0.47 () {60 vars in 1152 bytes} [Thu Jul 21 08:31:17 2022] POST /identity/v3/projects => generated 363 bytes in 137 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 08:31:17.190404 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0b2c572-c3ec-445f-9073-17d6b40f3018 admin admin] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:31:17.248957 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: [pid: 80245|app: 0|req: 997/2003] 10.222.0.47 () {58 vars in 957 bytes} [Thu Jul 21 08:31:17 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 08:31:17.249900 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e614632f-5565-46e1-b836-b3d9d2dc9d40 None None] Authenticating user token {{(pid=80245) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:31:17.261844 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e614632f-5565-46e1-b836-b3d9d2dc9d40 None None] Validating token access rules against request {{(pid=80245) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:31:17.266668 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3775629e-13f2-4cf9-866e-301c54327b4e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '42d15fc0515945acacbbffcf2356c7a5', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80245) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:31:17.266922 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0b2c572-c3ec-445f-9073-17d6b40f3018 admin admin] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:31:17.267364 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3775629e-13f2-4cf9-866e-301c54327b4e admin admin] REQUEST_METHOD: `POST` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:31:17.267659 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3775629e-13f2-4cf9-866e-301c54327b4e admin admin] SCRIPT_NAME: `/identity` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:31:17.267942 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3775629e-13f2-4cf9-866e-301c54327b4e admin admin] PATH_INFO: `/v3/users` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:31:17.268640 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3775629e-13f2-4cf9-866e-301c54327b4e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:31:17.270597 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3775629e-13f2-4cf9-866e-301c54327b4e admin admin] RBAC: Authorization granted {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:31:17.271655 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe973ff3-8c7a-47ed-a1c4-9075632f8227 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '42d15fc0515945acacbbffcf2356c7a5', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:31:17.272554 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe973ff3-8c7a-47ed-a1c4-9075632f8227 admin admin] REQUEST_METHOD: `PUT` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:31:17.272834 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe973ff3-8c7a-47ed-a1c4-9075632f8227 admin admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:31:17.273107 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe973ff3-8c7a-47ed-a1c4-9075632f8227 admin admin] PATH_INFO: `/v3/projects/abad576e5d224349988a7fe6d25ff6df/users/a3f0f8badbc44664a2d0c572aac9f021/roles/e29870f6e9e2411a9db0d5bf45f32ca5` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:31:17.275072 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe973ff3-8c7a-47ed-a1c4-9075632f8227 admin admin] RBAC: Authorizing `identity:create_grant(project_id=abad576e5d224349988a7fe6d25ff6df, user_id=a3f0f8badbc44664a2d0c572aac9f021, role_id=e29870f6e9e2411a9db0d5bf45f32ca5)` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:31:17.277770 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe973ff3-8c7a-47ed-a1c4-9075632f8227 admin admin] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:31:17.289044 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 1007/2004] 10.222.0.47 () {60 vars in 1482 bytes} [Thu Jul 21 08:31:17 2022] PUT /identity/v3/projects/abad576e5d224349988a7fe6d25ff6df/users/a3f0f8badbc44664a2d0c572aac9f021/roles/e29870f6e9e2411a9db0d5bf45f32ca5 => generated 0 bytes in 99 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 08:31:17.299700 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe973ff3-8c7a-47ed-a1c4-9075632f8227 admin admin] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:31:17.299700 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.fernet_utils [None req-fe973ff3-8c7a-47ed-a1c4-9075632f8227 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80244) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 08:31:17.299892 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: [pid: 80245|app: 0|req: 998/2005] 10.222.0.47 () {60 vars in 1143 bytes} [Thu Jul 21 08:31:17 2022] POST /identity/v3/users => generated 282 bytes in 42 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 08:31:17.299892 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3775629e-13f2-4cf9-866e-301c54327b4e admin admin] Authenticating user token {{(pid=80245) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:31:17.306842 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3775629e-13f2-4cf9-866e-301c54327b4e admin admin] Validating token access rules against request {{(pid=80245) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:31:17.354913 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe973ff3-8c7a-47ed-a1c4-9075632f8227 admin admin] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:31:17.358631 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f99a4ab3-2bcb-4743-ad96-60f118ad9ac5 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4f40410b7674687ab5362ac982248ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4faef13aa50e4e36b6c265fb1398fa14', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:31:17.359348 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f99a4ab3-2bcb-4743-ad96-60f118ad9ac5 service cinder] REQUEST_METHOD: `GET` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:31:17.359634 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f99a4ab3-2bcb-4743-ad96-60f118ad9ac5 service cinder] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:31:17.359916 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f99a4ab3-2bcb-4743-ad96-60f118ad9ac5 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:31:17.365184 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66e398a1-a0d9-43f2-b7b2-2f7bffdd4bb5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '42d15fc0515945acacbbffcf2356c7a5', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80245) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:31:17.365855 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66e398a1-a0d9-43f2-b7b2-2f7bffdd4bb5 admin admin] REQUEST_METHOD: `GET` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:31:17.366141 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66e398a1-a0d9-43f2-b7b2-2f7bffdd4bb5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:31:17.366445 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66e398a1-a0d9-43f2-b7b2-2f7bffdd4bb5 admin admin] PATH_INFO: `/v3/roles` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:31:17.367222 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66e398a1-a0d9-43f2-b7b2-2f7bffdd4bb5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:31:17.368850 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66e398a1-a0d9-43f2-b7b2-2f7bffdd4bb5 admin admin] RBAC: Authorization granted {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:31:17.374141 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG dbcounter [-] [80245] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80245) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:31:17.379629 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: [pid: 80245|app: 0|req: 999/2006] 10.222.0.47 () {58 vars in 1121 bytes} [Thu Jul 21 08:31:17 2022] GET /identity/v3/roles => generated 1420 bytes in 83 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 21 08:31:17.380353 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG dbcounter [-] [80244] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80244) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:31:17.382081 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74c9431b-b3d5-448e-9dc4-d41d3d57fb1a None None] REQUEST_METHOD: `POST` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:31:17.382610 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74c9431b-b3d5-448e-9dc4-d41d3d57fb1a None None] SCRIPT_NAME: `/identity` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:31:17.383035 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74c9431b-b3d5-448e-9dc4-d41d3d57fb1a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:31:17.411170 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.auth.core [None req-74c9431b-b3d5-448e-9dc4-d41d3d57fb1a None None] MFA Rules not processed for user `a3f0f8badbc44664a2d0c572aac9f021`. Rule list: `[]` (Enabled: `True`). {{(pid=80245) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 08:31:17.412453 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.fernet_utils [None req-74c9431b-b3d5-448e-9dc4-d41d3d57fb1a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80245) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 08:31:17.430421 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f99a4ab3-2bcb-4743-ad96-60f118ad9ac5 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:31:17.432399 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f99a4ab3-2bcb-4743-ad96-60f118ad9ac5 service cinder] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:31:17.443353 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 1008/2007] 10.222.0.47 () {60 vars in 1365 bytes} [Thu Jul 21 08:31:17 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 154 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 08:31:17.444195 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f99a4ab3-2bcb-4743-ad96-60f118ad9ac5 service cinder] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:31:17.455529 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f99a4ab3-2bcb-4743-ad96-60f118ad9ac5 service cinder] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:31:17.460425 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5effca7-8214-4578-b9ca-283f92c3acfd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '42d15fc0515945acacbbffcf2356c7a5', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:31:17.461106 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5effca7-8214-4578-b9ca-283f92c3acfd admin admin] REQUEST_METHOD: `GET` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:31:17.461386 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5effca7-8214-4578-b9ca-283f92c3acfd admin admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:31:17.461658 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5effca7-8214-4578-b9ca-283f92c3acfd admin admin] PATH_INFO: `/v3/roles` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:31:17.463967 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5effca7-8214-4578-b9ca-283f92c3acfd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:31:17.465613 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5effca7-8214-4578-b9ca-283f92c3acfd admin admin] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:31:17.494445 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 1009/2008] 10.222.0.47 () {58 vars in 1121 bytes} [Thu Jul 21 08:31:17 2022] GET /identity/v3/roles => generated 1420 bytes in 49 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 08:31:17.496520 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5effca7-8214-4578-b9ca-283f92c3acfd admin admin] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:31:17.517147 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5effca7-8214-4578-b9ca-283f92c3acfd admin admin] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:31:17.536168 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: [pid: 80245|app: 0|req: 1000/2009] 10.222.0.47 () {58 vars in 957 bytes} [Thu Jul 21 08:31:17 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 153 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 08:31:17.539925 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fb9a056-ff20-44e0-9a57-9341dddec897 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '42d15fc0515945acacbbffcf2356c7a5', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:31:17.541124 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fb9a056-ff20-44e0-9a57-9341dddec897 admin admin] REQUEST_METHOD: `PUT` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:31:17.542172 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fb9a056-ff20-44e0-9a57-9341dddec897 admin admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:31:17.542487 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fb9a056-ff20-44e0-9a57-9341dddec897 admin admin] PATH_INFO: `/v3/projects/860580f1a6a24128afc5814f6b16b320/users/88b257780a2944ec9255c3ea88b6df70/roles/3a074738221e4601b5205715ba87d240` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:31:17.552671 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74c9431b-b3d5-448e-9dc4-d41d3d57fb1a None None] Authenticating user token {{(pid=80245) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:31:17.565762 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fb9a056-ff20-44e0-9a57-9341dddec897 admin admin] RBAC: Authorizing `identity:create_grant(project_id=860580f1a6a24128afc5814f6b16b320, user_id=88b257780a2944ec9255c3ea88b6df70, role_id=3a074738221e4601b5205715ba87d240)` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:31:17.567952 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74c9431b-b3d5-448e-9dc4-d41d3d57fb1a None None] Validating token access rules against request {{(pid=80245) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:31:17.569712 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fb9a056-ff20-44e0-9a57-9341dddec897 admin admin] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:31:17.573338 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e5cc6e1-df8e-4330-ab28-3962e39f2af0 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4f40410b7674687ab5362ac982248ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4faef13aa50e4e36b6c265fb1398fa14', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80245) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:31:17.573627 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e5cc6e1-df8e-4330-ab28-3962e39f2af0 service cinder] REQUEST_METHOD: `GET` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:31:17.579198 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e5cc6e1-df8e-4330-ab28-3962e39f2af0 service cinder] SCRIPT_NAME: `/identity` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:31:17.579198 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e5cc6e1-df8e-4330-ab28-3962e39f2af0 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:31:17.585482 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e5cc6e1-df8e-4330-ab28-3962e39f2af0 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:31:17.587027 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e5cc6e1-df8e-4330-ab28-3962e39f2af0 service cinder] RBAC: Authorization granted {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:31:17.591443 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 1010/2010] 10.222.0.47 () {60 vars in 1482 bytes} [Thu Jul 21 08:31:17 2022] PUT /identity/v3/projects/860580f1a6a24128afc5814f6b16b320/users/88b257780a2944ec9255c3ea88b6df70/roles/3a074738221e4601b5205715ba87d240 => generated 0 bytes in 96 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 08:31:17.592687 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fb9a056-ff20-44e0-9a57-9341dddec897 admin admin] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:31:17.597320 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.fernet_utils [None req-4fb9a056-ff20-44e0-9a57-9341dddec897 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80244) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 08:31:17.599641 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: [pid: 80245|app: 0|req: 1001/2011] 10.222.0.47 () {60 vars in 1365 bytes} [Thu Jul 21 08:31:17 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 47 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 21 08:31:17.604165 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c129329-3d1d-46ed-b8dd-a0804c33b0c3 None None] REQUEST_METHOD: `POST` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:31:17.604463 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c129329-3d1d-46ed-b8dd-a0804c33b0c3 None None] SCRIPT_NAME: `/identity` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:31:17.604748 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c129329-3d1d-46ed-b8dd-a0804c33b0c3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:31:17.640789 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.auth.core [None req-7c129329-3d1d-46ed-b8dd-a0804c33b0c3 None None] MFA Rules not processed for user `88b257780a2944ec9255c3ea88b6df70`. Rule list: `[]` (Enabled: `True`). {{(pid=80245) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 08:31:17.642017 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.fernet_utils [None req-7c129329-3d1d-46ed-b8dd-a0804c33b0c3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80245) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 08:31:17.674929 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fb9a056-ff20-44e0-9a57-9341dddec897 admin admin] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:31:17.684481 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fbf09a5-179c-47d8-a74b-dc0438ff1fa4 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4f40410b7674687ab5362ac982248ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4faef13aa50e4e36b6c265fb1398fa14', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:31:17.685981 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fbf09a5-179c-47d8-a74b-dc0438ff1fa4 service cinder] REQUEST_METHOD: `GET` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:31:17.686384 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fbf09a5-179c-47d8-a74b-dc0438ff1fa4 service cinder] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:31:17.686811 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fbf09a5-179c-47d8-a74b-dc0438ff1fa4 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:31:17.689138 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.fernet_utils [None req-7fbf09a5-179c-47d8-a74b-dc0438ff1fa4 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80244) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 08:31:17.723862 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: [pid: 80245|app: 0|req: 1002/2012] 10.222.0.47 () {58 vars in 957 bytes} [Thu Jul 21 08:31:17 2022] POST /identity/v3/auth/tokens => generated 3230 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 21 08:31:17.734174 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c129329-3d1d-46ed-b8dd-a0804c33b0c3 None None] Authenticating user token {{(pid=80245) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:31:17.768438 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fbf09a5-179c-47d8-a74b-dc0438ff1fa4 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:31:17.770622 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fbf09a5-179c-47d8-a74b-dc0438ff1fa4 service cinder] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:31:17.788233 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 1011/2013] 10.222.0.47 () {60 vars in 1365 bytes} [Thu Jul 21 08:31:17 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 196 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 08:31:17.828546 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fbf09a5-179c-47d8-a74b-dc0438ff1fa4 service cinder] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:31:17.838846 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fbf09a5-179c-47d8-a74b-dc0438ff1fa4 service cinder] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:31:17.845526 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9696a753-8cac-4e7e-8cdf-d6b11ae189f0 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4f40410b7674687ab5362ac982248ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4faef13aa50e4e36b6c265fb1398fa14', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:31:17.846331 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9696a753-8cac-4e7e-8cdf-d6b11ae189f0 service cinder] REQUEST_METHOD: `GET` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:31:17.846619 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9696a753-8cac-4e7e-8cdf-d6b11ae189f0 service cinder] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:31:17.846900 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9696a753-8cac-4e7e-8cdf-d6b11ae189f0 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:31:17.848942 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c129329-3d1d-46ed-b8dd-a0804c33b0c3 None None] Validating token access rules against request {{(pid=80245) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:31:17.849229 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.fernet_utils [None req-9696a753-8cac-4e7e-8cdf-d6b11ae189f0 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80244) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 08:31:17.855047 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b85cf0bb-ee1b-42ee-994c-88ab47b3e455 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '42d15fc0515945acacbbffcf2356c7a5', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80245) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:31:17.855254 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b85cf0bb-ee1b-42ee-994c-88ab47b3e455 admin admin] REQUEST_METHOD: `POST` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:31:17.855509 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b85cf0bb-ee1b-42ee-994c-88ab47b3e455 admin admin] SCRIPT_NAME: `/identity` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:31:17.856437 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b85cf0bb-ee1b-42ee-994c-88ab47b3e455 admin admin] PATH_INFO: `/v3/projects` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:31:17.857489 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b85cf0bb-ee1b-42ee-994c-88ab47b3e455 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:31:17.862467 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b85cf0bb-ee1b-42ee-994c-88ab47b3e455 admin admin] RBAC: Authorization granted {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:31:17.867979 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG dbcounter [-] [80245] Writing DB stats keystone:SELECT=100 {{(pid=80245) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:31:17.907150 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG dbcounter [-] [80244] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80244) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:31:17.919387 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: [pid: 80245|app: 0|req: 1003/2014] 10.222.0.47 () {60 vars in 1152 bytes} [Thu Jul 21 08:31:17 2022] POST /identity/v3/projects => generated 365 bytes in 189 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 21 08:31:17.921827 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b85cf0bb-ee1b-42ee-994c-88ab47b3e455 admin admin] Authenticating user token {{(pid=80245) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:31:17.957556 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9696a753-8cac-4e7e-8cdf-d6b11ae189f0 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:31:17.959032 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9696a753-8cac-4e7e-8cdf-d6b11ae189f0 service cinder] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:31:17.975404 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 1012/2015] 10.222.0.47 () {60 vars in 1365 bytes} [Thu Jul 21 08:31:17 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 147 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 21 08:31:18.014210 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b85cf0bb-ee1b-42ee-994c-88ab47b3e455 admin admin] Validating token access rules against request {{(pid=80245) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:31:18.025522 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea423e8a-85aa-4bcf-ba15-a123f65811e7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '42d15fc0515945acacbbffcf2356c7a5', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80245) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:31:18.026278 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea423e8a-85aa-4bcf-ba15-a123f65811e7 admin admin] REQUEST_METHOD: `POST` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:31:18.026623 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea423e8a-85aa-4bcf-ba15-a123f65811e7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:31:18.026924 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea423e8a-85aa-4bcf-ba15-a123f65811e7 admin admin] PATH_INFO: `/v3/users` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:31:18.027678 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea423e8a-85aa-4bcf-ba15-a123f65811e7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:31:18.030391 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea423e8a-85aa-4bcf-ba15-a123f65811e7 admin admin] RBAC: Authorization granted {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:31:18.060810 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: [pid: 80245|app: 0|req: 1004/2016] 10.222.0.47 () {60 vars in 1143 bytes} [Thu Jul 21 08:31:17 2022] POST /identity/v3/users => generated 289 bytes in 140 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 08:31:18.065144 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9696a753-8cac-4e7e-8cdf-d6b11ae189f0 service cinder] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:31:18.088333 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9696a753-8cac-4e7e-8cdf-d6b11ae189f0 service cinder] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:31:18.096136 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b49f473-4d90-44c9-b17a-48ed6df70c97 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '42d15fc0515945acacbbffcf2356c7a5', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:31:18.097632 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b49f473-4d90-44c9-b17a-48ed6df70c97 admin admin] REQUEST_METHOD: `GET` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:31:18.098207 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b49f473-4d90-44c9-b17a-48ed6df70c97 admin admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:31:18.098601 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b49f473-4d90-44c9-b17a-48ed6df70c97 admin admin] PATH_INFO: `/v3/roles` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:31:18.099537 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b49f473-4d90-44c9-b17a-48ed6df70c97 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:31:18.101776 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b49f473-4d90-44c9-b17a-48ed6df70c97 admin admin] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:31:18.121208 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 1013/2017] 10.222.0.47 () {58 vars in 1121 bytes} [Thu Jul 21 08:31:18 2022] GET /identity/v3/roles => generated 1420 bytes in 57 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 08:31:18.126288 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea423e8a-85aa-4bcf-ba15-a123f65811e7 admin admin] Authenticating user token {{(pid=80245) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:31:18.141387 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea423e8a-85aa-4bcf-ba15-a123f65811e7 admin admin] Validating token access rules against request {{(pid=80245) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:31:18.148334 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ed52b97-a3ae-4cab-9187-c21a90d2c312 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '42d15fc0515945acacbbffcf2356c7a5', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80245) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:31:18.149912 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ed52b97-a3ae-4cab-9187-c21a90d2c312 admin admin] REQUEST_METHOD: `PUT` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:31:18.150273 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ed52b97-a3ae-4cab-9187-c21a90d2c312 admin admin] SCRIPT_NAME: `/identity` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:31:18.150648 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ed52b97-a3ae-4cab-9187-c21a90d2c312 admin admin] PATH_INFO: `/v3/projects/9c69d9da4c59496f9cbae25f678a8dec/users/12e5e7b5dbdd427d8c435f8b6ff8599f/roles/e29870f6e9e2411a9db0d5bf45f32ca5` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:31:18.177724 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ed52b97-a3ae-4cab-9187-c21a90d2c312 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9c69d9da4c59496f9cbae25f678a8dec, user_id=12e5e7b5dbdd427d8c435f8b6ff8599f, role_id=e29870f6e9e2411a9db0d5bf45f32ca5)` {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:31:18.187010 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ed52b97-a3ae-4cab-9187-c21a90d2c312 admin admin] RBAC: Authorization granted {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:31:18.195310 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b49f473-4d90-44c9-b17a-48ed6df70c97 admin admin] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:31:18.213066 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b49f473-4d90-44c9-b17a-48ed6df70c97 admin admin] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:31:18.218423 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: [pid: 80245|app: 0|req: 1005/2018] 10.222.0.47 () {60 vars in 1482 bytes} [Thu Jul 21 08:31:18 2022] PUT /identity/v3/projects/9c69d9da4c59496f9cbae25f678a8dec/users/12e5e7b5dbdd427d8c435f8b6ff8599f/roles/e29870f6e9e2411a9db0d5bf45f32ca5 => generated 0 bytes in 93 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 08:31:18.224804 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ed52b97-a3ae-4cab-9187-c21a90d2c312 admin admin] Authenticating user token {{(pid=80245) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:31:18.332192 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34a3b29e-60d6-421a-8f53-6f328cbd5827 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '42d15fc0515945acacbbffcf2356c7a5', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:31:18.334090 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34a3b29e-60d6-421a-8f53-6f328cbd5827 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:31:18.334090 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34a3b29e-60d6-421a-8f53-6f328cbd5827 admin admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:31:18.334090 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34a3b29e-60d6-421a-8f53-6f328cbd5827 admin admin] PATH_INFO: `/v3/users/afe5dd01ff9c46b18a6835cc582faa32` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:31:18.336159 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34a3b29e-60d6-421a-8f53-6f328cbd5827 admin admin] RBAC: Authorizing `identity:delete_user(user_id=afe5dd01ff9c46b18a6835cc582faa32)` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:31:18.339798 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34a3b29e-60d6-421a-8f53-6f328cbd5827 admin admin] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:31:18.373020 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ed52b97-a3ae-4cab-9187-c21a90d2c312 admin admin] Validating token access rules against request {{(pid=80245) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:31:18.383178 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c2be120-2aca-4889-9090-9e5800a92b01 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '42d15fc0515945acacbbffcf2356c7a5', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80245) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:31:18.384205 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c2be120-2aca-4889-9090-9e5800a92b01 admin admin] REQUEST_METHOD: `GET` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:31:18.384345 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c2be120-2aca-4889-9090-9e5800a92b01 admin admin] SCRIPT_NAME: `/identity` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:31:18.384672 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c2be120-2aca-4889-9090-9e5800a92b01 admin admin] PATH_INFO: `/v3/roles` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:31:18.385490 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c2be120-2aca-4889-9090-9e5800a92b01 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:31:18.389160 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c2be120-2aca-4889-9090-9e5800a92b01 admin admin] RBAC: Authorization granted {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:31:18.403721 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: [pid: 80245|app: 0|req: 1006/2019] 10.222.0.47 () {58 vars in 1121 bytes} [Thu Jul 21 08:31:18 2022] GET /identity/v3/roles => generated 1420 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 08:31:18.408267 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c2be120-2aca-4889-9090-9e5800a92b01 admin admin] Authenticating user token {{(pid=80245) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:31:18.424045 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.notifications [None req-34a3b29e-60d6-421a-8f53-6f328cbd5827 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'afe5dd01ff9c46b18a6835cc582faa32'} {{(pid=80244) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 08:31:18.426687 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c2be120-2aca-4889-9090-9e5800a92b01 admin admin] Validating token access rules against request {{(pid=80245) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:31:18.436773 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.notifications [None req-34a3b29e-60d6-421a-8f53-6f328cbd5827 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'afe5dd01ff9c46b18a6835cc582faa32'} {{(pid=80244) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 08:31:18.437949 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ca0b64a-465a-484d-ac9e-cf69cf642164 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '42d15fc0515945acacbbffcf2356c7a5', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80245) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:31:18.439688 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ca0b64a-465a-484d-ac9e-cf69cf642164 admin admin] REQUEST_METHOD: `PUT` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:31:18.439688 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ca0b64a-465a-484d-ac9e-cf69cf642164 admin admin] SCRIPT_NAME: `/identity` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:31:18.440494 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ca0b64a-465a-484d-ac9e-cf69cf642164 admin admin] PATH_INFO: `/v3/projects/9c69d9da4c59496f9cbae25f678a8dec/users/12e5e7b5dbdd427d8c435f8b6ff8599f/roles/e29870f6e9e2411a9db0d5bf45f32ca5` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:31:18.443034 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ca0b64a-465a-484d-ac9e-cf69cf642164 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9c69d9da4c59496f9cbae25f678a8dec, user_id=12e5e7b5dbdd427d8c435f8b6ff8599f, role_id=e29870f6e9e2411a9db0d5bf45f32ca5)` {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:31:18.446307 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ca0b64a-465a-484d-ac9e-cf69cf642164 admin admin] RBAC: Authorization granted {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:31:18.479369 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: [pid: 80245|app: 0|req: 1007/2020] 10.222.0.47 () {60 vars in 1482 bytes} [Thu Jul 21 08:31:18 2022] PUT /identity/v3/projects/9c69d9da4c59496f9cbae25f678a8dec/users/12e5e7b5dbdd427d8c435f8b6ff8599f/roles/e29870f6e9e2411a9db0d5bf45f32ca5 => generated 0 bytes in 69 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 21 08:31:18.480046 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.notifications [None req-34a3b29e-60d6-421a-8f53-6f328cbd5827 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'afe5dd01ff9c46b18a6835cc582faa32'} {{(pid=80244) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 08:31:18.481298 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.notifications [None req-34a3b29e-60d6-421a-8f53-6f328cbd5827 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'afe5dd01ff9c46b18a6835cc582faa32'} {{(pid=80244) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 08:31:18.492528 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35a6f310-12cd-4a19-9626-779ce2ba4a25 None None] REQUEST_METHOD: `POST` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:31:18.492911 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35a6f310-12cd-4a19-9626-779ce2ba4a25 None None] SCRIPT_NAME: `/identity` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:31:18.493655 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35a6f310-12cd-4a19-9626-779ce2ba4a25 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:31:18.498155 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.notifications [None req-34a3b29e-60d6-421a-8f53-6f328cbd5827 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'afe5dd01ff9c46b18a6835cc582faa32'} {{(pid=80244) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 08:31:18.504885 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 1014/2021] 10.222.0.47 () {58 vars in 1223 bytes} [Thu Jul 21 08:31:18 2022] DELETE /identity/v3/users/afe5dd01ff9c46b18a6835cc582faa32 => generated 0 bytes in 310 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 08:31:18.509181 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6ed5cde-f179-4b95-bf23-5a83d8145d35 None None] REQUEST_METHOD: `POST` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:31:18.509495 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6ed5cde-f179-4b95-bf23-5a83d8145d35 None None] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:31:18.509758 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6ed5cde-f179-4b95-bf23-5a83d8145d35 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:31:18.524144 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.auth.core [None req-35a6f310-12cd-4a19-9626-779ce2ba4a25 None None] MFA Rules not processed for user `12e5e7b5dbdd427d8c435f8b6ff8599f`. Rule list: `[]` (Enabled: `True`). {{(pid=80245) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 08:31:18.525416 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.fernet_utils [None req-35a6f310-12cd-4a19-9626-779ce2ba4a25 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80245) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 08:31:18.530483 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.auth.core [None req-c6ed5cde-f179-4b95-bf23-5a83d8145d35 None None] MFA Rules not processed for user `2dce40b8d74c41ec94cc36e6ff05d412`. Rule list: `[]` (Enabled: `True`). {{(pid=80244) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 08:31:18.531722 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.fernet_utils [None req-c6ed5cde-f179-4b95-bf23-5a83d8145d35 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80244) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 08:31:18.551630 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG dbcounter [-] [80245] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=80245) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:31:18.574222 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG dbcounter [-] [80244] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80244) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:31:18.597449 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 1015/2022] 10.222.0.47 () {58 vars in 957 bytes} [Thu Jul 21 08:31:18 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 21 08:31:18.603042 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: [pid: 80245|app: 0|req: 1008/2023] 10.222.0.47 () {58 vars in 957 bytes} [Thu Jul 21 08:31:18 2022] POST /identity/v3/auth/tokens => generated 3299 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 21 08:31:18.608253 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6ed5cde-f179-4b95-bf23-5a83d8145d35 None None] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:31:18.610013 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.fernet_utils [None req-c6ed5cde-f179-4b95-bf23-5a83d8145d35 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80244) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 08:31:18.614699 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35a6f310-12cd-4a19-9626-779ce2ba4a25 None None] Authenticating user token {{(pid=80245) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:31:18.616034 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.fernet_utils [None req-35a6f310-12cd-4a19-9626-779ce2ba4a25 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80245) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 08:31:18.670345 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6ed5cde-f179-4b95-bf23-5a83d8145d35 None None] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:31:18.674201 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38b6ea28-1c53-4eee-aabe-2f248993af44 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20ca3c3bd3774691b3374cfbfec6e494', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4faef13aa50e4e36b6c265fb1398fa14', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:31:18.674587 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35a6f310-12cd-4a19-9626-779ce2ba4a25 None None] Validating token access rules against request {{(pid=80245) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:31:18.674857 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38b6ea28-1c53-4eee-aabe-2f248993af44 service neutron] REQUEST_METHOD: `GET` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:31:18.675206 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38b6ea28-1c53-4eee-aabe-2f248993af44 service neutron] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:31:18.675458 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38b6ea28-1c53-4eee-aabe-2f248993af44 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:31:18.678275 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5414cdc-c869-4b53-99a7-4a8be150bd09 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4f40410b7674687ab5362ac982248ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4faef13aa50e4e36b6c265fb1398fa14', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80245) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:31:18.678928 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5414cdc-c869-4b53-99a7-4a8be150bd09 service cinder] REQUEST_METHOD: `GET` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:31:18.679223 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5414cdc-c869-4b53-99a7-4a8be150bd09 service cinder] SCRIPT_NAME: `/identity` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:31:18.679533 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5414cdc-c869-4b53-99a7-4a8be150bd09 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:31:18.685673 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38b6ea28-1c53-4eee-aabe-2f248993af44 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:31:18.687594 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38b6ea28-1c53-4eee-aabe-2f248993af44 service neutron] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:31:18.689455 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5414cdc-c869-4b53-99a7-4a8be150bd09 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:31:18.690885 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5414cdc-c869-4b53-99a7-4a8be150bd09 service cinder] RBAC: Authorization granted {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:31:18.697985 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 1016/2024] 10.222.0.47 () {60 vars in 1365 bytes} [Thu Jul 21 08:31:18 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 21 08:31:18.702664 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: [pid: 80245|app: 0|req: 1009/2025] 10.222.0.47 () {60 vars in 1365 bytes} [Thu Jul 21 08:31:18 2022] GET /identity/v3/auth/tokens => generated 3299 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 21 08:31:19.068965 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5414cdc-c869-4b53-99a7-4a8be150bd09 service cinder] Authenticating user token {{(pid=80245) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:31:19.070299 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.fernet_utils [None req-a5414cdc-c869-4b53-99a7-4a8be150bd09 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80245) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 08:31:19.086625 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5414cdc-c869-4b53-99a7-4a8be150bd09 service cinder] Validating token access rules against request {{(pid=80245) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:31:19.093706 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb4a501a-9e42-4890-b377-cce6976081a3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '42d15fc0515945acacbbffcf2356c7a5', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80245) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:31:19.129441 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb4a501a-9e42-4890-b377-cce6976081a3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:31:19.132454 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb4a501a-9e42-4890-b377-cce6976081a3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:31:19.132838 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb4a501a-9e42-4890-b377-cce6976081a3 admin admin] PATH_INFO: `/v3/projects/08a358f65b42429d92b327574b6f4f7c` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:31:19.136259 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb4a501a-9e42-4890-b377-cce6976081a3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=08a358f65b42429d92b327574b6f4f7c)` {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:31:19.138482 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb4a501a-9e42-4890-b377-cce6976081a3 admin admin] RBAC: Authorization granted {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:31:19.203856 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38b6ea28-1c53-4eee-aabe-2f248993af44 service neutron] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:31:19.205297 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.fernet_utils [None req-38b6ea28-1c53-4eee-aabe-2f248993af44 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80244) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 08:31:19.218755 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38b6ea28-1c53-4eee-aabe-2f248993af44 service neutron] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:31:19.227469 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.notifications [None req-bb4a501a-9e42-4890-b377-cce6976081a3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '08a358f65b42429d92b327574b6f4f7c', 'request_id': 'req-bb4a501a-9e42-4890-b377-cce6976081a3'} {{(pid=80245) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 08:31:19.235283 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.notifications [None req-bb4a501a-9e42-4890-b377-cce6976081a3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '08a358f65b42429d92b327574b6f4f7c', 'request_id': 'req-bb4a501a-9e42-4890-b377-cce6976081a3'} {{(pid=80245) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 08:31:19.243161 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.notifications [None req-bb4a501a-9e42-4890-b377-cce6976081a3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '08a358f65b42429d92b327574b6f4f7c', 'request_id': 'req-bb4a501a-9e42-4890-b377-cce6976081a3'} {{(pid=80245) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 08:31:19.258677 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.notifications [None req-bb4a501a-9e42-4890-b377-cce6976081a3 admin admin] The token cache is being invalidate because project 08a358f65b42429d92b327574b6f4f7c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80245) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 21 08:31:19.259150 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.notifications [None req-bb4a501a-9e42-4890-b377-cce6976081a3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80245) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 08:31:19.260050 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.notifications [None req-bb4a501a-9e42-4890-b377-cce6976081a3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80245) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 08:31:19.263681 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: [pid: 80245|app: 0|req: 1010/2026] 10.222.0.47 () {58 vars in 1232 bytes} [Thu Jul 21 08:31:19 2022] DELETE /identity/v3/projects/08a358f65b42429d92b327574b6f4f7c => generated 0 bytes in 195 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 08:31:19.267596 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb4a501a-9e42-4890-b377-cce6976081a3 admin admin] Authenticating user token {{(pid=80245) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:31:19.269674 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.fernet_utils [None req-bb4a501a-9e42-4890-b377-cce6976081a3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80245) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 08:31:19.302229 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5429ed96-069c-49a4-8780-3a5e9bc4a5c0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '42d15fc0515945acacbbffcf2356c7a5', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:31:19.303412 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5429ed96-069c-49a4-8780-3a5e9bc4a5c0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:31:19.303763 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5429ed96-069c-49a4-8780-3a5e9bc4a5c0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:31:19.304111 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5429ed96-069c-49a4-8780-3a5e9bc4a5c0 admin admin] PATH_INFO: `/v3/users/88b257780a2944ec9255c3ea88b6df70` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:31:19.305388 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5429ed96-069c-49a4-8780-3a5e9bc4a5c0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=88b257780a2944ec9255c3ea88b6df70)` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:31:19.307441 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5429ed96-069c-49a4-8780-3a5e9bc4a5c0 admin admin] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:31:19.330275 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG dbcounter [-] [80245] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80245) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:31:19.374503 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb4a501a-9e42-4890-b377-cce6976081a3 admin admin] Validating token access rules against request {{(pid=80245) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:31:19.378274 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.notifications [None req-5429ed96-069c-49a4-8780-3a5e9bc4a5c0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '88b257780a2944ec9255c3ea88b6df70'} {{(pid=80244) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 08:31:19.381433 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-118a64da-0ed8-468c-b338-b10eff211d39 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '42d15fc0515945acacbbffcf2356c7a5', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80245) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:31:19.382430 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-118a64da-0ed8-468c-b338-b10eff211d39 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:31:19.382757 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-118a64da-0ed8-468c-b338-b10eff211d39 admin admin] SCRIPT_NAME: `/identity` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:31:19.383135 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-118a64da-0ed8-468c-b338-b10eff211d39 admin admin] PATH_INFO: `/v3/users/cd807929c2a44571b264657cf1778ef1` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:31:19.384673 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-118a64da-0ed8-468c-b338-b10eff211d39 admin admin] RBAC: Authorizing `identity:delete_user(user_id=cd807929c2a44571b264657cf1778ef1)` {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:31:19.387636 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-118a64da-0ed8-468c-b338-b10eff211d39 admin admin] RBAC: Authorization granted {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:31:19.389079 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.notifications [None req-5429ed96-069c-49a4-8780-3a5e9bc4a5c0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '88b257780a2944ec9255c3ea88b6df70'} {{(pid=80244) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 08:31:19.412677 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.notifications [None req-5429ed96-069c-49a4-8780-3a5e9bc4a5c0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '88b257780a2944ec9255c3ea88b6df70'} {{(pid=80244) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 08:31:19.413531 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.notifications [None req-5429ed96-069c-49a4-8780-3a5e9bc4a5c0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '88b257780a2944ec9255c3ea88b6df70'} {{(pid=80244) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 08:31:19.421383 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.notifications [None req-5429ed96-069c-49a4-8780-3a5e9bc4a5c0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '88b257780a2944ec9255c3ea88b6df70'} {{(pid=80244) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 08:31:19.433300 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 1017/2027] 10.222.0.47 () {58 vars in 1223 bytes} [Thu Jul 21 08:31:19 2022] DELETE /identity/v3/users/88b257780a2944ec9255c3ea88b6df70 => generated 0 bytes in 230 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 21 08:31:19.438752 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fbc1cfb-03b5-405d-a7a6-8eecc87ae663 None None] REQUEST_METHOD: `POST` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:31:19.439283 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fbc1cfb-03b5-405d-a7a6-8eecc87ae663 None None] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:31:19.439896 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fbc1cfb-03b5-405d-a7a6-8eecc87ae663 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:31:19.447760 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.notifications [None req-118a64da-0ed8-468c-b338-b10eff211d39 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'cd807929c2a44571b264657cf1778ef1'} {{(pid=80245) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 08:31:19.459957 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.notifications [None req-118a64da-0ed8-468c-b338-b10eff211d39 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'cd807929c2a44571b264657cf1778ef1'} {{(pid=80245) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 08:31:19.463265 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG dbcounter [-] [80244] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80244) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:31:19.478850 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.auth.core [None req-9fbc1cfb-03b5-405d-a7a6-8eecc87ae663 None None] MFA Rules not processed for user `2dce40b8d74c41ec94cc36e6ff05d412`. Rule list: `[]` (Enabled: `True`). {{(pid=80244) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 08:31:19.480218 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.fernet_utils [None req-9fbc1cfb-03b5-405d-a7a6-8eecc87ae663 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80244) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 08:31:19.490828 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.notifications [None req-118a64da-0ed8-468c-b338-b10eff211d39 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'cd807929c2a44571b264657cf1778ef1'} {{(pid=80245) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 08:31:19.512605 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.notifications [None req-118a64da-0ed8-468c-b338-b10eff211d39 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'cd807929c2a44571b264657cf1778ef1'} {{(pid=80245) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 08:31:19.514760 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.notifications [None req-118a64da-0ed8-468c-b338-b10eff211d39 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'cd807929c2a44571b264657cf1778ef1'} {{(pid=80245) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 08:31:19.531978 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: [pid: 80245|app: 0|req: 1011/2028] 10.222.0.47 () {58 vars in 1223 bytes} [Thu Jul 21 08:31:19 2022] DELETE /identity/v3/users/cd807929c2a44571b264657cf1778ef1 => generated 0 bytes in 265 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 08:31:19.622480 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 1018/2029] 10.222.0.47 () {58 vars in 957 bytes} [Thu Jul 21 08:31:19 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 184 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 21 08:31:19.643586 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-118a64da-0ed8-468c-b338-b10eff211d39 admin admin] Authenticating user token {{(pid=80245) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:31:19.645064 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.fernet_utils [None req-118a64da-0ed8-468c-b338-b10eff211d39 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80245) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 08:31:19.741998 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-118a64da-0ed8-468c-b338-b10eff211d39 admin admin] Validating token access rules against request {{(pid=80245) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:31:19.755599 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03b496b5-c0c0-44b8-b4b4-31db7bfbf90e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20ca3c3bd3774691b3374cfbfec6e494', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4faef13aa50e4e36b6c265fb1398fa14', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80245) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:31:19.756326 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03b496b5-c0c0-44b8-b4b4-31db7bfbf90e service neutron] REQUEST_METHOD: `GET` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:31:19.756712 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03b496b5-c0c0-44b8-b4b4-31db7bfbf90e service neutron] SCRIPT_NAME: `/identity` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:31:19.756997 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03b496b5-c0c0-44b8-b4b4-31db7bfbf90e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:31:19.896398 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03b496b5-c0c0-44b8-b4b4-31db7bfbf90e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:31:19.896398 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03b496b5-c0c0-44b8-b4b4-31db7bfbf90e service neutron] RBAC: Authorization granted {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:31:19.905906 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: [pid: 80245|app: 0|req: 1012/2030] 10.222.0.47 () {60 vars in 1365 bytes} [Thu Jul 21 08:31:19 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 263 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 21 08:31:20.225635 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fbc1cfb-03b5-405d-a7a6-8eecc87ae663 None None] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:31:20.227346 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.fernet_utils [None req-9fbc1cfb-03b5-405d-a7a6-8eecc87ae663 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80244) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 08:31:20.237574 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03b496b5-c0c0-44b8-b4b4-31db7bfbf90e service neutron] Authenticating user token {{(pid=80245) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:31:20.239291 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.fernet_utils [None req-03b496b5-c0c0-44b8-b4b4-31db7bfbf90e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80245) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 08:31:20.260315 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fbc1cfb-03b5-405d-a7a6-8eecc87ae663 None None] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:31:20.262982 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03b496b5-c0c0-44b8-b4b4-31db7bfbf90e service neutron] Validating token access rules against request {{(pid=80245) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:31:20.270535 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc30fb7f-7ed9-4836-b4dd-26211de62eed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '42d15fc0515945acacbbffcf2356c7a5', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:31:20.272172 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc30fb7f-7ed9-4836-b4dd-26211de62eed admin admin] REQUEST_METHOD: `DELETE` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:31:20.272669 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eaf0200-f80d-41e6-918f-8abe66892b92 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '42d15fc0515945acacbbffcf2356c7a5', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80245) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:31:20.272803 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc30fb7f-7ed9-4836-b4dd-26211de62eed admin admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:31:20.273323 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc30fb7f-7ed9-4836-b4dd-26211de62eed admin admin] PATH_INFO: `/v3/projects/860580f1a6a24128afc5814f6b16b320` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:31:20.273796 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eaf0200-f80d-41e6-918f-8abe66892b92 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:31:20.274156 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eaf0200-f80d-41e6-918f-8abe66892b92 admin admin] SCRIPT_NAME: `/identity` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:31:20.274504 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eaf0200-f80d-41e6-918f-8abe66892b92 admin admin] PATH_INFO: `/v3/projects/89c38fb288a142cca8a4a79887f20425` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:31:20.275631 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc30fb7f-7ed9-4836-b4dd-26211de62eed admin admin] RBAC: Authorizing `identity:delete_project(project_id=860580f1a6a24128afc5814f6b16b320)` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:31:20.276052 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6eaf0200-f80d-41e6-918f-8abe66892b92 admin admin] RBAC: Authorizing `identity:delete_project(project_id=89c38fb288a142cca8a4a79887f20425)` {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:31:20.278665 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6eaf0200-f80d-41e6-918f-8abe66892b92 admin admin] RBAC: Authorization granted {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:31:20.279150 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc30fb7f-7ed9-4836-b4dd-26211de62eed admin admin] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:31:20.294133 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG dbcounter [-] [80245] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80245) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:31:20.344699 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.notifications [None req-bc30fb7f-7ed9-4836-b4dd-26211de62eed admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '860580f1a6a24128afc5814f6b16b320', 'request_id': 'req-bc30fb7f-7ed9-4836-b4dd-26211de62eed'} {{(pid=80244) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 08:31:20.353719 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.notifications [None req-bc30fb7f-7ed9-4836-b4dd-26211de62eed admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '860580f1a6a24128afc5814f6b16b320', 'request_id': 'req-bc30fb7f-7ed9-4836-b4dd-26211de62eed'} {{(pid=80244) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 08:31:20.353957 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.notifications [None req-6eaf0200-f80d-41e6-918f-8abe66892b92 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '89c38fb288a142cca8a4a79887f20425', 'request_id': 'req-6eaf0200-f80d-41e6-918f-8abe66892b92'} {{(pid=80245) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 08:31:20.361396 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.notifications [None req-bc30fb7f-7ed9-4836-b4dd-26211de62eed admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '860580f1a6a24128afc5814f6b16b320', 'request_id': 'req-bc30fb7f-7ed9-4836-b4dd-26211de62eed'} {{(pid=80244) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 08:31:20.361605 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.notifications [None req-6eaf0200-f80d-41e6-918f-8abe66892b92 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '89c38fb288a142cca8a4a79887f20425', 'request_id': 'req-6eaf0200-f80d-41e6-918f-8abe66892b92'} {{(pid=80245) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 08:31:20.368773 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.notifications [None req-6eaf0200-f80d-41e6-918f-8abe66892b92 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '89c38fb288a142cca8a4a79887f20425', 'request_id': 'req-6eaf0200-f80d-41e6-918f-8abe66892b92'} {{(pid=80245) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 08:31:20.373616 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.notifications [None req-bc30fb7f-7ed9-4836-b4dd-26211de62eed admin admin] The token cache is being invalidate because project 860580f1a6a24128afc5814f6b16b320 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80244) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 21 08:31:20.373986 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.notifications [None req-bc30fb7f-7ed9-4836-b4dd-26211de62eed admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80244) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 08:31:20.374713 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.notifications [None req-bc30fb7f-7ed9-4836-b4dd-26211de62eed admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80244) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 08:31:20.376430 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 1019/2031] 10.222.0.47 () {58 vars in 1232 bytes} [Thu Jul 21 08:31:20 2022] DELETE /identity/v3/projects/860580f1a6a24128afc5814f6b16b320 => generated 0 bytes in 151 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 08:31:20.381244 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc30fb7f-7ed9-4836-b4dd-26211de62eed admin admin] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:31:20.384846 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.fernet_utils [None req-bc30fb7f-7ed9-4836-b4dd-26211de62eed admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80244) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 08:31:20.389904 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.notifications [None req-6eaf0200-f80d-41e6-918f-8abe66892b92 admin admin] The token cache is being invalidate because project 89c38fb288a142cca8a4a79887f20425 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80245) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 21 08:31:20.390249 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.notifications [None req-6eaf0200-f80d-41e6-918f-8abe66892b92 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80245) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 08:31:20.391027 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.notifications [None req-6eaf0200-f80d-41e6-918f-8abe66892b92 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80245) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 08:31:20.397455 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: [pid: 80245|app: 0|req: 1013/2032] 10.222.0.47 () {58 vars in 1232 bytes} [Thu Jul 21 08:31:20 2022] DELETE /identity/v3/projects/89c38fb288a142cca8a4a79887f20425 => generated 0 bytes in 157 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 21 08:31:20.399080 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc3dea52-c03e-43e2-a36f-2cd41cdb3465 None None] REQUEST_METHOD: `POST` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:31:20.399676 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc3dea52-c03e-43e2-a36f-2cd41cdb3465 None None] SCRIPT_NAME: `/identity` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:31:20.399964 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc3dea52-c03e-43e2-a36f-2cd41cdb3465 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:31:20.424726 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.auth.core [None req-dc3dea52-c03e-43e2-a36f-2cd41cdb3465 None None] MFA Rules not processed for user `2dce40b8d74c41ec94cc36e6ff05d412`. Rule list: `[]` (Enabled: `True`). {{(pid=80245) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 08:31:20.426150 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.fernet_utils [None req-dc3dea52-c03e-43e2-a36f-2cd41cdb3465 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80245) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 08:31:20.533480 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc30fb7f-7ed9-4836-b4dd-26211de62eed admin admin] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:31:20.575467 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: [pid: 80245|app: 0|req: 1014/2033] 10.222.0.47 () {58 vars in 957 bytes} [Thu Jul 21 08:31:20 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 178 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 08:31:20.581963 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG dbcounter [-] [80244] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80244) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:31:20.583956 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86d69c93-25fe-48af-b1fb-ded14516c7e6 None None] REQUEST_METHOD: `POST` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:31:20.584248 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86d69c93-25fe-48af-b1fb-ded14516c7e6 None None] SCRIPT_NAME: `/identity` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:31:20.584529 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86d69c93-25fe-48af-b1fb-ded14516c7e6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:31:20.606889 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.auth.core [None req-86d69c93-25fe-48af-b1fb-ded14516c7e6 None None] MFA Rules not processed for user `2dce40b8d74c41ec94cc36e6ff05d412`. Rule list: `[]` (Enabled: `True`). {{(pid=80245) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 08:31:20.608147 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.fernet_utils [None req-86d69c93-25fe-48af-b1fb-ded14516c7e6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80245) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 08:31:20.628746 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: [pid: 80245|app: 0|req: 1015/2034] 10.222.0.47 () {58 vars in 957 bytes} [Thu Jul 21 08:31:20 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 41 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 08:31:20.628746 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86d69c93-25fe-48af-b1fb-ded14516c7e6 None None] Authenticating user token {{(pid=80245) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:31:20.638247 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86d69c93-25fe-48af-b1fb-ded14516c7e6 None None] Validating token access rules against request {{(pid=80245) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:31:20.640247 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35470b2c-e658-4858-893f-aad0b9bdd94b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '42d15fc0515945acacbbffcf2356c7a5', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:31:20.641100 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35470b2c-e658-4858-893f-aad0b9bdd94b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:31:20.641433 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35470b2c-e658-4858-893f-aad0b9bdd94b admin admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:31:20.641741 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35470b2c-e658-4858-893f-aad0b9bdd94b admin admin] PATH_INFO: `/v3/users/12e5e7b5dbdd427d8c435f8b6ff8599f` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:31:20.643933 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35470b2c-e658-4858-893f-aad0b9bdd94b admin admin] RBAC: Authorizing `identity:delete_user(user_id=12e5e7b5dbdd427d8c435f8b6ff8599f)` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:31:20.644937 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f00a55d9-25a9-4272-a86a-887e34556319 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '42d15fc0515945acacbbffcf2356c7a5', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80245) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:31:20.645607 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f00a55d9-25a9-4272-a86a-887e34556319 admin admin] REQUEST_METHOD: `GET` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:31:20.645890 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f00a55d9-25a9-4272-a86a-887e34556319 admin admin] SCRIPT_NAME: `/identity` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:31:20.646169 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f00a55d9-25a9-4272-a86a-887e34556319 admin admin] PATH_INFO: `/v3/domains` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:31:20.649511 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f00a55d9-25a9-4272-a86a-887e34556319 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80245) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 08:31:20.649511 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f00a55d9-25a9-4272-a86a-887e34556319 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:31:20.649670 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35470b2c-e658-4858-893f-aad0b9bdd94b admin admin] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:31:20.649752 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f00a55d9-25a9-4272-a86a-887e34556319 admin admin] RBAC: Authorization granted {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:31:20.664061 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: [pid: 80245|app: 0|req: 1016/2035] 10.222.0.47 () {58 vars in 1152 bytes} [Thu Jul 21 08:31:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 08:31:20.672720 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f00a55d9-25a9-4272-a86a-887e34556319 admin admin] Authenticating user token {{(pid=80245) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:31:20.688195 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f00a55d9-25a9-4272-a86a-887e34556319 admin admin] Validating token access rules against request {{(pid=80245) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:31:20.693603 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-033156e2-1a3f-4cd9-a2ed-f73ce56b2178 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '42d15fc0515945acacbbffcf2356c7a5', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80245) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:31:20.694391 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-033156e2-1a3f-4cd9-a2ed-f73ce56b2178 admin admin] REQUEST_METHOD: `POST` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:31:20.694702 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-033156e2-1a3f-4cd9-a2ed-f73ce56b2178 admin admin] SCRIPT_NAME: `/identity` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:31:20.695097 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-033156e2-1a3f-4cd9-a2ed-f73ce56b2178 admin admin] PATH_INFO: `/v3/projects` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:31:20.695971 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-033156e2-1a3f-4cd9-a2ed-f73ce56b2178 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:31:20.698079 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-033156e2-1a3f-4cd9-a2ed-f73ce56b2178 admin admin] RBAC: Authorization granted {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:31:20.729797 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.notifications [None req-35470b2c-e658-4858-893f-aad0b9bdd94b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '12e5e7b5dbdd427d8c435f8b6ff8599f'} {{(pid=80244) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 08:31:20.731473 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: [pid: 80245|app: 0|req: 1017/2036] 10.222.0.47 () {60 vars in 1152 bytes} [Thu Jul 21 08:31:20 2022] POST /identity/v3/projects => generated 375 bytes in 63 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 08:31:20.739312 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-033156e2-1a3f-4cd9-a2ed-f73ce56b2178 admin admin] Authenticating user token {{(pid=80245) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:31:20.742843 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.notifications [None req-35470b2c-e658-4858-893f-aad0b9bdd94b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '12e5e7b5dbdd427d8c435f8b6ff8599f'} {{(pid=80244) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 08:31:20.763512 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.notifications [None req-35470b2c-e658-4858-893f-aad0b9bdd94b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '12e5e7b5dbdd427d8c435f8b6ff8599f'} {{(pid=80244) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 08:31:20.764278 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.notifications [None req-35470b2c-e658-4858-893f-aad0b9bdd94b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '12e5e7b5dbdd427d8c435f8b6ff8599f'} {{(pid=80244) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 08:31:20.774139 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.notifications [None req-35470b2c-e658-4858-893f-aad0b9bdd94b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '12e5e7b5dbdd427d8c435f8b6ff8599f'} {{(pid=80244) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 08:31:20.801057 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 1020/2037] 10.222.0.47 () {58 vars in 1223 bytes} [Thu Jul 21 08:31:20 2022] DELETE /identity/v3/users/12e5e7b5dbdd427d8c435f8b6ff8599f => generated 0 bytes in 420 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 21 08:31:20.830559 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG dbcounter [-] [80245] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80245) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:31:20.843526 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-033156e2-1a3f-4cd9-a2ed-f73ce56b2178 admin admin] Validating token access rules against request {{(pid=80245) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:31:20.952433 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35470b2c-e658-4858-893f-aad0b9bdd94b admin admin] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:31:20.954571 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.fernet_utils [None req-35470b2c-e658-4858-893f-aad0b9bdd94b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80244) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 08:31:21.012178 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a0e4a35-5ce4-429a-b3b7-20ac7acb3b54 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '42d15fc0515945acacbbffcf2356c7a5', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80245) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:31:21.014224 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a0e4a35-5ce4-429a-b3b7-20ac7acb3b54 admin admin] REQUEST_METHOD: `POST` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:31:21.014523 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a0e4a35-5ce4-429a-b3b7-20ac7acb3b54 admin admin] SCRIPT_NAME: `/identity` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:31:21.014924 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a0e4a35-5ce4-429a-b3b7-20ac7acb3b54 admin admin] PATH_INFO: `/v3/users` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:31:21.016059 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a0e4a35-5ce4-429a-b3b7-20ac7acb3b54 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:31:21.022415 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a0e4a35-5ce4-429a-b3b7-20ac7acb3b54 admin admin] RBAC: Authorization granted {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:31:21.091543 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: [pid: 80245|app: 0|req: 1018/2038] 10.222.0.47 () {60 vars in 1143 bytes} [Thu Jul 21 08:31:20 2022] POST /identity/v3/users => generated 288 bytes in 351 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 08:31:21.094788 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a0e4a35-5ce4-429a-b3b7-20ac7acb3b54 admin admin] Authenticating user token {{(pid=80245) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:31:21.097908 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.fernet_utils [None req-2a0e4a35-5ce4-429a-b3b7-20ac7acb3b54 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80245) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 08:31:21.112965 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a0e4a35-5ce4-429a-b3b7-20ac7acb3b54 admin admin] Validating token access rules against request {{(pid=80245) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:31:21.118013 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e52e552-c951-406e-bc71-6c846e90b582 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '42d15fc0515945acacbbffcf2356c7a5', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80245) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:31:21.118700 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e52e552-c951-406e-bc71-6c846e90b582 admin admin] REQUEST_METHOD: `GET` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:31:21.118982 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e52e552-c951-406e-bc71-6c846e90b582 admin admin] SCRIPT_NAME: `/identity` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:31:21.119260 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e52e552-c951-406e-bc71-6c846e90b582 admin admin] PATH_INFO: `/v3/roles` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:31:21.120165 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e52e552-c951-406e-bc71-6c846e90b582 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:31:21.122451 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e52e552-c951-406e-bc71-6c846e90b582 admin admin] RBAC: Authorization granted {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:31:21.127636 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35470b2c-e658-4858-893f-aad0b9bdd94b admin admin] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:31:21.133387 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c031ddf9-2b38-48f8-97e9-9378401fd073 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '42d15fc0515945acacbbffcf2356c7a5', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:31:21.134434 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c031ddf9-2b38-48f8-97e9-9378401fd073 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:31:21.137899 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: [pid: 80245|app: 0|req: 1019/2039] 10.222.0.47 () {58 vars in 1121 bytes} [Thu Jul 21 08:31:21 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 08:31:21.138071 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c031ddf9-2b38-48f8-97e9-9378401fd073 admin admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:31:21.138071 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c031ddf9-2b38-48f8-97e9-9378401fd073 admin admin] PATH_INFO: `/v3/users/848c949f57a34c84bb40c86cd4729d11` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:31:21.138729 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c031ddf9-2b38-48f8-97e9-9378401fd073 admin admin] RBAC: Authorizing `identity:delete_user(user_id=848c949f57a34c84bb40c86cd4729d11)` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:31:21.139145 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e52e552-c951-406e-bc71-6c846e90b582 admin admin] Authenticating user token {{(pid=80245) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:31:21.142731 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c031ddf9-2b38-48f8-97e9-9378401fd073 admin admin] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:31:21.151752 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e52e552-c951-406e-bc71-6c846e90b582 admin admin] Validating token access rules against request {{(pid=80245) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:31:21.157753 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e970982a-e17a-4c2a-bb2a-c0676fa419ff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '42d15fc0515945acacbbffcf2356c7a5', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80245) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:31:21.158446 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e970982a-e17a-4c2a-bb2a-c0676fa419ff admin admin] REQUEST_METHOD: `GET` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:31:21.158745 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e970982a-e17a-4c2a-bb2a-c0676fa419ff admin admin] SCRIPT_NAME: `/identity` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:31:21.158993 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e970982a-e17a-4c2a-bb2a-c0676fa419ff admin admin] PATH_INFO: `/v3/roles` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:31:21.160511 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e970982a-e17a-4c2a-bb2a-c0676fa419ff admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:31:21.162153 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e970982a-e17a-4c2a-bb2a-c0676fa419ff admin admin] RBAC: Authorization granted {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:31:21.185103 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: [pid: 80245|app: 0|req: 1020/2040] 10.222.0.47 () {58 vars in 1121 bytes} [Thu Jul 21 08:31:21 2022] GET /identity/v3/roles => generated 1420 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 08:31:21.186379 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e970982a-e17a-4c2a-bb2a-c0676fa419ff admin admin] Authenticating user token {{(pid=80245) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:31:21.198861 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e970982a-e17a-4c2a-bb2a-c0676fa419ff admin admin] Validating token access rules against request {{(pid=80245) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:31:21.205692 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff1c79c8-7a61-4015-a067-52cbbc9e205d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '42d15fc0515945acacbbffcf2356c7a5', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80245) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:31:21.207096 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff1c79c8-7a61-4015-a067-52cbbc9e205d admin admin] REQUEST_METHOD: `PUT` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:31:21.207638 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff1c79c8-7a61-4015-a067-52cbbc9e205d admin admin] SCRIPT_NAME: `/identity` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:31:21.209431 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff1c79c8-7a61-4015-a067-52cbbc9e205d admin admin] PATH_INFO: `/v3/projects/63233c4fbfb843c894d87f2d4af28067/users/9a84db7484ad48089fc1303fa54b8670/roles/3a074738221e4601b5205715ba87d240` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:31:21.218077 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.notifications [None req-c031ddf9-2b38-48f8-97e9-9378401fd073 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '848c949f57a34c84bb40c86cd4729d11'} {{(pid=80244) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 08:31:21.234497 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.notifications [None req-c031ddf9-2b38-48f8-97e9-9378401fd073 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '848c949f57a34c84bb40c86cd4729d11'} {{(pid=80244) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 08:31:21.250625 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff1c79c8-7a61-4015-a067-52cbbc9e205d admin admin] RBAC: Authorizing `identity:create_grant(project_id=63233c4fbfb843c894d87f2d4af28067, user_id=9a84db7484ad48089fc1303fa54b8670, role_id=3a074738221e4601b5205715ba87d240)` {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:31:21.251106 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG dbcounter [-] [80244] Writing DB stats keystone:SELECT=80,keystone:DELETE=18,keystone:INSERT=2 {{(pid=80244) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:31:21.253929 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff1c79c8-7a61-4015-a067-52cbbc9e205d admin admin] RBAC: Authorization granted {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:31:21.272525 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: [pid: 80245|app: 0|req: 1021/2041] 10.222.0.47 () {60 vars in 1482 bytes} [Thu Jul 21 08:31:21 2022] PUT /identity/v3/projects/63233c4fbfb843c894d87f2d4af28067/users/9a84db7484ad48089fc1303fa54b8670/roles/3a074738221e4601b5205715ba87d240 => generated 0 bytes in 86 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 08:31:21.275130 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.notifications [None req-c031ddf9-2b38-48f8-97e9-9378401fd073 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '848c949f57a34c84bb40c86cd4729d11'} {{(pid=80244) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 08:31:21.282950 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.notifications [None req-c031ddf9-2b38-48f8-97e9-9378401fd073 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '848c949f57a34c84bb40c86cd4729d11'} {{(pid=80244) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 08:31:21.285624 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32c941c6-3cee-4795-b4dc-8c75f3068d84 None None] REQUEST_METHOD: `POST` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:31:21.285919 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32c941c6-3cee-4795-b4dc-8c75f3068d84 None None] SCRIPT_NAME: `/identity` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:31:21.286199 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32c941c6-3cee-4795-b4dc-8c75f3068d84 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:31:21.296043 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.notifications [None req-c031ddf9-2b38-48f8-97e9-9378401fd073 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '848c949f57a34c84bb40c86cd4729d11'} {{(pid=80244) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 08:31:21.314890 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 1021/2042] 10.222.0.47 () {58 vars in 1223 bytes} [Thu Jul 21 08:31:20 2022] DELETE /identity/v3/users/848c949f57a34c84bb40c86cd4729d11 => generated 0 bytes in 364 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 21 08:31:21.320960 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ae696ba-6fc9-4e15-b9e4-523dc3aa4997 None None] REQUEST_METHOD: `POST` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:31:21.321522 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ae696ba-6fc9-4e15-b9e4-523dc3aa4997 None None] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:31:21.321957 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ae696ba-6fc9-4e15-b9e4-523dc3aa4997 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:31:21.325709 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.auth.core [None req-32c941c6-3cee-4795-b4dc-8c75f3068d84 None None] MFA Rules not processed for user `9a84db7484ad48089fc1303fa54b8670`. Rule list: `[]` (Enabled: `True`). {{(pid=80245) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 08:31:21.327694 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.fernet_utils [None req-32c941c6-3cee-4795-b4dc-8c75f3068d84 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80245) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 08:31:21.360647 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.auth.core [None req-8ae696ba-6fc9-4e15-b9e4-523dc3aa4997 None None] MFA Rules not processed for user `2dce40b8d74c41ec94cc36e6ff05d412`. Rule list: `[]` (Enabled: `True`). {{(pid=80244) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 08:31:21.366848 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.fernet_utils [None req-8ae696ba-6fc9-4e15-b9e4-523dc3aa4997 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80244) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 08:31:21.491125 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: [pid: 80245|app: 0|req: 1022/2043] 10.222.0.47 () {58 vars in 957 bytes} [Thu Jul 21 08:31:21 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 206 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 21 08:31:21.492517 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32c941c6-3cee-4795-b4dc-8c75f3068d84 None None] Authenticating user token {{(pid=80245) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:31:21.494397 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.fernet_utils [None req-32c941c6-3cee-4795-b4dc-8c75f3068d84 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80245) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 08:31:21.533585 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 1022/2044] 10.222.0.47 () {58 vars in 957 bytes} [Thu Jul 21 08:31:21 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 214 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 08:31:21.534761 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ae696ba-6fc9-4e15-b9e4-523dc3aa4997 None None] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:31:21.546811 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.fernet_utils [None req-8ae696ba-6fc9-4e15-b9e4-523dc3aa4997 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80244) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 08:31:21.586564 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG dbcounter [-] [80245] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80245) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:31:21.641864 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32c941c6-3cee-4795-b4dc-8c75f3068d84 None None] Validating token access rules against request {{(pid=80245) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:31:21.648002 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8091048d-156c-4789-9665-fd93ad57fb03 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '42d15fc0515945acacbbffcf2356c7a5', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80245) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:31:21.648982 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8091048d-156c-4789-9665-fd93ad57fb03 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:31:21.649379 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8091048d-156c-4789-9665-fd93ad57fb03 admin admin] SCRIPT_NAME: `/identity` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:31:21.649665 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8091048d-156c-4789-9665-fd93ad57fb03 admin admin] PATH_INFO: `/v3/projects/9c69d9da4c59496f9cbae25f678a8dec` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:31:21.651429 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8091048d-156c-4789-9665-fd93ad57fb03 admin admin] RBAC: Authorizing `identity:delete_project(project_id=9c69d9da4c59496f9cbae25f678a8dec)` {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:31:21.654188 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8091048d-156c-4789-9665-fd93ad57fb03 admin admin] RBAC: Authorization granted {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:31:21.695041 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ae696ba-6fc9-4e15-b9e4-523dc3aa4997 None None] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:31:21.699412 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1daedbdc-a718-49d5-ba9e-b6f634b5f219 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9b0524aed8c74abd8949b860d6b93936', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4faef13aa50e4e36b6c265fb1398fa14', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:31:21.700397 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1daedbdc-a718-49d5-ba9e-b6f634b5f219 service glance] REQUEST_METHOD: `GET` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:31:21.700772 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1daedbdc-a718-49d5-ba9e-b6f634b5f219 service glance] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:31:21.701185 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1daedbdc-a718-49d5-ba9e-b6f634b5f219 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:31:21.712529 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1daedbdc-a718-49d5-ba9e-b6f634b5f219 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:31:21.714436 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1daedbdc-a718-49d5-ba9e-b6f634b5f219 service glance] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:31:21.726232 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 1023/2045] 10.222.0.47 () {60 vars in 1365 bytes} [Thu Jul 21 08:31:21 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 192 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 08:31:21.738695 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1daedbdc-a718-49d5-ba9e-b6f634b5f219 service glance] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:31:21.742084 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.fernet_utils [None req-1daedbdc-a718-49d5-ba9e-b6f634b5f219 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80244) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 08:31:21.749287 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.notifications [None req-8091048d-156c-4789-9665-fd93ad57fb03 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9c69d9da4c59496f9cbae25f678a8dec', 'request_id': 'req-8091048d-156c-4789-9665-fd93ad57fb03'} {{(pid=80245) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 08:31:21.777711 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.notifications [None req-8091048d-156c-4789-9665-fd93ad57fb03 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9c69d9da4c59496f9cbae25f678a8dec', 'request_id': 'req-8091048d-156c-4789-9665-fd93ad57fb03'} {{(pid=80245) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 08:31:21.798021 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.notifications [None req-8091048d-156c-4789-9665-fd93ad57fb03 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9c69d9da4c59496f9cbae25f678a8dec', 'request_id': 'req-8091048d-156c-4789-9665-fd93ad57fb03'} {{(pid=80245) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 08:31:21.828683 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.notifications [None req-8091048d-156c-4789-9665-fd93ad57fb03 admin admin] The token cache is being invalidate because project 9c69d9da4c59496f9cbae25f678a8dec was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80245) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 21 08:31:21.829445 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.notifications [None req-8091048d-156c-4789-9665-fd93ad57fb03 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80245) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 08:31:21.830744 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.notifications [None req-8091048d-156c-4789-9665-fd93ad57fb03 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80245) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 08:31:21.831196 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1daedbdc-a718-49d5-ba9e-b6f634b5f219 service glance] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:31:21.833387 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: [pid: 80245|app: 0|req: 1023/2046] 10.222.0.47 () {58 vars in 1232 bytes} [Thu Jul 21 08:31:21 2022] DELETE /identity/v3/projects/9c69d9da4c59496f9cbae25f678a8dec => generated 0 bytes in 341 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 08:31:21.839667 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0928b169-ee7a-4091-88ba-05ad70597a6c None None] REQUEST_METHOD: `POST` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:31:21.842282 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0928b169-ee7a-4091-88ba-05ad70597a6c None None] SCRIPT_NAME: `/identity` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:31:21.842918 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0928b169-ee7a-4091-88ba-05ad70597a6c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:31:21.877094 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG dbcounter [-] [80244] Writing DB stats keystone:DELETE=2,keystone:SELECT=98 {{(pid=80244) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:31:21.892545 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.auth.core [None req-0928b169-ee7a-4091-88ba-05ad70597a6c None None] MFA Rules not processed for user `2dce40b8d74c41ec94cc36e6ff05d412`. Rule list: `[]` (Enabled: `True`). {{(pid=80245) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 08:31:21.893993 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.fernet_utils [None req-0928b169-ee7a-4091-88ba-05ad70597a6c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80245) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 08:31:21.959111 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-319f4e08-c863-4c9c-bd05-393edfa36220 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20ca3c3bd3774691b3374cfbfec6e494', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4faef13aa50e4e36b6c265fb1398fa14', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:31:21.960007 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-319f4e08-c863-4c9c-bd05-393edfa36220 service neutron] REQUEST_METHOD: `GET` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:31:21.960447 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-319f4e08-c863-4c9c-bd05-393edfa36220 service neutron] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:31:21.960778 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-319f4e08-c863-4c9c-bd05-393edfa36220 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:31:21.963071 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.fernet_utils [None req-319f4e08-c863-4c9c-bd05-393edfa36220 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80244) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 08:31:22.003803 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: [pid: 80245|app: 0|req: 1024/2047] 10.222.0.47 () {58 vars in 957 bytes} [Thu Jul 21 08:31:21 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 08:31:22.005378 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0928b169-ee7a-4091-88ba-05ad70597a6c None None] Authenticating user token {{(pid=80245) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:31:22.007999 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.fernet_utils [None req-0928b169-ee7a-4091-88ba-05ad70597a6c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80245) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 08:31:22.022618 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0928b169-ee7a-4091-88ba-05ad70597a6c None None] Validating token access rules against request {{(pid=80245) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:31:22.028898 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb362daf-54bd-4a27-9a50-71e9fd6c2937 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '42d15fc0515945acacbbffcf2356c7a5', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80245) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:31:22.029791 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb362daf-54bd-4a27-9a50-71e9fd6c2937 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:31:22.030078 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb362daf-54bd-4a27-9a50-71e9fd6c2937 admin admin] SCRIPT_NAME: `/identity` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:31:22.030342 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb362daf-54bd-4a27-9a50-71e9fd6c2937 admin admin] PATH_INFO: `/v3/users/e35645c0565c4de29bac3e0ad8f1f245` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:31:22.034422 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb362daf-54bd-4a27-9a50-71e9fd6c2937 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e35645c0565c4de29bac3e0ad8f1f245)` {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:31:22.036403 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb362daf-54bd-4a27-9a50-71e9fd6c2937 admin admin] RBAC: Authorization granted {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:31:22.076271 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-319f4e08-c863-4c9c-bd05-393edfa36220 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:31:22.077234 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-319f4e08-c863-4c9c-bd05-393edfa36220 service neutron] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:31:22.090684 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 1024/2048] 10.222.0.47 () {60 vars in 1365 bytes} [Thu Jul 21 08:31:21 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 352 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 08:31:22.092361 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18e048b7-bf99-47bf-a2eb-5c3da6025758 None None] REQUEST_METHOD: `POST` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:31:22.092817 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18e048b7-bf99-47bf-a2eb-5c3da6025758 None None] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:31:22.093014 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18e048b7-bf99-47bf-a2eb-5c3da6025758 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:31:22.112439 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.notifications [None req-fb362daf-54bd-4a27-9a50-71e9fd6c2937 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e35645c0565c4de29bac3e0ad8f1f245'} {{(pid=80245) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 08:31:22.118156 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.auth.core [None req-18e048b7-bf99-47bf-a2eb-5c3da6025758 None None] MFA Rules not processed for user `2dce40b8d74c41ec94cc36e6ff05d412`. Rule list: `[]` (Enabled: `True`). {{(pid=80244) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 08:31:22.119864 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.fernet_utils [None req-18e048b7-bf99-47bf-a2eb-5c3da6025758 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80244) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 08:31:22.123508 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.notifications [None req-fb362daf-54bd-4a27-9a50-71e9fd6c2937 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e35645c0565c4de29bac3e0ad8f1f245'} {{(pid=80245) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 08:31:22.133782 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 1025/2049] 10.222.0.47 () {58 vars in 957 bytes} [Thu Jul 21 08:31:22 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 43 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 08:31:22.139569 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18e048b7-bf99-47bf-a2eb-5c3da6025758 None None] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:31:22.143357 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG dbcounter [-] [80245] Writing DB stats keystone:SELECT=82,keystone:DELETE=16,keystone:INSERT=2 {{(pid=80245) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:31:22.153823 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18e048b7-bf99-47bf-a2eb-5c3da6025758 None None] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:31:22.161295 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.notifications [None req-fb362daf-54bd-4a27-9a50-71e9fd6c2937 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e35645c0565c4de29bac3e0ad8f1f245'} {{(pid=80245) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 08:31:22.164822 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bf3a0ec-ba39-4f0d-a13e-bafe36dfa2f7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '42d15fc0515945acacbbffcf2356c7a5', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:31:22.165678 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bf3a0ec-ba39-4f0d-a13e-bafe36dfa2f7 admin admin] REQUEST_METHOD: `GET` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:31:22.166056 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bf3a0ec-ba39-4f0d-a13e-bafe36dfa2f7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:31:22.166429 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bf3a0ec-ba39-4f0d-a13e-bafe36dfa2f7 admin admin] PATH_INFO: `/v3/domains` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:31:22.167620 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bf3a0ec-ba39-4f0d-a13e-bafe36dfa2f7 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80244) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 08:31:22.168077 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bf3a0ec-ba39-4f0d-a13e-bafe36dfa2f7 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:31:22.171596 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bf3a0ec-ba39-4f0d-a13e-bafe36dfa2f7 admin admin] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:31:22.174665 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.notifications [None req-fb362daf-54bd-4a27-9a50-71e9fd6c2937 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e35645c0565c4de29bac3e0ad8f1f245'} {{(pid=80245) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 08:31:22.176417 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.notifications [None req-fb362daf-54bd-4a27-9a50-71e9fd6c2937 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e35645c0565c4de29bac3e0ad8f1f245'} {{(pid=80245) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 08:31:22.194829 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: [pid: 80245|app: 0|req: 1025/2050] 10.222.0.47 () {58 vars in 1223 bytes} [Thu Jul 21 08:31:22 2022] DELETE /identity/v3/users/e35645c0565c4de29bac3e0ad8f1f245 => generated 0 bytes in 190 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 08:31:22.196103 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 1026/2051] 10.222.0.47 () {58 vars in 1152 bytes} [Thu Jul 21 08:31:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 57 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 08:31:22.200752 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb362daf-54bd-4a27-9a50-71e9fd6c2937 admin admin] Authenticating user token {{(pid=80245) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:31:22.203110 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acfc8aee-3bc9-40c0-b24f-a7bce05dcc51 None None] REQUEST_METHOD: `POST` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:31:22.205151 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.fernet_utils [None req-fb362daf-54bd-4a27-9a50-71e9fd6c2937 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80245) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 08:31:22.205683 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acfc8aee-3bc9-40c0-b24f-a7bce05dcc51 None None] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:31:22.206726 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acfc8aee-3bc9-40c0-b24f-a7bce05dcc51 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:31:22.240976 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.auth.core [None req-acfc8aee-3bc9-40c0-b24f-a7bce05dcc51 None None] MFA Rules not processed for user `2dce40b8d74c41ec94cc36e6ff05d412`. Rule list: `[]` (Enabled: `True`). {{(pid=80244) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 08:31:22.243987 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.fernet_utils [None req-acfc8aee-3bc9-40c0-b24f-a7bce05dcc51 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80244) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 08:31:22.333715 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb362daf-54bd-4a27-9a50-71e9fd6c2937 admin admin] Validating token access rules against request {{(pid=80245) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:31:22.348789 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2b9e123-445d-49d4-a394-07d3a266baff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '42d15fc0515945acacbbffcf2356c7a5', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80245) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:31:22.349915 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2b9e123-445d-49d4-a394-07d3a266baff admin admin] REQUEST_METHOD: `POST` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:31:22.351345 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2b9e123-445d-49d4-a394-07d3a266baff admin admin] SCRIPT_NAME: `/identity` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:31:22.352081 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2b9e123-445d-49d4-a394-07d3a266baff admin admin] PATH_INFO: `/v3/projects` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:31:22.354459 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2b9e123-445d-49d4-a394-07d3a266baff admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:31:22.358976 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2b9e123-445d-49d4-a394-07d3a266baff admin admin] RBAC: Authorization granted {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:31:22.397095 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 1027/2052] 10.222.0.47 () {58 vars in 957 bytes} [Thu Jul 21 08:31:22 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 198 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 08:31:22.409611 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acfc8aee-3bc9-40c0-b24f-a7bce05dcc51 None None] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:31:22.412368 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.fernet_utils [None req-acfc8aee-3bc9-40c0-b24f-a7bce05dcc51 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80244) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 08:31:22.444786 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: [pid: 80245|app: 0|req: 1026/2053] 10.222.0.47 () {60 vars in 1152 bytes} [Thu Jul 21 08:31:22 2022] POST /identity/v3/projects => generated 409 bytes in 249 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 08:31:22.446468 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2b9e123-445d-49d4-a394-07d3a266baff admin admin] Authenticating user token {{(pid=80245) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:31:22.550485 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG dbcounter [-] [80244] Writing DB stats keystone:SELECT=100 {{(pid=80244) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:31:22.564105 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acfc8aee-3bc9-40c0-b24f-a7bce05dcc51 None None] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:31:22.590326 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2b9e123-445d-49d4-a394-07d3a266baff admin admin] Validating token access rules against request {{(pid=80245) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:31:22.597497 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-190c2853-66ca-4038-b401-a1880b9aa605 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '42d15fc0515945acacbbffcf2356c7a5', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80245) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:31:22.599361 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-190c2853-66ca-4038-b401-a1880b9aa605 admin admin] REQUEST_METHOD: `POST` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:31:22.599361 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-190c2853-66ca-4038-b401-a1880b9aa605 admin admin] SCRIPT_NAME: `/identity` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:31:22.599361 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-190c2853-66ca-4038-b401-a1880b9aa605 admin admin] PATH_INFO: `/v3/users` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:31:22.599963 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-190c2853-66ca-4038-b401-a1880b9aa605 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:31:22.617427 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-190c2853-66ca-4038-b401-a1880b9aa605 admin admin] RBAC: Authorization granted {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:31:22.645127 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: [pid: 80245|app: 0|req: 1027/2054] 10.222.0.47 () {60 vars in 1143 bytes} [Thu Jul 21 08:31:22 2022] POST /identity/v3/users => generated 305 bytes in 199 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 08:31:22.646098 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-190c2853-66ca-4038-b401-a1880b9aa605 admin admin] Authenticating user token {{(pid=80245) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:31:22.649173 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.fernet_utils [None req-190c2853-66ca-4038-b401-a1880b9aa605 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80245) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 08:31:22.660747 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f532043e-1b10-46ef-89d3-dfb0203cc446 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20ca3c3bd3774691b3374cfbfec6e494', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4faef13aa50e4e36b6c265fb1398fa14', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:31:22.661458 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f532043e-1b10-46ef-89d3-dfb0203cc446 service neutron] REQUEST_METHOD: `GET` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:31:22.661733 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f532043e-1b10-46ef-89d3-dfb0203cc446 service neutron] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:31:22.662006 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f532043e-1b10-46ef-89d3-dfb0203cc446 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:31:22.663215 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-190c2853-66ca-4038-b401-a1880b9aa605 admin admin] Validating token access rules against request {{(pid=80245) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:31:22.669785 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80392eeb-c2c5-4d51-b84b-a7a0e301a848 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '42d15fc0515945acacbbffcf2356c7a5', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80245) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:31:22.671390 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80392eeb-c2c5-4d51-b84b-a7a0e301a848 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:31:22.671613 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80392eeb-c2c5-4d51-b84b-a7a0e301a848 admin admin] SCRIPT_NAME: `/identity` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:31:22.671871 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80392eeb-c2c5-4d51-b84b-a7a0e301a848 admin admin] PATH_INFO: `/v3/projects/02978068e4fd4beca2e126b8d06d2b7c` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:31:22.673088 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80392eeb-c2c5-4d51-b84b-a7a0e301a848 admin admin] RBAC: Authorizing `identity:delete_project(project_id=02978068e4fd4beca2e126b8d06d2b7c)` {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:31:22.674241 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f532043e-1b10-46ef-89d3-dfb0203cc446 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:31:22.675213 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80392eeb-c2c5-4d51-b84b-a7a0e301a848 admin admin] RBAC: Authorization granted {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:31:22.676050 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f532043e-1b10-46ef-89d3-dfb0203cc446 service neutron] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:31:22.688840 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 1028/2055] 10.222.0.47 () {60 vars in 1365 bytes} [Thu Jul 21 08:31:22 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 281 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 08:31:22.690276 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f532043e-1b10-46ef-89d3-dfb0203cc446 service neutron] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:31:22.701538 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f532043e-1b10-46ef-89d3-dfb0203cc446 service neutron] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:31:22.711905 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f263efad-e319-47b3-b2dd-557bb91a05e9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '42d15fc0515945acacbbffcf2356c7a5', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:31:22.712724 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f263efad-e319-47b3-b2dd-557bb91a05e9 admin admin] REQUEST_METHOD: `GET` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:31:22.713017 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f263efad-e319-47b3-b2dd-557bb91a05e9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:31:22.713558 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f263efad-e319-47b3-b2dd-557bb91a05e9 admin admin] PATH_INFO: `/v3/roles` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:31:22.714420 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f263efad-e319-47b3-b2dd-557bb91a05e9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:31:22.716938 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f263efad-e319-47b3-b2dd-557bb91a05e9 admin admin] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:31:22.734971 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 1029/2056] 10.222.0.47 () {58 vars in 1121 bytes} [Thu Jul 21 08:31:22 2022] GET /identity/v3/roles => generated 1420 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 08:31:22.740546 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f263efad-e319-47b3-b2dd-557bb91a05e9 admin admin] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:31:22.757439 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f263efad-e319-47b3-b2dd-557bb91a05e9 admin admin] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:31:22.773879 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.notifications [None req-80392eeb-c2c5-4d51-b84b-a7a0e301a848 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '02978068e4fd4beca2e126b8d06d2b7c', 'request_id': 'req-80392eeb-c2c5-4d51-b84b-a7a0e301a848'} {{(pid=80245) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 08:31:22.778351 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG dbcounter [-] [80245] Writing DB stats keystone:SELECT=91,keystone:DELETE=5,keystone:INSERT=4 {{(pid=80245) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:31:22.797610 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.notifications [None req-80392eeb-c2c5-4d51-b84b-a7a0e301a848 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '02978068e4fd4beca2e126b8d06d2b7c', 'request_id': 'req-80392eeb-c2c5-4d51-b84b-a7a0e301a848'} {{(pid=80245) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 08:31:22.807799 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.notifications [None req-80392eeb-c2c5-4d51-b84b-a7a0e301a848 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '02978068e4fd4beca2e126b8d06d2b7c', 'request_id': 'req-80392eeb-c2c5-4d51-b84b-a7a0e301a848'} {{(pid=80245) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 08:31:22.829443 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.notifications [None req-80392eeb-c2c5-4d51-b84b-a7a0e301a848 admin admin] The token cache is being invalidate because project 02978068e4fd4beca2e126b8d06d2b7c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80245) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 21 08:31:22.829955 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.notifications [None req-80392eeb-c2c5-4d51-b84b-a7a0e301a848 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80245) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 08:31:22.831697 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.notifications [None req-80392eeb-c2c5-4d51-b84b-a7a0e301a848 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80245) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 08:31:22.834073 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: [pid: 80245|app: 0|req: 1028/2057] 10.222.0.47 () {58 vars in 1232 bytes} [Thu Jul 21 08:31:22 2022] DELETE /identity/v3/projects/02978068e4fd4beca2e126b8d06d2b7c => generated 0 bytes in 188 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 08:31:22.843222 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80392eeb-c2c5-4d51-b84b-a7a0e301a848 admin admin] Authenticating user token {{(pid=80245) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:31:22.847505 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.fernet_utils [None req-80392eeb-c2c5-4d51-b84b-a7a0e301a848 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80245) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 08:31:22.907617 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2583256a-efc3-4c24-92e0-494403e4f919 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '42d15fc0515945acacbbffcf2356c7a5', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:31:22.908324 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2583256a-efc3-4c24-92e0-494403e4f919 admin admin] REQUEST_METHOD: `GET` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:31:22.908610 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2583256a-efc3-4c24-92e0-494403e4f919 admin admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:31:22.908887 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2583256a-efc3-4c24-92e0-494403e4f919 admin admin] PATH_INFO: `/v3/roles` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:31:22.909785 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2583256a-efc3-4c24-92e0-494403e4f919 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:31:22.911284 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2583256a-efc3-4c24-92e0-494403e4f919 admin admin] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:31:22.935394 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 1030/2058] 10.222.0.47 () {58 vars in 1121 bytes} [Thu Jul 21 08:31:22 2022] GET /identity/v3/roles => generated 1420 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 21 08:31:22.944495 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2583256a-efc3-4c24-92e0-494403e4f919 admin admin] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:31:22.948419 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.fernet_utils [None req-2583256a-efc3-4c24-92e0-494403e4f919 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80244) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 08:31:22.975722 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2583256a-efc3-4c24-92e0-494403e4f919 admin admin] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:31:22.988631 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7c84cf1-2a01-4a2e-a46b-7afbfbba9730 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '42d15fc0515945acacbbffcf2356c7a5', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:31:22.989684 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7c84cf1-2a01-4a2e-a46b-7afbfbba9730 admin admin] REQUEST_METHOD: `PUT` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:31:22.992131 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7c84cf1-2a01-4a2e-a46b-7afbfbba9730 admin admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:31:22.992448 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7c84cf1-2a01-4a2e-a46b-7afbfbba9730 admin admin] PATH_INFO: `/v3/projects/5642b20ea95e4ddba1e2f38f5db9ef80/users/b970723297fe4217ae31c4f11fa8801a/roles/3a074738221e4601b5205715ba87d240` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:31:22.994757 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80392eeb-c2c5-4d51-b84b-a7a0e301a848 admin admin] Validating token access rules against request {{(pid=80245) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:31:23.003157 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8f759a1-8dcd-4a21-ac84-d207ac453477 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '42d15fc0515945acacbbffcf2356c7a5', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80245) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:31:23.005681 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8f759a1-8dcd-4a21-ac84-d207ac453477 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:31:23.006335 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8f759a1-8dcd-4a21-ac84-d207ac453477 admin admin] SCRIPT_NAME: `/identity` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:31:23.007171 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8f759a1-8dcd-4a21-ac84-d207ac453477 admin admin] PATH_INFO: `/v3/users/a3f0f8badbc44664a2d0c572aac9f021` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:31:23.008877 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8f759a1-8dcd-4a21-ac84-d207ac453477 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a3f0f8badbc44664a2d0c572aac9f021)` {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:31:23.011258 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8f759a1-8dcd-4a21-ac84-d207ac453477 admin admin] RBAC: Authorization granted {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:31:23.032782 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7c84cf1-2a01-4a2e-a46b-7afbfbba9730 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5642b20ea95e4ddba1e2f38f5db9ef80, user_id=b970723297fe4217ae31c4f11fa8801a, role_id=3a074738221e4601b5205715ba87d240)` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:31:23.037063 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7c84cf1-2a01-4a2e-a46b-7afbfbba9730 admin admin] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:31:23.079842 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 1031/2059] 10.222.0.47 () {60 vars in 1482 bytes} [Thu Jul 21 08:31:22 2022] PUT /identity/v3/projects/5642b20ea95e4ddba1e2f38f5db9ef80/users/b970723297fe4217ae31c4f11fa8801a/roles/3a074738221e4601b5205715ba87d240 => generated 0 bytes in 138 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 08:31:23.095961 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03b1795f-b13b-428a-aa62-4ac2f116f2b1 None None] REQUEST_METHOD: `POST` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:31:23.096196 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03b1795f-b13b-428a-aa62-4ac2f116f2b1 None None] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:31:23.097763 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03b1795f-b13b-428a-aa62-4ac2f116f2b1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:31:23.152075 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.notifications [None req-d8f759a1-8dcd-4a21-ac84-d207ac453477 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a3f0f8badbc44664a2d0c572aac9f021'} {{(pid=80245) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 08:31:23.163517 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.notifications [None req-d8f759a1-8dcd-4a21-ac84-d207ac453477 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a3f0f8badbc44664a2d0c572aac9f021'} {{(pid=80245) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 08:31:23.179998 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.auth.core [None req-03b1795f-b13b-428a-aa62-4ac2f116f2b1 None None] MFA Rules not processed for user `b970723297fe4217ae31c4f11fa8801a`. Rule list: `[]` (Enabled: `True`). {{(pid=80244) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 08:31:23.181398 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.fernet_utils [None req-03b1795f-b13b-428a-aa62-4ac2f116f2b1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80244) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 08:31:23.186923 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.notifications [None req-d8f759a1-8dcd-4a21-ac84-d207ac453477 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a3f0f8badbc44664a2d0c572aac9f021'} {{(pid=80245) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 08:31:23.197602 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.notifications [None req-d8f759a1-8dcd-4a21-ac84-d207ac453477 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a3f0f8badbc44664a2d0c572aac9f021'} {{(pid=80245) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 08:31:23.198704 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.notifications [None req-d8f759a1-8dcd-4a21-ac84-d207ac453477 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a3f0f8badbc44664a2d0c572aac9f021'} {{(pid=80245) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 08:31:23.209182 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: [pid: 80245|app: 0|req: 1029/2060] 10.222.0.47 () {58 vars in 1223 bytes} [Thu Jul 21 08:31:22 2022] DELETE /identity/v3/users/a3f0f8badbc44664a2d0c572aac9f021 => generated 0 bytes in 369 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 08:31:23.295079 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG dbcounter [-] [80244] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80244) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:31:23.312082 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 1032/2061] 10.222.0.47 () {58 vars in 957 bytes} [Thu Jul 21 08:31:23 2022] POST /identity/v3/auth/tokens => generated 3276 bytes in 222 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 08:31:23.316326 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8f759a1-8dcd-4a21-ac84-d207ac453477 admin admin] Authenticating user token {{(pid=80245) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:31:23.317816 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.fernet_utils [None req-d8f759a1-8dcd-4a21-ac84-d207ac453477 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80245) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 08:31:23.492281 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG dbcounter [-] [80245] Writing DB stats keystone:SELECT=85,keystone:DELETE=13,keystone:INSERT=2 {{(pid=80245) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:31:23.499163 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8f759a1-8dcd-4a21-ac84-d207ac453477 admin admin] Validating token access rules against request {{(pid=80245) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:31:23.514528 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03b1795f-b13b-428a-aa62-4ac2f116f2b1 None None] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:31:23.517400 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.fernet_utils [None req-03b1795f-b13b-428a-aa62-4ac2f116f2b1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80244) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 08:31:23.559292 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c7c9936-b78c-4bfa-84a7-ca768e18afd5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '42d15fc0515945acacbbffcf2356c7a5', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80245) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:31:23.560081 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c7c9936-b78c-4bfa-84a7-ca768e18afd5 admin admin] REQUEST_METHOD: `POST` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:31:23.560350 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c7c9936-b78c-4bfa-84a7-ca768e18afd5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:31:23.560635 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c7c9936-b78c-4bfa-84a7-ca768e18afd5 admin admin] PATH_INFO: `/v3/projects` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:31:23.561370 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c7c9936-b78c-4bfa-84a7-ca768e18afd5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:31:23.581861 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c7c9936-b78c-4bfa-84a7-ca768e18afd5 admin admin] RBAC: Authorization granted {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:31:23.668532 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: [pid: 80245|app: 0|req: 1030/2062] 10.222.0.47 () {60 vars in 1152 bytes} [Thu Jul 21 08:31:23 2022] POST /identity/v3/projects => generated 409 bytes in 353 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 08:31:23.672944 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c7c9936-b78c-4bfa-84a7-ca768e18afd5 admin admin] Authenticating user token {{(pid=80245) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:31:23.732680 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03b1795f-b13b-428a-aa62-4ac2f116f2b1 None None] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:31:23.758331 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c7c9936-b78c-4bfa-84a7-ca768e18afd5 admin admin] Validating token access rules against request {{(pid=80245) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:31:23.765619 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c6cb4dc-1035-49cb-958a-715a6e91d3a4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '42d15fc0515945acacbbffcf2356c7a5', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80245) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:31:23.766490 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c6cb4dc-1035-49cb-958a-715a6e91d3a4 admin admin] REQUEST_METHOD: `POST` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:31:23.766853 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c6cb4dc-1035-49cb-958a-715a6e91d3a4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:31:23.770098 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c6cb4dc-1035-49cb-958a-715a6e91d3a4 admin admin] PATH_INFO: `/v3/users` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:31:23.770098 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c6cb4dc-1035-49cb-958a-715a6e91d3a4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:31:23.770971 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c6cb4dc-1035-49cb-958a-715a6e91d3a4 admin admin] RBAC: Authorization granted {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:31:23.808572 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: [pid: 80245|app: 0|req: 1031/2063] 10.222.0.47 () {60 vars in 1143 bytes} [Thu Jul 21 08:31:23 2022] POST /identity/v3/users => generated 311 bytes in 132 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 08:31:23.808572 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c6cb4dc-1035-49cb-958a-715a6e91d3a4 admin admin] Authenticating user token {{(pid=80245) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:31:23.822838 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c6cb4dc-1035-49cb-958a-715a6e91d3a4 admin admin] Validating token access rules against request {{(pid=80245) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:31:23.830869 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8933d36-f6e7-4d18-94b6-c30e62de9f87 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '42d15fc0515945acacbbffcf2356c7a5', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80245) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:31:23.831869 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09a0ceca-b944-469d-b708-d5bb51d3c442 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '42d15fc0515945acacbbffcf2356c7a5', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:31:23.832391 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8933d36-f6e7-4d18-94b6-c30e62de9f87 admin admin] REQUEST_METHOD: `GET` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:31:23.832687 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8933d36-f6e7-4d18-94b6-c30e62de9f87 admin admin] SCRIPT_NAME: `/identity` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:31:23.832971 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8933d36-f6e7-4d18-94b6-c30e62de9f87 admin admin] PATH_INFO: `/v3/roles` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:31:23.834024 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8933d36-f6e7-4d18-94b6-c30e62de9f87 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:31:23.834478 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09a0ceca-b944-469d-b708-d5bb51d3c442 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:31:23.834945 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09a0ceca-b944-469d-b708-d5bb51d3c442 admin admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:31:23.836022 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8933d36-f6e7-4d18-94b6-c30e62de9f87 admin admin] RBAC: Authorization granted {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:31:23.840409 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09a0ceca-b944-469d-b708-d5bb51d3c442 admin admin] PATH_INFO: `/v3/projects/3aabc4c5aa8b4e5698680ec589143be6` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:31:23.842478 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09a0ceca-b944-469d-b708-d5bb51d3c442 admin admin] RBAC: Authorizing `identity:delete_project(project_id=3aabc4c5aa8b4e5698680ec589143be6)` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:31:23.845045 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09a0ceca-b944-469d-b708-d5bb51d3c442 admin admin] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:31:23.857172 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: [pid: 80245|app: 0|req: 1032/2064] 10.222.0.47 () {58 vars in 1121 bytes} [Thu Jul 21 08:31:23 2022] GET /identity/v3/roles => generated 1420 bytes in 49 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 08:31:23.869047 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8933d36-f6e7-4d18-94b6-c30e62de9f87 admin admin] Authenticating user token {{(pid=80245) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:31:23.886599 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8933d36-f6e7-4d18-94b6-c30e62de9f87 admin admin] Validating token access rules against request {{(pid=80245) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:31:23.895197 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c704ab53-878a-4a51-813d-00cd0d746015 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '42d15fc0515945acacbbffcf2356c7a5', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80245) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:31:23.896183 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c704ab53-878a-4a51-813d-00cd0d746015 admin admin] REQUEST_METHOD: `PUT` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:31:23.896472 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c704ab53-878a-4a51-813d-00cd0d746015 admin admin] SCRIPT_NAME: `/identity` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:31:23.897136 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c704ab53-878a-4a51-813d-00cd0d746015 admin admin] PATH_INFO: `/v3/projects/fdf6ddfb5bd744c5ac2d9b12434229eb/users/5c620a3a71e44e89a2d4f0a529a4898b/roles/e29870f6e9e2411a9db0d5bf45f32ca5` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:31:23.932106 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c704ab53-878a-4a51-813d-00cd0d746015 admin admin] RBAC: Authorizing `identity:create_grant(project_id=fdf6ddfb5bd744c5ac2d9b12434229eb, user_id=5c620a3a71e44e89a2d4f0a529a4898b, role_id=e29870f6e9e2411a9db0d5bf45f32ca5)` {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:31:23.940915 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c704ab53-878a-4a51-813d-00cd0d746015 admin admin] RBAC: Authorization granted {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:31:23.943865 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.notifications [None req-09a0ceca-b944-469d-b708-d5bb51d3c442 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3aabc4c5aa8b4e5698680ec589143be6', 'request_id': 'req-09a0ceca-b944-469d-b708-d5bb51d3c442'} {{(pid=80244) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 08:31:23.963613 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.notifications [None req-09a0ceca-b944-469d-b708-d5bb51d3c442 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3aabc4c5aa8b4e5698680ec589143be6', 'request_id': 'req-09a0ceca-b944-469d-b708-d5bb51d3c442'} {{(pid=80244) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 08:31:24.000947 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: [pid: 80245|app: 0|req: 1033/2065] 10.222.0.47 () {60 vars in 1482 bytes} [Thu Jul 21 08:31:23 2022] PUT /identity/v3/projects/fdf6ddfb5bd744c5ac2d9b12434229eb/users/5c620a3a71e44e89a2d4f0a529a4898b/roles/e29870f6e9e2411a9db0d5bf45f32ca5 => generated 0 bytes in 130 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 08:31:24.002799 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c704ab53-878a-4a51-813d-00cd0d746015 admin admin] Authenticating user token {{(pid=80245) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:31:24.008056 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.notifications [None req-09a0ceca-b944-469d-b708-d5bb51d3c442 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3aabc4c5aa8b4e5698680ec589143be6', 'request_id': 'req-09a0ceca-b944-469d-b708-d5bb51d3c442'} {{(pid=80244) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 08:31:24.026039 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.notifications [None req-09a0ceca-b944-469d-b708-d5bb51d3c442 admin admin] The token cache is being invalidate because project 3aabc4c5aa8b4e5698680ec589143be6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80244) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 21 08:31:24.026818 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.notifications [None req-09a0ceca-b944-469d-b708-d5bb51d3c442 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80244) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 08:31:24.028264 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.notifications [None req-09a0ceca-b944-469d-b708-d5bb51d3c442 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80244) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 08:31:24.030173 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 1033/2066] 10.222.0.47 () {58 vars in 1232 bytes} [Thu Jul 21 08:31:23 2022] DELETE /identity/v3/projects/3aabc4c5aa8b4e5698680ec589143be6 => generated 0 bytes in 517 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 08:31:24.030974 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09a0ceca-b944-469d-b708-d5bb51d3c442 admin admin] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:31:24.032330 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.fernet_utils [None req-09a0ceca-b944-469d-b708-d5bb51d3c442 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80244) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 08:31:24.094781 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG dbcounter [-] [80244] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80244) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:31:24.103016 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c704ab53-878a-4a51-813d-00cd0d746015 admin admin] Validating token access rules against request {{(pid=80245) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:31:24.109536 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06ecee22-f5b5-47e6-a072-112d2d71a67d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '42d15fc0515945acacbbffcf2356c7a5', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80245) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:31:24.110477 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06ecee22-f5b5-47e6-a072-112d2d71a67d admin admin] REQUEST_METHOD: `GET` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:31:24.110973 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06ecee22-f5b5-47e6-a072-112d2d71a67d admin admin] SCRIPT_NAME: `/identity` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:31:24.111569 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06ecee22-f5b5-47e6-a072-112d2d71a67d admin admin] PATH_INFO: `/v3/roles` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:31:24.112528 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06ecee22-f5b5-47e6-a072-112d2d71a67d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:31:24.114671 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06ecee22-f5b5-47e6-a072-112d2d71a67d admin admin] RBAC: Authorization granted {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:31:24.128896 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: [pid: 80245|app: 0|req: 1034/2067] 10.222.0.47 () {58 vars in 1121 bytes} [Thu Jul 21 08:31:24 2022] GET /identity/v3/roles => generated 1420 bytes in 127 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 08:31:24.130368 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06ecee22-f5b5-47e6-a072-112d2d71a67d admin admin] Authenticating user token {{(pid=80245) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:31:24.132423 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.fernet_utils [None req-06ecee22-f5b5-47e6-a072-112d2d71a67d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80245) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 08:31:24.154144 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06ecee22-f5b5-47e6-a072-112d2d71a67d admin admin] Validating token access rules against request {{(pid=80245) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:31:24.167354 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-405fc833-0e5d-4b06-960f-e026a0e5b6ae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '42d15fc0515945acacbbffcf2356c7a5', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80245) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:31:24.167354 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-405fc833-0e5d-4b06-960f-e026a0e5b6ae admin admin] REQUEST_METHOD: `DELETE` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:31:24.167354 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-405fc833-0e5d-4b06-960f-e026a0e5b6ae admin admin] SCRIPT_NAME: `/identity` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:31:24.167354 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-405fc833-0e5d-4b06-960f-e026a0e5b6ae admin admin] PATH_INFO: `/v3/users/b60c6a5b83434534ac72bf2d555c5848` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:31:24.173362 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-405fc833-0e5d-4b06-960f-e026a0e5b6ae admin admin] RBAC: Authorizing `identity:delete_user(user_id=b60c6a5b83434534ac72bf2d555c5848)` {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:31:24.176919 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-405fc833-0e5d-4b06-960f-e026a0e5b6ae admin admin] RBAC: Authorization granted {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:31:24.185580 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09a0ceca-b944-469d-b708-d5bb51d3c442 admin admin] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:31:24.197245 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d4e02d2-9fc5-4293-a9f9-d68ae3a67c74 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '42d15fc0515945acacbbffcf2356c7a5', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:31:24.200695 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d4e02d2-9fc5-4293-a9f9-d68ae3a67c74 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:31:24.200929 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d4e02d2-9fc5-4293-a9f9-d68ae3a67c74 admin admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:31:24.201350 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d4e02d2-9fc5-4293-a9f9-d68ae3a67c74 admin admin] PATH_INFO: `/v3/projects/abad576e5d224349988a7fe6d25ff6df` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:31:24.204553 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d4e02d2-9fc5-4293-a9f9-d68ae3a67c74 admin admin] RBAC: Authorizing `identity:delete_project(project_id=abad576e5d224349988a7fe6d25ff6df)` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:31:24.206275 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d4e02d2-9fc5-4293-a9f9-d68ae3a67c74 admin admin] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:31:24.237561 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG dbcounter [-] [80245] Writing DB stats keystone:SELECT=90,keystone:INSERT=5,keystone:DELETE=5 {{(pid=80245) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:31:24.272615 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.notifications [None req-405fc833-0e5d-4b06-960f-e026a0e5b6ae admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b60c6a5b83434534ac72bf2d555c5848'} {{(pid=80245) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 08:31:24.285640 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.notifications [None req-405fc833-0e5d-4b06-960f-e026a0e5b6ae admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b60c6a5b83434534ac72bf2d555c5848'} {{(pid=80245) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 08:31:24.299505 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.notifications [None req-4d4e02d2-9fc5-4293-a9f9-d68ae3a67c74 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'abad576e5d224349988a7fe6d25ff6df', 'request_id': 'req-4d4e02d2-9fc5-4293-a9f9-d68ae3a67c74'} {{(pid=80244) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 08:31:24.308305 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.notifications [None req-405fc833-0e5d-4b06-960f-e026a0e5b6ae admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b60c6a5b83434534ac72bf2d555c5848'} {{(pid=80245) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 08:31:24.310847 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.notifications [None req-4d4e02d2-9fc5-4293-a9f9-d68ae3a67c74 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'abad576e5d224349988a7fe6d25ff6df', 'request_id': 'req-4d4e02d2-9fc5-4293-a9f9-d68ae3a67c74'} {{(pid=80244) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 08:31:24.318864 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.notifications [None req-405fc833-0e5d-4b06-960f-e026a0e5b6ae admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b60c6a5b83434534ac72bf2d555c5848'} {{(pid=80245) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 08:31:24.320003 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.notifications [None req-405fc833-0e5d-4b06-960f-e026a0e5b6ae admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b60c6a5b83434534ac72bf2d555c5848'} {{(pid=80245) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 08:31:24.320977 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.notifications [None req-4d4e02d2-9fc5-4293-a9f9-d68ae3a67c74 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'abad576e5d224349988a7fe6d25ff6df', 'request_id': 'req-4d4e02d2-9fc5-4293-a9f9-d68ae3a67c74'} {{(pid=80244) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 08:31:24.328840 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: [pid: 80245|app: 0|req: 1035/2068] 10.222.0.47 () {58 vars in 1223 bytes} [Thu Jul 21 08:31:24 2022] DELETE /identity/v3/users/b60c6a5b83434534ac72bf2d555c5848 => generated 0 bytes in 199 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 08:31:24.330069 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-405fc833-0e5d-4b06-960f-e026a0e5b6ae admin admin] Authenticating user token {{(pid=80245) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:31:24.335769 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.notifications [None req-4d4e02d2-9fc5-4293-a9f9-d68ae3a67c74 admin admin] The token cache is being invalidate because project abad576e5d224349988a7fe6d25ff6df was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80244) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 21 08:31:24.337081 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.notifications [None req-4d4e02d2-9fc5-4293-a9f9-d68ae3a67c74 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80244) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 08:31:24.337927 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.notifications [None req-4d4e02d2-9fc5-4293-a9f9-d68ae3a67c74 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80244) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 08:31:24.339754 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 1034/2069] 10.222.0.47 () {58 vars in 1232 bytes} [Thu Jul 21 08:31:24 2022] DELETE /identity/v3/projects/abad576e5d224349988a7fe6d25ff6df => generated 0 bytes in 309 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 08:31:24.340311 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.fernet_utils [None req-405fc833-0e5d-4b06-960f-e026a0e5b6ae admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80245) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 08:31:24.352301 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2af5021-a879-4790-a187-b6f9209926c8 None None] REQUEST_METHOD: `POST` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:31:24.352751 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2af5021-a879-4790-a187-b6f9209926c8 None None] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:31:24.354798 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2af5021-a879-4790-a187-b6f9209926c8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:31:24.401069 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.auth.core [None req-b2af5021-a879-4790-a187-b6f9209926c8 None None] MFA Rules not processed for user `2dce40b8d74c41ec94cc36e6ff05d412`. Rule list: `[]` (Enabled: `True`). {{(pid=80244) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 08:31:24.405857 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.fernet_utils [None req-b2af5021-a879-4790-a187-b6f9209926c8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80244) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 08:31:24.420202 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-405fc833-0e5d-4b06-960f-e026a0e5b6ae admin admin] Validating token access rules against request {{(pid=80245) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:31:24.426863 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ac5a63d-254c-4105-be7d-d9693e706162 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '42d15fc0515945acacbbffcf2356c7a5', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80245) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:31:24.427279 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 1035/2070] 10.222.0.47 () {58 vars in 957 bytes} [Thu Jul 21 08:31:24 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 08:31:24.428303 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ac5a63d-254c-4105-be7d-d9693e706162 admin admin] REQUEST_METHOD: `PUT` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:31:24.428638 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ac5a63d-254c-4105-be7d-d9693e706162 admin admin] SCRIPT_NAME: `/identity` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:31:24.428926 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ac5a63d-254c-4105-be7d-d9693e706162 admin admin] PATH_INFO: `/v3/projects/fdf6ddfb5bd744c5ac2d9b12434229eb/users/5c620a3a71e44e89a2d4f0a529a4898b/roles/e29870f6e9e2411a9db0d5bf45f32ca5` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:31:24.431128 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ac5a63d-254c-4105-be7d-d9693e706162 admin admin] RBAC: Authorizing `identity:create_grant(project_id=fdf6ddfb5bd744c5ac2d9b12434229eb, user_id=5c620a3a71e44e89a2d4f0a529a4898b, role_id=e29870f6e9e2411a9db0d5bf45f32ca5)` {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:31:24.433984 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ac5a63d-254c-4105-be7d-d9693e706162 admin admin] RBAC: Authorization granted {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:31:24.438280 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91dd3889-df1d-443c-92fb-4644bcc6d43f None None] REQUEST_METHOD: `POST` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:31:24.438647 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91dd3889-df1d-443c-92fb-4644bcc6d43f None None] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:31:24.443744 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91dd3889-df1d-443c-92fb-4644bcc6d43f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:31:24.458964 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: [pid: 80245|app: 0|req: 1036/2071] 10.222.0.47 () {60 vars in 1482 bytes} [Thu Jul 21 08:31:24 2022] PUT /identity/v3/projects/fdf6ddfb5bd744c5ac2d9b12434229eb/users/5c620a3a71e44e89a2d4f0a529a4898b/roles/e29870f6e9e2411a9db0d5bf45f32ca5 => generated 0 bytes in 120 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 08:31:24.463688 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d8d32ad-ca69-43d2-9d85-102d35db6fbb None None] REQUEST_METHOD: `POST` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:31:24.463977 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d8d32ad-ca69-43d2-9d85-102d35db6fbb None None] SCRIPT_NAME: `/identity` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:31:24.464256 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d8d32ad-ca69-43d2-9d85-102d35db6fbb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:31:24.488892 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.auth.core [None req-91dd3889-df1d-443c-92fb-4644bcc6d43f None None] MFA Rules not processed for user `2dce40b8d74c41ec94cc36e6ff05d412`. Rule list: `[]` (Enabled: `True`). {{(pid=80244) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 08:31:24.490316 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.fernet_utils [None req-91dd3889-df1d-443c-92fb-4644bcc6d43f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80244) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 08:31:24.513478 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.auth.core [None req-5d8d32ad-ca69-43d2-9d85-102d35db6fbb None None] MFA Rules not processed for user `5c620a3a71e44e89a2d4f0a529a4898b`. Rule list: `[]` (Enabled: `True`). {{(pid=80245) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 08:31:24.514848 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.fernet_utils [None req-5d8d32ad-ca69-43d2-9d85-102d35db6fbb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80245) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 08:31:24.634238 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 1036/2072] 10.222.0.47 () {58 vars in 957 bytes} [Thu Jul 21 08:31:24 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 200 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 08:31:24.639701 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91dd3889-df1d-443c-92fb-4644bcc6d43f None None] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:31:24.657910 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91dd3889-df1d-443c-92fb-4644bcc6d43f None None] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:31:24.664968 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: [pid: 80245|app: 0|req: 1037/2073] 10.222.0.47 () {58 vars in 957 bytes} [Thu Jul 21 08:31:24 2022] POST /identity/v3/auth/tokens => generated 3343 bytes in 188 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 08:31:24.672693 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e04dd73-933c-4149-8f83-8f7752beb9d1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '42d15fc0515945acacbbffcf2356c7a5', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:31:24.673577 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e04dd73-933c-4149-8f83-8f7752beb9d1 admin admin] REQUEST_METHOD: `GET` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:31:24.673922 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e04dd73-933c-4149-8f83-8f7752beb9d1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:31:24.674207 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e04dd73-933c-4149-8f83-8f7752beb9d1 admin admin] PATH_INFO: `/v3/domains` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:31:24.675511 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e04dd73-933c-4149-8f83-8f7752beb9d1 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80244) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 08:31:24.675838 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e04dd73-933c-4149-8f83-8f7752beb9d1 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:31:24.677687 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e04dd73-933c-4149-8f83-8f7752beb9d1 admin admin] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:31:24.691432 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d8d32ad-ca69-43d2-9d85-102d35db6fbb None None] Authenticating user token {{(pid=80245) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:31:24.693320 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.fernet_utils [None req-5d8d32ad-ca69-43d2-9d85-102d35db6fbb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80245) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 08:31:24.715512 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 1037/2074] 10.222.0.47 () {58 vars in 1152 bytes} [Thu Jul 21 08:31:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 65 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 21 08:31:24.717218 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e04dd73-933c-4149-8f83-8f7752beb9d1 admin admin] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:31:24.752053 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e04dd73-933c-4149-8f83-8f7752beb9d1 admin admin] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:31:24.766348 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f4bf473-7e0c-4878-942d-2c4c10a2c5b1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '42d15fc0515945acacbbffcf2356c7a5', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:31:24.767619 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f4bf473-7e0c-4878-942d-2c4c10a2c5b1 admin admin] REQUEST_METHOD: `POST` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:31:24.767910 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f4bf473-7e0c-4878-942d-2c4c10a2c5b1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:31:24.768187 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f4bf473-7e0c-4878-942d-2c4c10a2c5b1 admin admin] PATH_INFO: `/v3/projects` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:31:24.769348 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f4bf473-7e0c-4878-942d-2c4c10a2c5b1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:31:24.773884 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f4bf473-7e0c-4878-942d-2c4c10a2c5b1 admin admin] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:31:24.779428 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG dbcounter [-] [80245] Writing DB stats keystone:SELECT=93,keystone:DELETE=5,keystone:INSERT=2 {{(pid=80245) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:31:24.847047 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 1038/2075] 10.222.0.47 () {60 vars in 1152 bytes} [Thu Jul 21 08:31:24 2022] POST /identity/v3/projects => generated 375 bytes in 130 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 08:31:24.851955 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f4bf473-7e0c-4878-942d-2c4c10a2c5b1 admin admin] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:31:24.853375 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d8d32ad-ca69-43d2-9d85-102d35db6fbb None None] Validating token access rules against request {{(pid=80245) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:31:24.922155 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG dbcounter [-] [80244] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80244) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:31:24.994765 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66ab801c-32a2-4204-a898-7a2099bf8673 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4f40410b7674687ab5362ac982248ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4faef13aa50e4e36b6c265fb1398fa14', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80245) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:31:24.995749 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66ab801c-32a2-4204-a898-7a2099bf8673 service cinder] REQUEST_METHOD: `GET` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:31:24.996070 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66ab801c-32a2-4204-a898-7a2099bf8673 service cinder] SCRIPT_NAME: `/identity` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:31:24.996427 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66ab801c-32a2-4204-a898-7a2099bf8673 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:31:25.010579 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f4bf473-7e0c-4878-942d-2c4c10a2c5b1 admin admin] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:31:25.028989 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f966a901-1901-4e3d-8d6a-f100ddf4c09f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '42d15fc0515945acacbbffcf2356c7a5', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:31:25.029888 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f966a901-1901-4e3d-8d6a-f100ddf4c09f admin admin] REQUEST_METHOD: `POST` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:31:25.030270 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f966a901-1901-4e3d-8d6a-f100ddf4c09f admin admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:31:25.030644 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f966a901-1901-4e3d-8d6a-f100ddf4c09f admin admin] PATH_INFO: `/v3/users` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:31:25.031462 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f966a901-1901-4e3d-8d6a-f100ddf4c09f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:31:25.036227 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f966a901-1901-4e3d-8d6a-f100ddf4c09f admin admin] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:31:25.086472 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 1039/2076] 10.222.0.47 () {60 vars in 1143 bytes} [Thu Jul 21 08:31:24 2022] POST /identity/v3/users => generated 288 bytes in 235 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 21 08:31:25.090834 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f966a901-1901-4e3d-8d6a-f100ddf4c09f admin admin] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:31:25.095898 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.fernet_utils [None req-f966a901-1901-4e3d-8d6a-f100ddf4c09f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80244) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 08:31:25.127448 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f966a901-1901-4e3d-8d6a-f100ddf4c09f admin admin] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:31:25.138107 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66ab801c-32a2-4204-a898-7a2099bf8673 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:31:25.140000 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66ab801c-32a2-4204-a898-7a2099bf8673 service cinder] RBAC: Authorization granted {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:31:25.142159 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-116dc4ab-4931-49e6-8b8c-e30d7fa4c3d5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '42d15fc0515945acacbbffcf2356c7a5', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:31:25.143154 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-116dc4ab-4931-49e6-8b8c-e30d7fa4c3d5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:31:25.143534 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-116dc4ab-4931-49e6-8b8c-e30d7fa4c3d5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:31:25.143885 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-116dc4ab-4931-49e6-8b8c-e30d7fa4c3d5 admin admin] PATH_INFO: `/v3/projects/217c0284201f4cf294e98a44d8269e3c` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:31:25.146242 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-116dc4ab-4931-49e6-8b8c-e30d7fa4c3d5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=217c0284201f4cf294e98a44d8269e3c)` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:31:25.150686 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-116dc4ab-4931-49e6-8b8c-e30d7fa4c3d5 admin admin] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:31:25.159986 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: [pid: 80245|app: 0|req: 1038/2077] 10.222.0.47 () {60 vars in 1365 bytes} [Thu Jul 21 08:31:24 2022] GET /identity/v3/auth/tokens => generated 3343 bytes in 467 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 08:31:25.159986 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66ab801c-32a2-4204-a898-7a2099bf8673 service cinder] Authenticating user token {{(pid=80245) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:31:25.175385 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66ab801c-32a2-4204-a898-7a2099bf8673 service cinder] Validating token access rules against request {{(pid=80245) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:31:25.181840 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf789e9b-e08a-4191-bcd6-55417bd2c293 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '42d15fc0515945acacbbffcf2356c7a5', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80245) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:31:25.182623 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf789e9b-e08a-4191-bcd6-55417bd2c293 admin admin] REQUEST_METHOD: `GET` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:31:25.183314 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf789e9b-e08a-4191-bcd6-55417bd2c293 admin admin] SCRIPT_NAME: `/identity` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:31:25.184658 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf789e9b-e08a-4191-bcd6-55417bd2c293 admin admin] PATH_INFO: `/v3/roles` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:31:25.185918 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf789e9b-e08a-4191-bcd6-55417bd2c293 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:31:25.187679 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf789e9b-e08a-4191-bcd6-55417bd2c293 admin admin] RBAC: Authorization granted {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:31:25.203179 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: [pid: 80245|app: 0|req: 1039/2078] 10.222.0.47 () {58 vars in 1121 bytes} [Thu Jul 21 08:31:25 2022] GET /identity/v3/roles => generated 1420 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 08:31:25.209179 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf789e9b-e08a-4191-bcd6-55417bd2c293 admin admin] Authenticating user token {{(pid=80245) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:31:25.238283 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf789e9b-e08a-4191-bcd6-55417bd2c293 admin admin] Validating token access rules against request {{(pid=80245) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:31:25.282347 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.notifications [None req-116dc4ab-4931-49e6-8b8c-e30d7fa4c3d5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '217c0284201f4cf294e98a44d8269e3c', 'request_id': 'req-116dc4ab-4931-49e6-8b8c-e30d7fa4c3d5'} {{(pid=80244) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 08:31:25.295247 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.notifications [None req-116dc4ab-4931-49e6-8b8c-e30d7fa4c3d5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '217c0284201f4cf294e98a44d8269e3c', 'request_id': 'req-116dc4ab-4931-49e6-8b8c-e30d7fa4c3d5'} {{(pid=80244) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 08:31:25.307225 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.notifications [None req-116dc4ab-4931-49e6-8b8c-e30d7fa4c3d5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '217c0284201f4cf294e98a44d8269e3c', 'request_id': 'req-116dc4ab-4931-49e6-8b8c-e30d7fa4c3d5'} {{(pid=80244) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 08:31:25.338935 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.notifications [None req-116dc4ab-4931-49e6-8b8c-e30d7fa4c3d5 admin admin] The token cache is being invalidate because project 217c0284201f4cf294e98a44d8269e3c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80244) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 21 08:31:25.340123 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.notifications [None req-116dc4ab-4931-49e6-8b8c-e30d7fa4c3d5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80244) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 08:31:25.342685 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.notifications [None req-116dc4ab-4931-49e6-8b8c-e30d7fa4c3d5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80244) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 08:31:25.350074 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 1040/2079] 10.222.0.47 () {58 vars in 1232 bytes} [Thu Jul 21 08:31:25 2022] DELETE /identity/v3/projects/217c0284201f4cf294e98a44d8269e3c => generated 0 bytes in 260 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 08:31:25.351620 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01724759-9669-4f54-bc6b-02787e82d6f3 None None] REQUEST_METHOD: `POST` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:31:25.351911 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01724759-9669-4f54-bc6b-02787e82d6f3 None None] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:31:25.352191 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01724759-9669-4f54-bc6b-02787e82d6f3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:31:25.403473 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.auth.core [None req-01724759-9669-4f54-bc6b-02787e82d6f3 None None] MFA Rules not processed for user `2dce40b8d74c41ec94cc36e6ff05d412`. Rule list: `[]` (Enabled: `True`). {{(pid=80244) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 08:31:25.404825 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.fernet_utils [None req-01724759-9669-4f54-bc6b-02787e82d6f3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80244) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 08:31:25.408665 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a78d1188-5858-4f0d-9498-b663f5351f7a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '42d15fc0515945acacbbffcf2356c7a5', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80245) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:31:25.409609 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a78d1188-5858-4f0d-9498-b663f5351f7a admin admin] REQUEST_METHOD: `GET` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:31:25.411132 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a78d1188-5858-4f0d-9498-b663f5351f7a admin admin] SCRIPT_NAME: `/identity` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:31:25.411627 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a78d1188-5858-4f0d-9498-b663f5351f7a admin admin] PATH_INFO: `/v3/roles` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:31:25.420217 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a78d1188-5858-4f0d-9498-b663f5351f7a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:31:25.422156 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a78d1188-5858-4f0d-9498-b663f5351f7a admin admin] RBAC: Authorization granted {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:31:25.428750 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 1041/2080] 10.222.0.47 () {58 vars in 957 bytes} [Thu Jul 21 08:31:25 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 78 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 08:31:25.440173 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: [pid: 80245|app: 0|req: 1040/2081] 10.222.0.47 () {58 vars in 1121 bytes} [Thu Jul 21 08:31:25 2022] GET /identity/v3/roles => generated 1420 bytes in 231 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 08:31:25.441286 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a78d1188-5858-4f0d-9498-b663f5351f7a admin admin] Authenticating user token {{(pid=80245) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:31:25.443088 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.fernet_utils [None req-a78d1188-5858-4f0d-9498-b663f5351f7a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80245) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 08:31:25.443953 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f63e5def-dadf-4335-b5b4-73083f349bb2 None None] REQUEST_METHOD: `POST` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:31:25.445243 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f63e5def-dadf-4335-b5b4-73083f349bb2 None None] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:31:25.446629 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f63e5def-dadf-4335-b5b4-73083f349bb2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:31:25.457784 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a78d1188-5858-4f0d-9498-b663f5351f7a admin admin] Validating token access rules against request {{(pid=80245) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:31:25.463362 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc639227-7875-4b4f-871d-90b65f88f5fd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '42d15fc0515945acacbbffcf2356c7a5', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80245) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:31:25.464214 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc639227-7875-4b4f-871d-90b65f88f5fd admin admin] REQUEST_METHOD: `PUT` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:31:25.464502 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc639227-7875-4b4f-871d-90b65f88f5fd admin admin] SCRIPT_NAME: `/identity` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:31:25.464841 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc639227-7875-4b4f-871d-90b65f88f5fd admin admin] PATH_INFO: `/v3/projects/75b9dc5bbdce44e8acb76c36390f5946/users/c42cb2d145c8453283d3b4a291e3663e/roles/3a074738221e4601b5205715ba87d240` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:31:25.479455 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG dbcounter [-] [80245] Writing DB stats keystone:SELECT=100 {{(pid=80245) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:31:25.485260 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.auth.core [None req-f63e5def-dadf-4335-b5b4-73083f349bb2 None None] MFA Rules not processed for user `2dce40b8d74c41ec94cc36e6ff05d412`. Rule list: `[]` (Enabled: `True`). {{(pid=80244) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 08:31:25.486494 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.fernet_utils [None req-f63e5def-dadf-4335-b5b4-73083f349bb2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80244) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 08:31:25.489316 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc639227-7875-4b4f-871d-90b65f88f5fd admin admin] RBAC: Authorizing `identity:create_grant(project_id=75b9dc5bbdce44e8acb76c36390f5946, user_id=c42cb2d145c8453283d3b4a291e3663e, role_id=3a074738221e4601b5205715ba87d240)` {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:31:25.494270 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc639227-7875-4b4f-871d-90b65f88f5fd admin admin] RBAC: Authorization granted {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:31:25.505714 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 1042/2082] 10.222.0.47 () {58 vars in 957 bytes} [Thu Jul 21 08:31:25 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 68 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 08:31:25.509675 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f63e5def-dadf-4335-b5b4-73083f349bb2 None None] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:31:25.532607 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f63e5def-dadf-4335-b5b4-73083f349bb2 None None] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:31:25.552086 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: [pid: 80245|app: 0|req: 1041/2083] 10.222.0.47 () {60 vars in 1482 bytes} [Thu Jul 21 08:31:25 2022] PUT /identity/v3/projects/75b9dc5bbdce44e8acb76c36390f5946/users/c42cb2d145c8453283d3b4a291e3663e/roles/3a074738221e4601b5205715ba87d240 => generated 0 bytes in 103 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 08:31:25.552086 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc639227-7875-4b4f-871d-90b65f88f5fd admin admin] Authenticating user token {{(pid=80245) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:31:25.556263 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.fernet_utils [None req-cc639227-7875-4b4f-871d-90b65f88f5fd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80245) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 08:31:25.641754 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c49da08f-76ff-443a-87b9-ba797de5ac30 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '42d15fc0515945acacbbffcf2356c7a5', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:31:25.645345 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c49da08f-76ff-443a-87b9-ba797de5ac30 admin admin] REQUEST_METHOD: `GET` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:31:25.646949 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c49da08f-76ff-443a-87b9-ba797de5ac30 admin admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:31:25.648044 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c49da08f-76ff-443a-87b9-ba797de5ac30 admin admin] PATH_INFO: `/v3/domains` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:31:25.650915 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c49da08f-76ff-443a-87b9-ba797de5ac30 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80244) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 08:31:25.652500 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c49da08f-76ff-443a-87b9-ba797de5ac30 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:31:25.654456 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c49da08f-76ff-443a-87b9-ba797de5ac30 admin admin] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:31:25.665618 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc639227-7875-4b4f-871d-90b65f88f5fd admin admin] Validating token access rules against request {{(pid=80245) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:31:25.672990 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83d79862-6c42-42ce-943a-9f2344e03c20 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4f40410b7674687ab5362ac982248ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4faef13aa50e4e36b6c265fb1398fa14', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80245) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:31:25.673668 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83d79862-6c42-42ce-943a-9f2344e03c20 service cinder] REQUEST_METHOD: `GET` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:31:25.673957 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83d79862-6c42-42ce-943a-9f2344e03c20 service cinder] SCRIPT_NAME: `/identity` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:31:25.674238 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83d79862-6c42-42ce-943a-9f2344e03c20 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:31:25.676849 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 1043/2084] 10.222.0.47 () {58 vars in 1152 bytes} [Thu Jul 21 08:31:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 08:31:25.677896 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-749d457a-ef66-4c3f-900b-0c2f40ec7643 None None] REQUEST_METHOD: `POST` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:31:25.678191 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-749d457a-ef66-4c3f-900b-0c2f40ec7643 None None] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:31:25.678608 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-749d457a-ef66-4c3f-900b-0c2f40ec7643 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:31:25.680045 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.fernet_utils [None req-83d79862-6c42-42ce-943a-9f2344e03c20 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80245) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 08:31:25.739150 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.auth.core [None req-749d457a-ef66-4c3f-900b-0c2f40ec7643 None None] MFA Rules not processed for user `c42cb2d145c8453283d3b4a291e3663e`. Rule list: `[]` (Enabled: `True`). {{(pid=80244) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 08:31:25.740535 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.fernet_utils [None req-749d457a-ef66-4c3f-900b-0c2f40ec7643 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80244) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 08:31:25.750116 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG dbcounter [-] [80244] Writing DB stats keystone:SELECT=89,keystone:INSERT=4,keystone:DELETE=7 {{(pid=80244) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:31:25.832976 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83d79862-6c42-42ce-943a-9f2344e03c20 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:31:25.834767 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83d79862-6c42-42ce-943a-9f2344e03c20 service cinder] RBAC: Authorization granted {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:31:25.860136 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 1044/2085] 10.222.0.47 () {58 vars in 957 bytes} [Thu Jul 21 08:31:25 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 183 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 08:31:25.861102 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-749d457a-ef66-4c3f-900b-0c2f40ec7643 None None] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:31:25.865540 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: [pid: 80245|app: 0|req: 1042/2086] 10.222.0.47 () {60 vars in 1365 bytes} [Thu Jul 21 08:31:25 2022] GET /identity/v3/auth/tokens => generated 3276 bytes in 317 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 08:31:25.877725 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-749d457a-ef66-4c3f-900b-0c2f40ec7643 None None] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:31:25.891196 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e50f4fa4-5c45-4ca9-b922-1635222ffa30 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '42d15fc0515945acacbbffcf2356c7a5', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:31:25.891721 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e50f4fa4-5c45-4ca9-b922-1635222ffa30 admin admin] REQUEST_METHOD: `POST` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:31:25.892099 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e50f4fa4-5c45-4ca9-b922-1635222ffa30 admin admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:31:25.893782 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e50f4fa4-5c45-4ca9-b922-1635222ffa30 admin admin] PATH_INFO: `/v3/projects` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:31:25.895229 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e50f4fa4-5c45-4ca9-b922-1635222ffa30 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:31:25.898515 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e50f4fa4-5c45-4ca9-b922-1635222ffa30 admin admin] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:31:25.916450 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83d79862-6c42-42ce-943a-9f2344e03c20 service cinder] Authenticating user token {{(pid=80245) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:31:25.921934 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.fernet_utils [None req-83d79862-6c42-42ce-943a-9f2344e03c20 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80245) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 08:31:25.934204 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83d79862-6c42-42ce-943a-9f2344e03c20 service cinder] Validating token access rules against request {{(pid=80245) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:31:25.938792 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-807a6ebf-8979-4592-a249-7fe7b74e8e71 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4f40410b7674687ab5362ac982248ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4faef13aa50e4e36b6c265fb1398fa14', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80245) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:31:25.939704 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-807a6ebf-8979-4592-a249-7fe7b74e8e71 service cinder] REQUEST_METHOD: `GET` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:31:25.939993 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-807a6ebf-8979-4592-a249-7fe7b74e8e71 service cinder] SCRIPT_NAME: `/identity` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:31:25.940276 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-807a6ebf-8979-4592-a249-7fe7b74e8e71 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:31:25.958385 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-807a6ebf-8979-4592-a249-7fe7b74e8e71 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:31:25.960030 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-807a6ebf-8979-4592-a249-7fe7b74e8e71 service cinder] RBAC: Authorization granted {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:31:25.962909 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 1045/2087] 10.222.0.47 () {60 vars in 1152 bytes} [Thu Jul 21 08:31:25 2022] POST /identity/v3/projects => generated 379 bytes in 102 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 08:31:25.968490 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e50f4fa4-5c45-4ca9-b922-1635222ffa30 admin admin] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:31:25.989716 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: [pid: 80245|app: 0|req: 1043/2088] 10.222.0.47 () {60 vars in 1365 bytes} [Thu Jul 21 08:31:25 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 75 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 21 08:31:26.063359 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e50f4fa4-5c45-4ca9-b922-1635222ffa30 admin admin] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:31:26.070220 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a12cd7f6-3bbd-464a-80a7-66b0e00e3d3b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '42d15fc0515945acacbbffcf2356c7a5', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:31:26.073607 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a12cd7f6-3bbd-464a-80a7-66b0e00e3d3b admin admin] REQUEST_METHOD: `POST` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:31:26.073859 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a12cd7f6-3bbd-464a-80a7-66b0e00e3d3b admin admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:31:26.074189 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a12cd7f6-3bbd-464a-80a7-66b0e00e3d3b admin admin] PATH_INFO: `/v3/users` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:31:26.074919 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a12cd7f6-3bbd-464a-80a7-66b0e00e3d3b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:31:26.080322 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a12cd7f6-3bbd-464a-80a7-66b0e00e3d3b admin admin] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:31:26.143832 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 1046/2089] 10.222.0.47 () {60 vars in 1143 bytes} [Thu Jul 21 08:31:25 2022] POST /identity/v3/users => generated 290 bytes in 173 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 08:31:26.145575 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-807a6ebf-8979-4592-a249-7fe7b74e8e71 service cinder] Authenticating user token {{(pid=80245) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:31:26.164235 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-807a6ebf-8979-4592-a249-7fe7b74e8e71 service cinder] Validating token access rules against request {{(pid=80245) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:31:26.176038 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a592f595-4aa1-4c3b-bd05-27eb3e904758 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '42d15fc0515945acacbbffcf2356c7a5', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80245) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:31:26.177706 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a592f595-4aa1-4c3b-bd05-27eb3e904758 admin admin] REQUEST_METHOD: `GET` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:31:26.177997 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a592f595-4aa1-4c3b-bd05-27eb3e904758 admin admin] SCRIPT_NAME: `/identity` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:31:26.180474 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a592f595-4aa1-4c3b-bd05-27eb3e904758 admin admin] PATH_INFO: `/v3/roles` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:31:26.181367 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a592f595-4aa1-4c3b-bd05-27eb3e904758 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:31:26.186275 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a592f595-4aa1-4c3b-bd05-27eb3e904758 admin admin] RBAC: Authorization granted {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:31:26.219677 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: [pid: 80245|app: 0|req: 1044/2090] 10.222.0.47 () {58 vars in 1121 bytes} [Thu Jul 21 08:31:26 2022] GET /identity/v3/roles => generated 1420 bytes in 75 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 08:31:26.224432 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a12cd7f6-3bbd-464a-80a7-66b0e00e3d3b admin admin] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:31:26.242191 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a12cd7f6-3bbd-464a-80a7-66b0e00e3d3b admin admin] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:31:26.247639 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a258ba72-4071-4756-ada1-2f089aa8301a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '42d15fc0515945acacbbffcf2356c7a5', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:31:26.248369 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a258ba72-4071-4756-ada1-2f089aa8301a admin admin] REQUEST_METHOD: `GET` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:31:26.248650 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a258ba72-4071-4756-ada1-2f089aa8301a admin admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:31:26.249099 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a258ba72-4071-4756-ada1-2f089aa8301a admin admin] PATH_INFO: `/v3/roles` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:31:26.249731 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a258ba72-4071-4756-ada1-2f089aa8301a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:31:26.251370 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a258ba72-4071-4756-ada1-2f089aa8301a admin admin] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:31:26.276340 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 1047/2091] 10.222.0.47 () {58 vars in 1121 bytes} [Thu Jul 21 08:31:26 2022] GET /identity/v3/roles => generated 1420 bytes in 53 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 08:31:26.280951 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a592f595-4aa1-4c3b-bd05-27eb3e904758 admin admin] Authenticating user token {{(pid=80245) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:31:26.303985 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a592f595-4aa1-4c3b-bd05-27eb3e904758 admin admin] Validating token access rules against request {{(pid=80245) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:31:26.317128 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3cc2d90-afbb-47a7-b144-74ab24bbe4c0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '42d15fc0515945acacbbffcf2356c7a5', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80245) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:31:26.319790 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3cc2d90-afbb-47a7-b144-74ab24bbe4c0 admin admin] REQUEST_METHOD: `PUT` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:31:26.320301 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3cc2d90-afbb-47a7-b144-74ab24bbe4c0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:31:26.321556 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3cc2d90-afbb-47a7-b144-74ab24bbe4c0 admin admin] PATH_INFO: `/v3/projects/084f5750d01041f0b58b919e4b254af0/users/95684c4a7fc74ee5a92ac87fc5053435/roles/3a074738221e4601b5205715ba87d240` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:31:26.346430 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3cc2d90-afbb-47a7-b144-74ab24bbe4c0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=084f5750d01041f0b58b919e4b254af0, user_id=95684c4a7fc74ee5a92ac87fc5053435, role_id=3a074738221e4601b5205715ba87d240)` {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:31:26.350644 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3cc2d90-afbb-47a7-b144-74ab24bbe4c0 admin admin] RBAC: Authorization granted {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:31:26.382138 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: [pid: 80245|app: 0|req: 1045/2092] 10.222.0.47 () {60 vars in 1482 bytes} [Thu Jul 21 08:31:26 2022] PUT /identity/v3/projects/084f5750d01041f0b58b919e4b254af0/users/95684c4a7fc74ee5a92ac87fc5053435/roles/3a074738221e4601b5205715ba87d240 => generated 0 bytes in 102 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 08:31:26.392599 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dea1ecd2-f479-421e-98fb-cd5752ebbe8b None None] REQUEST_METHOD: `POST` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:31:26.393147 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dea1ecd2-f479-421e-98fb-cd5752ebbe8b None None] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:31:26.393147 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dea1ecd2-f479-421e-98fb-cd5752ebbe8b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:31:26.451865 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.auth.core [None req-dea1ecd2-f479-421e-98fb-cd5752ebbe8b None None] MFA Rules not processed for user `95684c4a7fc74ee5a92ac87fc5053435`. Rule list: `[]` (Enabled: `True`). {{(pid=80244) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 08:31:26.453253 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.fernet_utils [None req-dea1ecd2-f479-421e-98fb-cd5752ebbe8b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80244) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 08:31:26.560473 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG dbcounter [-] [80244] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80244) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:31:26.614733 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 1048/2093] 10.222.0.47 () {58 vars in 957 bytes} [Thu Jul 21 08:31:26 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 223 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 08:31:26.619852 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3cc2d90-afbb-47a7-b144-74ab24bbe4c0 admin admin] Authenticating user token {{(pid=80245) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:31:26.696870 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG dbcounter [-] [80245] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80245) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:31:26.701067 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3cc2d90-afbb-47a7-b144-74ab24bbe4c0 admin admin] Validating token access rules against request {{(pid=80245) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:31:26.708727 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0038d153-0e05-45d7-8e4c-0739b5b7bcbe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '42d15fc0515945acacbbffcf2356c7a5', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80245) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:31:26.709351 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0038d153-0e05-45d7-8e4c-0739b5b7bcbe admin admin] REQUEST_METHOD: `POST` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:31:26.710256 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0038d153-0e05-45d7-8e4c-0739b5b7bcbe admin admin] SCRIPT_NAME: `/identity` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:31:26.710995 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0038d153-0e05-45d7-8e4c-0739b5b7bcbe admin admin] PATH_INFO: `/v3/projects` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:31:26.712358 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0038d153-0e05-45d7-8e4c-0739b5b7bcbe admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:31:26.715306 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0038d153-0e05-45d7-8e4c-0739b5b7bcbe admin admin] RBAC: Authorization granted {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:31:26.747539 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: [pid: 80245|app: 0|req: 1046/2094] 10.222.0.47 () {60 vars in 1152 bytes} [Thu Jul 21 08:31:26 2022] POST /identity/v3/projects => generated 381 bytes in 128 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 08:31:26.748500 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dea1ecd2-f479-421e-98fb-cd5752ebbe8b None None] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:31:26.760156 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dea1ecd2-f479-421e-98fb-cd5752ebbe8b None None] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:31:26.827243 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc3bc34a-6914-4094-8ea6-43d096037def admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '42d15fc0515945acacbbffcf2356c7a5', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:31:26.828235 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc3bc34a-6914-4094-8ea6-43d096037def admin admin] REQUEST_METHOD: `POST` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:31:26.828519 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc3bc34a-6914-4094-8ea6-43d096037def admin admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:31:26.828836 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc3bc34a-6914-4094-8ea6-43d096037def admin admin] PATH_INFO: `/v3/users` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:31:26.829533 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc3bc34a-6914-4094-8ea6-43d096037def admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:31:26.831493 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc3bc34a-6914-4094-8ea6-43d096037def admin admin] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:31:26.852988 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 1049/2095] 10.222.0.47 () {60 vars in 1143 bytes} [Thu Jul 21 08:31:26 2022] POST /identity/v3/users => generated 297 bytes in 105 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 21 08:31:26.866668 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0038d153-0e05-45d7-8e4c-0739b5b7bcbe admin admin] Authenticating user token {{(pid=80245) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:31:26.870752 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0038d153-0e05-45d7-8e4c-0739b5b7bcbe admin admin] Validating token access rules against request {{(pid=80245) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:31:26.875763 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f4c261e-6b06-43cf-ab3c-59016e10a0c0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '42d15fc0515945acacbbffcf2356c7a5', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80245) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:31:26.877062 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f4c261e-6b06-43cf-ab3c-59016e10a0c0 admin admin] REQUEST_METHOD: `GET` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:31:26.877062 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f4c261e-6b06-43cf-ab3c-59016e10a0c0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:31:26.877062 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f4c261e-6b06-43cf-ab3c-59016e10a0c0 admin admin] PATH_INFO: `/v3/roles` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:31:26.877732 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f4c261e-6b06-43cf-ab3c-59016e10a0c0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:31:26.879656 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f4c261e-6b06-43cf-ab3c-59016e10a0c0 admin admin] RBAC: Authorization granted {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:31:26.891662 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: [pid: 80245|app: 0|req: 1047/2096] 10.222.0.47 () {58 vars in 1121 bytes} [Thu Jul 21 08:31:26 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 08:31:26.896099 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc3bc34a-6914-4094-8ea6-43d096037def admin admin] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:31:26.907123 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc3bc34a-6914-4094-8ea6-43d096037def admin admin] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:31:26.911964 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c30d082-542d-480c-a8c4-0907e36b05a4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '42d15fc0515945acacbbffcf2356c7a5', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:31:26.912846 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c30d082-542d-480c-a8c4-0907e36b05a4 admin admin] REQUEST_METHOD: `PUT` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:31:26.913128 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c30d082-542d-480c-a8c4-0907e36b05a4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:31:26.913404 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c30d082-542d-480c-a8c4-0907e36b05a4 admin admin] PATH_INFO: `/v3/projects/3bd551cb148e48aca56ead1aa7b6885a/users/78ad0a20074946b082ea02bddca54f04/roles/e29870f6e9e2411a9db0d5bf45f32ca5` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:31:26.930552 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c30d082-542d-480c-a8c4-0907e36b05a4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3bd551cb148e48aca56ead1aa7b6885a, user_id=78ad0a20074946b082ea02bddca54f04, role_id=e29870f6e9e2411a9db0d5bf45f32ca5)` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:31:26.933764 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c30d082-542d-480c-a8c4-0907e36b05a4 admin admin] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:31:26.946678 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 1050/2097] 10.222.0.47 () {60 vars in 1482 bytes} [Thu Jul 21 08:31:26 2022] PUT /identity/v3/projects/3bd551cb148e48aca56ead1aa7b6885a/users/78ad0a20074946b082ea02bddca54f04/roles/e29870f6e9e2411a9db0d5bf45f32ca5 => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 08:31:26.951261 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f4c261e-6b06-43cf-ab3c-59016e10a0c0 admin admin] Authenticating user token {{(pid=80245) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:31:26.961448 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f4c261e-6b06-43cf-ab3c-59016e10a0c0 admin admin] Validating token access rules against request {{(pid=80245) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:31:27.021373 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-483e5524-fdf2-4416-be82-88f7a9a69b26 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '42d15fc0515945acacbbffcf2356c7a5', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80245) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:31:27.022050 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-483e5524-fdf2-4416-be82-88f7a9a69b26 admin admin] REQUEST_METHOD: `GET` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:31:27.022686 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-483e5524-fdf2-4416-be82-88f7a9a69b26 admin admin] SCRIPT_NAME: `/identity` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:31:27.022686 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-483e5524-fdf2-4416-be82-88f7a9a69b26 admin admin] PATH_INFO: `/v3/roles` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:31:27.023373 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-483e5524-fdf2-4416-be82-88f7a9a69b26 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:31:27.024977 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-483e5524-fdf2-4416-be82-88f7a9a69b26 admin admin] RBAC: Authorization granted {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:31:27.036691 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: [pid: 80245|app: 0|req: 1048/2098] 10.222.0.47 () {58 vars in 1121 bytes} [Thu Jul 21 08:31:26 2022] GET /identity/v3/roles => generated 1420 bytes in 86 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 08:31:27.040916 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c30d082-542d-480c-a8c4-0907e36b05a4 admin admin] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:31:27.051370 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c30d082-542d-480c-a8c4-0907e36b05a4 admin admin] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:31:27.055914 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dcccbd4-7740-46f5-91ee-2d86d7034d21 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '42d15fc0515945acacbbffcf2356c7a5', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:31:27.056798 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dcccbd4-7740-46f5-91ee-2d86d7034d21 admin admin] REQUEST_METHOD: `PUT` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:31:27.057075 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dcccbd4-7740-46f5-91ee-2d86d7034d21 admin admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:31:27.057362 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dcccbd4-7740-46f5-91ee-2d86d7034d21 admin admin] PATH_INFO: `/v3/projects/3bd551cb148e48aca56ead1aa7b6885a/users/78ad0a20074946b082ea02bddca54f04/roles/e29870f6e9e2411a9db0d5bf45f32ca5` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:31:27.059390 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dcccbd4-7740-46f5-91ee-2d86d7034d21 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3bd551cb148e48aca56ead1aa7b6885a, user_id=78ad0a20074946b082ea02bddca54f04, role_id=e29870f6e9e2411a9db0d5bf45f32ca5)` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:31:27.062017 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dcccbd4-7740-46f5-91ee-2d86d7034d21 admin admin] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:31:27.072594 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 1051/2099] 10.222.0.47 () {60 vars in 1482 bytes} [Thu Jul 21 08:31:27 2022] PUT /identity/v3/projects/3bd551cb148e48aca56ead1aa7b6885a/users/78ad0a20074946b082ea02bddca54f04/roles/e29870f6e9e2411a9db0d5bf45f32ca5 => generated 0 bytes in 32 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 08:31:27.083543 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1c7556d-bcf8-4e2b-96a5-bc000a86a174 None None] REQUEST_METHOD: `POST` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:31:27.083846 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1c7556d-bcf8-4e2b-96a5-bc000a86a174 None None] SCRIPT_NAME: `/identity` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:31:27.084131 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1c7556d-bcf8-4e2b-96a5-bc000a86a174 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:31:27.112492 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.auth.core [None req-b1c7556d-bcf8-4e2b-96a5-bc000a86a174 None None] MFA Rules not processed for user `78ad0a20074946b082ea02bddca54f04`. Rule list: `[]` (Enabled: `True`). {{(pid=80245) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 08:31:27.113727 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.fernet_utils [None req-b1c7556d-bcf8-4e2b-96a5-bc000a86a174 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80245) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 08:31:27.199916 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: [pid: 80245|app: 0|req: 1049/2100] 10.222.0.47 () {58 vars in 957 bytes} [Thu Jul 21 08:31:27 2022] POST /identity/v3/auth/tokens => generated 3315 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 08:31:27.213333 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dcccbd4-7740-46f5-91ee-2d86d7034d21 admin admin] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:31:27.276585 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dcccbd4-7740-46f5-91ee-2d86d7034d21 admin admin] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:31:27.281102 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a0182ce-f4b6-4d4e-b9ed-8be7ede443d4 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4f40410b7674687ab5362ac982248ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4faef13aa50e4e36b6c265fb1398fa14', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:31:27.281776 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a0182ce-f4b6-4d4e-b9ed-8be7ede443d4 service cinder] REQUEST_METHOD: `GET` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:31:27.282068 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a0182ce-f4b6-4d4e-b9ed-8be7ede443d4 service cinder] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:31:27.282350 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a0182ce-f4b6-4d4e-b9ed-8be7ede443d4 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:31:27.292676 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a0182ce-f4b6-4d4e-b9ed-8be7ede443d4 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:31:27.294171 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a0182ce-f4b6-4d4e-b9ed-8be7ede443d4 service cinder] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:31:27.304688 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 1052/2101] 10.222.0.47 () {60 vars in 1365 bytes} [Thu Jul 21 08:31:27 2022] GET /identity/v3/auth/tokens => generated 3315 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 08:31:27.711958 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1c7556d-bcf8-4e2b-96a5-bc000a86a174 None None] Authenticating user token {{(pid=80245) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:31:27.726661 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1c7556d-bcf8-4e2b-96a5-bc000a86a174 None None] Validating token access rules against request {{(pid=80245) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:31:27.740677 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71a92a03-3c3a-41fe-bfad-268139e8253e tempest-VolumeQuotaClassesTest-1797270569 tempest-VolumeQuotaClassesTest-1797270569-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78ad0a20074946b082ea02bddca54f04', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3bd551cb148e48aca56ead1aa7b6885a', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80245) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:31:27.741477 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71a92a03-3c3a-41fe-bfad-268139e8253e tempest-VolumeQuotaClassesTest-1797270569 tempest-VolumeQuotaClassesTest-1797270569-project-admin] REQUEST_METHOD: `GET` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:31:27.741990 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71a92a03-3c3a-41fe-bfad-268139e8253e tempest-VolumeQuotaClassesTest-1797270569 tempest-VolumeQuotaClassesTest-1797270569-project-admin] SCRIPT_NAME: `/identity` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:31:27.742201 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71a92a03-3c3a-41fe-bfad-268139e8253e tempest-VolumeQuotaClassesTest-1797270569 tempest-VolumeQuotaClassesTest-1797270569-project-admin] PATH_INFO: `/v3/domains` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:31:27.742961 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71a92a03-3c3a-41fe-bfad-268139e8253e tempest-VolumeQuotaClassesTest-1797270569 tempest-VolumeQuotaClassesTest-1797270569-project-admin] RBAC: Adding query filter params (name=Default) {{(pid=80245) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 08:31:27.743274 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71a92a03-3c3a-41fe-bfad-268139e8253e tempest-VolumeQuotaClassesTest-1797270569 tempest-VolumeQuotaClassesTest-1797270569-project-admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:31:27.745019 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71a92a03-3c3a-41fe-bfad-268139e8253e tempest-VolumeQuotaClassesTest-1797270569 tempest-VolumeQuotaClassesTest-1797270569-project-admin] RBAC: Authorization granted {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:31:27.762988 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: [pid: 80245|app: 0|req: 1050/2102] 10.222.0.47 () {58 vars in 1152 bytes} [Thu Jul 21 08:31:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 49 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 08:31:27.764549 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a0182ce-f4b6-4d4e-b9ed-8be7ede443d4 service cinder] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:31:27.776095 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a0182ce-f4b6-4d4e-b9ed-8be7ede443d4 service cinder] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:31:27.780708 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d526cc6-468a-4f1e-a7ba-1b7fc46bb508 tempest-VolumeQuotaClassesTest-1797270569 tempest-VolumeQuotaClassesTest-1797270569-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78ad0a20074946b082ea02bddca54f04', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3bd551cb148e48aca56ead1aa7b6885a', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:31:27.781383 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d526cc6-468a-4f1e-a7ba-1b7fc46bb508 tempest-VolumeQuotaClassesTest-1797270569 tempest-VolumeQuotaClassesTest-1797270569-project-admin] REQUEST_METHOD: `POST` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:31:27.781669 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d526cc6-468a-4f1e-a7ba-1b7fc46bb508 tempest-VolumeQuotaClassesTest-1797270569 tempest-VolumeQuotaClassesTest-1797270569-project-admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:31:27.781949 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d526cc6-468a-4f1e-a7ba-1b7fc46bb508 tempest-VolumeQuotaClassesTest-1797270569 tempest-VolumeQuotaClassesTest-1797270569-project-admin] PATH_INFO: `/v3/projects` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:31:27.782644 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d526cc6-468a-4f1e-a7ba-1b7fc46bb508 tempest-VolumeQuotaClassesTest-1797270569 tempest-VolumeQuotaClassesTest-1797270569-project-admin] RBAC: Authorizing `identity:create_project()` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:31:27.784617 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d526cc6-468a-4f1e-a7ba-1b7fc46bb508 tempest-VolumeQuotaClassesTest-1797270569 tempest-VolumeQuotaClassesTest-1797270569-project-admin] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:31:27.813033 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 1053/2103] 10.222.0.47 () {60 vars in 1152 bytes} [Thu Jul 21 08:31:27 2022] POST /identity/v3/projects => generated 353 bytes in 49 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 08:31:27.817236 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71a92a03-3c3a-41fe-bfad-268139e8253e tempest-VolumeQuotaClassesTest-1797270569 tempest-VolumeQuotaClassesTest-1797270569-project-admin] Authenticating user token {{(pid=80245) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:31:27.827876 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71a92a03-3c3a-41fe-bfad-268139e8253e tempest-VolumeQuotaClassesTest-1797270569 tempest-VolumeQuotaClassesTest-1797270569-project-admin] Validating token access rules against request {{(pid=80245) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:31:27.866564 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG dbcounter [-] [80245] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80245) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:31:27.893835 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1900b652-59a6-46da-879d-87f33b3bdda3 tempest-VolumeQuotaClassesTest-1797270569 tempest-VolumeQuotaClassesTest-1797270569-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78ad0a20074946b082ea02bddca54f04', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3bd551cb148e48aca56ead1aa7b6885a', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80245) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:31:27.894528 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1900b652-59a6-46da-879d-87f33b3bdda3 tempest-VolumeQuotaClassesTest-1797270569 tempest-VolumeQuotaClassesTest-1797270569-project-admin] REQUEST_METHOD: `GET` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:31:27.894842 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1900b652-59a6-46da-879d-87f33b3bdda3 tempest-VolumeQuotaClassesTest-1797270569 tempest-VolumeQuotaClassesTest-1797270569-project-admin] SCRIPT_NAME: `/identity` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:31:27.895127 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1900b652-59a6-46da-879d-87f33b3bdda3 tempest-VolumeQuotaClassesTest-1797270569 tempest-VolumeQuotaClassesTest-1797270569-project-admin] PATH_INFO: `/v3/domains` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:31:27.895899 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1900b652-59a6-46da-879d-87f33b3bdda3 tempest-VolumeQuotaClassesTest-1797270569 tempest-VolumeQuotaClassesTest-1797270569-project-admin] RBAC: Adding query filter params (name=Default) {{(pid=80245) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 08:31:27.896216 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1900b652-59a6-46da-879d-87f33b3bdda3 tempest-VolumeQuotaClassesTest-1797270569 tempest-VolumeQuotaClassesTest-1797270569-project-admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:31:27.897868 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1900b652-59a6-46da-879d-87f33b3bdda3 tempest-VolumeQuotaClassesTest-1797270569 tempest-VolumeQuotaClassesTest-1797270569-project-admin] RBAC: Authorization granted {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:31:27.913820 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: [pid: 80245|app: 0|req: 1051/2104] 10.222.0.47 () {58 vars in 1152 bytes} [Thu Jul 21 08:31:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 97 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 08:31:27.966770 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d526cc6-468a-4f1e-a7ba-1b7fc46bb508 tempest-VolumeQuotaClassesTest-1797270569 tempest-VolumeQuotaClassesTest-1797270569-project-admin] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:31:27.977440 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d526cc6-468a-4f1e-a7ba-1b7fc46bb508 tempest-VolumeQuotaClassesTest-1797270569 tempest-VolumeQuotaClassesTest-1797270569-project-admin] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:31:27.984510 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96e51fa5-0aa8-42da-8185-aea31333c269 tempest-VolumeQuotaClassesTest-1797270569 tempest-VolumeQuotaClassesTest-1797270569-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '78ad0a20074946b082ea02bddca54f04', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3bd551cb148e48aca56ead1aa7b6885a', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:31:27.985686 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96e51fa5-0aa8-42da-8185-aea31333c269 tempest-VolumeQuotaClassesTest-1797270569 tempest-VolumeQuotaClassesTest-1797270569-project-admin] REQUEST_METHOD: `DELETE` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:31:27.985983 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96e51fa5-0aa8-42da-8185-aea31333c269 tempest-VolumeQuotaClassesTest-1797270569 tempest-VolumeQuotaClassesTest-1797270569-project-admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:31:27.986331 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96e51fa5-0aa8-42da-8185-aea31333c269 tempest-VolumeQuotaClassesTest-1797270569 tempest-VolumeQuotaClassesTest-1797270569-project-admin] PATH_INFO: `/v3/projects/a939de32a603424bbb74ff228a215786` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:31:27.987611 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96e51fa5-0aa8-42da-8185-aea31333c269 tempest-VolumeQuotaClassesTest-1797270569 tempest-VolumeQuotaClassesTest-1797270569-project-admin] RBAC: Authorizing `identity:delete_project(project_id=a939de32a603424bbb74ff228a215786)` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:31:27.990295 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96e51fa5-0aa8-42da-8185-aea31333c269 tempest-VolumeQuotaClassesTest-1797270569 tempest-VolumeQuotaClassesTest-1797270569-project-admin] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:31:28.015074 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG dbcounter [-] [80244] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=80244) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:31:28.045129 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.notifications [None req-96e51fa5-0aa8-42da-8185-aea31333c269 tempest-VolumeQuotaClassesTest-1797270569 tempest-VolumeQuotaClassesTest-1797270569-project-admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a939de32a603424bbb74ff228a215786', 'request_id': 'req-96e51fa5-0aa8-42da-8185-aea31333c269'} {{(pid=80244) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 08:31:28.052327 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.notifications [None req-96e51fa5-0aa8-42da-8185-aea31333c269 tempest-VolumeQuotaClassesTest-1797270569 tempest-VolumeQuotaClassesTest-1797270569-project-admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a939de32a603424bbb74ff228a215786', 'request_id': 'req-96e51fa5-0aa8-42da-8185-aea31333c269'} {{(pid=80244) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 08:31:28.059087 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.notifications [None req-96e51fa5-0aa8-42da-8185-aea31333c269 tempest-VolumeQuotaClassesTest-1797270569 tempest-VolumeQuotaClassesTest-1797270569-project-admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a939de32a603424bbb74ff228a215786', 'request_id': 'req-96e51fa5-0aa8-42da-8185-aea31333c269'} {{(pid=80244) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 08:31:28.069287 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.notifications [None req-96e51fa5-0aa8-42da-8185-aea31333c269 tempest-VolumeQuotaClassesTest-1797270569 tempest-VolumeQuotaClassesTest-1797270569-project-admin] The token cache is being invalidate because project a939de32a603424bbb74ff228a215786 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80244) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 21 08:31:28.069569 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.notifications [None req-96e51fa5-0aa8-42da-8185-aea31333c269 tempest-VolumeQuotaClassesTest-1797270569 tempest-VolumeQuotaClassesTest-1797270569-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80244) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 08:31:28.070204 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.notifications [None req-96e51fa5-0aa8-42da-8185-aea31333c269 tempest-VolumeQuotaClassesTest-1797270569 tempest-VolumeQuotaClassesTest-1797270569-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80244) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 08:31:28.071794 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 1054/2105] 10.222.0.47 () {58 vars in 1232 bytes} [Thu Jul 21 08:31:27 2022] DELETE /identity/v3/projects/a939de32a603424bbb74ff228a215786 => generated 0 bytes in 106 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 08:31:28.257205 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1900b652-59a6-46da-879d-87f33b3bdda3 tempest-VolumeQuotaClassesTest-1797270569 tempest-VolumeQuotaClassesTest-1797270569-project-admin] Authenticating user token {{(pid=80245) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:31:28.259042 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.fernet_utils [None req-1900b652-59a6-46da-879d-87f33b3bdda3 tempest-VolumeQuotaClassesTest-1797270569 tempest-VolumeQuotaClassesTest-1797270569-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80245) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 08:31:28.323737 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1900b652-59a6-46da-879d-87f33b3bdda3 tempest-VolumeQuotaClassesTest-1797270569 tempest-VolumeQuotaClassesTest-1797270569-project-admin] Validating token access rules against request {{(pid=80245) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:31:28.384688 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed889ced-d850-495c-a17f-214a6f8ac8c3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '42d15fc0515945acacbbffcf2356c7a5', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80245) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:31:28.385969 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed889ced-d850-495c-a17f-214a6f8ac8c3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:31:28.386436 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed889ced-d850-495c-a17f-214a6f8ac8c3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:31:28.387669 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed889ced-d850-495c-a17f-214a6f8ac8c3 admin admin] PATH_INFO: `/v3/users/95684c4a7fc74ee5a92ac87fc5053435` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:31:28.389181 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed889ced-d850-495c-a17f-214a6f8ac8c3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=95684c4a7fc74ee5a92ac87fc5053435)` {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:31:28.391495 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed889ced-d850-495c-a17f-214a6f8ac8c3 admin admin] RBAC: Authorization granted {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:31:28.440199 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.notifications [None req-ed889ced-d850-495c-a17f-214a6f8ac8c3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '95684c4a7fc74ee5a92ac87fc5053435'} {{(pid=80245) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 08:31:28.449478 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.notifications [None req-ed889ced-d850-495c-a17f-214a6f8ac8c3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '95684c4a7fc74ee5a92ac87fc5053435'} {{(pid=80245) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 08:31:28.463567 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.notifications [None req-ed889ced-d850-495c-a17f-214a6f8ac8c3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '95684c4a7fc74ee5a92ac87fc5053435'} {{(pid=80245) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 08:31:28.471071 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.notifications [None req-ed889ced-d850-495c-a17f-214a6f8ac8c3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '95684c4a7fc74ee5a92ac87fc5053435'} {{(pid=80245) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 08:31:28.471839 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.notifications [None req-ed889ced-d850-495c-a17f-214a6f8ac8c3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '95684c4a7fc74ee5a92ac87fc5053435'} {{(pid=80245) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 08:31:28.474104 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG dbcounter [-] [80245] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80245) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:31:28.478637 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: [pid: 80245|app: 0|req: 1052/2106] 10.222.0.47 () {58 vars in 1223 bytes} [Thu Jul 21 08:31:28 2022] DELETE /identity/v3/users/95684c4a7fc74ee5a92ac87fc5053435 => generated 0 bytes in 222 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 08:31:28.483988 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e8643d4-6a32-4da4-adb2-172fe47af3a5 None None] REQUEST_METHOD: `POST` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:31:28.484284 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e8643d4-6a32-4da4-adb2-172fe47af3a5 None None] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:31:28.484623 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e8643d4-6a32-4da4-adb2-172fe47af3a5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:31:28.503084 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.auth.core [None req-7e8643d4-6a32-4da4-adb2-172fe47af3a5 None None] MFA Rules not processed for user `2dce40b8d74c41ec94cc36e6ff05d412`. Rule list: `[]` (Enabled: `True`). {{(pid=80244) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 08:31:28.504595 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.fernet_utils [None req-7e8643d4-6a32-4da4-adb2-172fe47af3a5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80244) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 08:31:28.570423 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 1055/2107] 10.222.0.47 () {58 vars in 957 bytes} [Thu Jul 21 08:31:28 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 08:31:28.581596 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed889ced-d850-495c-a17f-214a6f8ac8c3 admin admin] Authenticating user token {{(pid=80245) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:31:28.583101 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.fernet_utils [None req-ed889ced-d850-495c-a17f-214a6f8ac8c3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80245) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 08:31:28.646555 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed889ced-d850-495c-a17f-214a6f8ac8c3 admin admin] Validating token access rules against request {{(pid=80245) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:31:28.650578 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-995914d4-f9da-42c2-8385-e6e19f607c40 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20ca3c3bd3774691b3374cfbfec6e494', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4faef13aa50e4e36b6c265fb1398fa14', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80245) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:31:28.651252 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-995914d4-f9da-42c2-8385-e6e19f607c40 service neutron] REQUEST_METHOD: `GET` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:31:28.651575 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-995914d4-f9da-42c2-8385-e6e19f607c40 service neutron] SCRIPT_NAME: `/identity` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:31:28.651856 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-995914d4-f9da-42c2-8385-e6e19f607c40 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:31:28.661782 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-995914d4-f9da-42c2-8385-e6e19f607c40 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:31:28.663197 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-995914d4-f9da-42c2-8385-e6e19f607c40 service neutron] RBAC: Authorization granted {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:31:28.673094 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: [pid: 80245|app: 0|req: 1053/2108] 10.222.0.47 () {60 vars in 1365 bytes} [Thu Jul 21 08:31:28 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 08:31:28.952275 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e8643d4-6a32-4da4-adb2-172fe47af3a5 None None] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:31:28.953875 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.fernet_utils [None req-7e8643d4-6a32-4da4-adb2-172fe47af3a5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80244) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 08:31:28.971562 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e8643d4-6a32-4da4-adb2-172fe47af3a5 None None] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:31:28.978209 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d3766fb-e909-4e5e-a700-169d64d5d037 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '42d15fc0515945acacbbffcf2356c7a5', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:31:28.979146 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d3766fb-e909-4e5e-a700-169d64d5d037 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:31:28.979468 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d3766fb-e909-4e5e-a700-169d64d5d037 admin admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:31:28.979864 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d3766fb-e909-4e5e-a700-169d64d5d037 admin admin] PATH_INFO: `/v3/projects/084f5750d01041f0b58b919e4b254af0` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:31:28.982432 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d3766fb-e909-4e5e-a700-169d64d5d037 admin admin] RBAC: Authorizing `identity:delete_project(project_id=084f5750d01041f0b58b919e4b254af0)` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:31:28.985263 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d3766fb-e909-4e5e-a700-169d64d5d037 admin admin] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:31:29.038344 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.notifications [None req-5d3766fb-e909-4e5e-a700-169d64d5d037 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '084f5750d01041f0b58b919e4b254af0', 'request_id': 'req-5d3766fb-e909-4e5e-a700-169d64d5d037'} {{(pid=80244) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 08:31:29.046959 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.notifications [None req-5d3766fb-e909-4e5e-a700-169d64d5d037 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '084f5750d01041f0b58b919e4b254af0', 'request_id': 'req-5d3766fb-e909-4e5e-a700-169d64d5d037'} {{(pid=80244) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 08:31:29.054389 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.notifications [None req-5d3766fb-e909-4e5e-a700-169d64d5d037 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '084f5750d01041f0b58b919e4b254af0', 'request_id': 'req-5d3766fb-e909-4e5e-a700-169d64d5d037'} {{(pid=80244) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 08:31:29.066610 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.notifications [None req-5d3766fb-e909-4e5e-a700-169d64d5d037 admin admin] The token cache is being invalidate because project 084f5750d01041f0b58b919e4b254af0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80244) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 21 08:31:29.066905 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.notifications [None req-5d3766fb-e909-4e5e-a700-169d64d5d037 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80244) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 08:31:29.067580 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.notifications [None req-5d3766fb-e909-4e5e-a700-169d64d5d037 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80244) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 08:31:29.069229 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 1056/2109] 10.222.0.47 () {58 vars in 1232 bytes} [Thu Jul 21 08:31:28 2022] DELETE /identity/v3/projects/084f5750d01041f0b58b919e4b254af0 => generated 0 bytes in 118 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 08:31:29.073403 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-995914d4-f9da-42c2-8385-e6e19f607c40 service neutron] Authenticating user token {{(pid=80245) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:31:29.083519 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-995914d4-f9da-42c2-8385-e6e19f607c40 service neutron] Validating token access rules against request {{(pid=80245) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:31:29.142211 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98681f8a-da4a-4742-bb97-a51d46fcfa09 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '42d15fc0515945acacbbffcf2356c7a5', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80245) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:31:29.143124 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98681f8a-da4a-4742-bb97-a51d46fcfa09 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:31:29.143445 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98681f8a-da4a-4742-bb97-a51d46fcfa09 admin admin] SCRIPT_NAME: `/identity` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:31:29.143731 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98681f8a-da4a-4742-bb97-a51d46fcfa09 admin admin] PATH_INFO: `/v3/users/78ad0a20074946b082ea02bddca54f04` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:31:29.144937 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98681f8a-da4a-4742-bb97-a51d46fcfa09 admin admin] RBAC: Authorizing `identity:delete_user(user_id=78ad0a20074946b082ea02bddca54f04)` {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:31:29.146868 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98681f8a-da4a-4742-bb97-a51d46fcfa09 admin admin] RBAC: Authorization granted {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:31:29.193797 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.notifications [None req-98681f8a-da4a-4742-bb97-a51d46fcfa09 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '78ad0a20074946b082ea02bddca54f04'} {{(pid=80245) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 08:31:29.202283 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.notifications [None req-98681f8a-da4a-4742-bb97-a51d46fcfa09 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '78ad0a20074946b082ea02bddca54f04'} {{(pid=80245) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 08:31:29.215454 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.notifications [None req-98681f8a-da4a-4742-bb97-a51d46fcfa09 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '78ad0a20074946b082ea02bddca54f04'} {{(pid=80245) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 08:31:29.222513 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.notifications [None req-98681f8a-da4a-4742-bb97-a51d46fcfa09 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '78ad0a20074946b082ea02bddca54f04'} {{(pid=80245) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 08:31:29.223174 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.notifications [None req-98681f8a-da4a-4742-bb97-a51d46fcfa09 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '78ad0a20074946b082ea02bddca54f04'} {{(pid=80245) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 08:31:29.229524 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: [pid: 80245|app: 0|req: 1054/2110] 10.222.0.47 () {58 vars in 1223 bytes} [Thu Jul 21 08:31:29 2022] DELETE /identity/v3/users/78ad0a20074946b082ea02bddca54f04 => generated 0 bytes in 157 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 08:31:29.503734 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d3766fb-e909-4e5e-a700-169d64d5d037 admin admin] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:31:29.505613 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.fernet_utils [None req-5d3766fb-e909-4e5e-a700-169d64d5d037 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80244) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 08:31:29.520016 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d3766fb-e909-4e5e-a700-169d64d5d037 admin admin] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:31:29.553685 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG dbcounter [-] [80244] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=80244) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:31:29.585241 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2c1a7f6-51a1-4a4c-941b-35f665eb1de4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '42d15fc0515945acacbbffcf2356c7a5', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:31:29.586113 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2c1a7f6-51a1-4a4c-941b-35f665eb1de4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:31:29.586398 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2c1a7f6-51a1-4a4c-941b-35f665eb1de4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:31:29.586675 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2c1a7f6-51a1-4a4c-941b-35f665eb1de4 admin admin] PATH_INFO: `/v3/projects/3bd551cb148e48aca56ead1aa7b6885a` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:31:29.587932 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2c1a7f6-51a1-4a4c-941b-35f665eb1de4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=3bd551cb148e48aca56ead1aa7b6885a)` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:31:29.589835 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2c1a7f6-51a1-4a4c-941b-35f665eb1de4 admin admin] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:31:29.642749 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.notifications [None req-c2c1a7f6-51a1-4a4c-941b-35f665eb1de4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3bd551cb148e48aca56ead1aa7b6885a', 'request_id': 'req-c2c1a7f6-51a1-4a4c-941b-35f665eb1de4'} {{(pid=80244) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 08:31:29.650743 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.notifications [None req-c2c1a7f6-51a1-4a4c-941b-35f665eb1de4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3bd551cb148e48aca56ead1aa7b6885a', 'request_id': 'req-c2c1a7f6-51a1-4a4c-941b-35f665eb1de4'} {{(pid=80244) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 08:31:29.657983 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.notifications [None req-c2c1a7f6-51a1-4a4c-941b-35f665eb1de4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3bd551cb148e48aca56ead1aa7b6885a', 'request_id': 'req-c2c1a7f6-51a1-4a4c-941b-35f665eb1de4'} {{(pid=80244) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 08:31:29.669306 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.notifications [None req-c2c1a7f6-51a1-4a4c-941b-35f665eb1de4 admin admin] The token cache is being invalidate because project 3bd551cb148e48aca56ead1aa7b6885a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80244) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 21 08:31:29.669598 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.notifications [None req-c2c1a7f6-51a1-4a4c-941b-35f665eb1de4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80244) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 08:31:29.670254 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.notifications [None req-c2c1a7f6-51a1-4a4c-941b-35f665eb1de4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80244) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 08:31:29.672104 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 1057/2111] 10.222.0.47 () {58 vars in 1232 bytes} [Thu Jul 21 08:31:29 2022] DELETE /identity/v3/projects/3bd551cb148e48aca56ead1aa7b6885a => generated 0 bytes in 169 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 21 08:31:29.677014 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-786a1952-2663-4846-8c38-0d8f4306332e None None] REQUEST_METHOD: `POST` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:31:29.677309 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-786a1952-2663-4846-8c38-0d8f4306332e None None] SCRIPT_NAME: `/identity` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:31:29.677614 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-786a1952-2663-4846-8c38-0d8f4306332e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:31:29.697237 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.auth.core [None req-786a1952-2663-4846-8c38-0d8f4306332e None None] MFA Rules not processed for user `2dce40b8d74c41ec94cc36e6ff05d412`. Rule list: `[]` (Enabled: `True`). {{(pid=80245) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 08:31:29.698381 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.fernet_utils [None req-786a1952-2663-4846-8c38-0d8f4306332e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80245) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 08:31:29.720105 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG dbcounter [-] [80245] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80245) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:31:29.766837 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: [pid: 80245|app: 0|req: 1055/2112] 10.222.0.47 () {58 vars in 957 bytes} [Thu Jul 21 08:31:29 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 08:31:29.775435 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c7e8eb2-4e3e-47bf-8da4-f45309955b3e None None] REQUEST_METHOD: `POST` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:31:29.775759 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c7e8eb2-4e3e-47bf-8da4-f45309955b3e None None] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:31:29.776089 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c7e8eb2-4e3e-47bf-8da4-f45309955b3e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:31:29.794408 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.auth.core [None req-6c7e8eb2-4e3e-47bf-8da4-f45309955b3e None None] MFA Rules not processed for user `2dce40b8d74c41ec94cc36e6ff05d412`. Rule list: `[]` (Enabled: `True`). {{(pid=80244) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 08:31:29.795671 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.fernet_utils [None req-6c7e8eb2-4e3e-47bf-8da4-f45309955b3e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80244) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 08:31:29.803654 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 1058/2113] 10.222.0.47 () {58 vars in 957 bytes} [Thu Jul 21 08:31:29 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 08:31:29.807856 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-786a1952-2663-4846-8c38-0d8f4306332e None None] Authenticating user token {{(pid=80245) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:31:29.818850 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-786a1952-2663-4846-8c38-0d8f4306332e None None] Validating token access rules against request {{(pid=80245) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:31:29.823532 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-944652ca-6f99-46b8-b63a-d4469626c613 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '42d15fc0515945acacbbffcf2356c7a5', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80245) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:31:29.824753 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-944652ca-6f99-46b8-b63a-d4469626c613 admin admin] REQUEST_METHOD: `GET` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:31:29.825165 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-944652ca-6f99-46b8-b63a-d4469626c613 admin admin] SCRIPT_NAME: `/identity` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:31:29.825447 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-944652ca-6f99-46b8-b63a-d4469626c613 admin admin] PATH_INFO: `/v3/domains` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:31:29.826178 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-944652ca-6f99-46b8-b63a-d4469626c613 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80245) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 08:31:29.826755 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-944652ca-6f99-46b8-b63a-d4469626c613 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:31:29.828373 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-944652ca-6f99-46b8-b63a-d4469626c613 admin admin] RBAC: Authorization granted {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:31:29.840441 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: [pid: 80245|app: 0|req: 1056/2114] 10.222.0.47 () {58 vars in 1152 bytes} [Thu Jul 21 08:31:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 08:31:29.844537 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c7e8eb2-4e3e-47bf-8da4-f45309955b3e None None] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:31:29.854279 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c7e8eb2-4e3e-47bf-8da4-f45309955b3e None None] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:31:29.859743 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b59e70e6-a980-46e7-a4fe-5c3b5d9b79a5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '42d15fc0515945acacbbffcf2356c7a5', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:31:29.860413 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b59e70e6-a980-46e7-a4fe-5c3b5d9b79a5 admin admin] REQUEST_METHOD: `POST` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:31:29.860740 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b59e70e6-a980-46e7-a4fe-5c3b5d9b79a5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:31:29.861015 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b59e70e6-a980-46e7-a4fe-5c3b5d9b79a5 admin admin] PATH_INFO: `/v3/projects` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:31:29.861721 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b59e70e6-a980-46e7-a4fe-5c3b5d9b79a5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:31:29.863657 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b59e70e6-a980-46e7-a4fe-5c3b5d9b79a5 admin admin] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:31:29.893293 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 1059/2115] 10.222.0.47 () {60 vars in 1152 bytes} [Thu Jul 21 08:31:29 2022] POST /identity/v3/projects => generated 387 bytes in 49 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 21 08:31:29.897034 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-944652ca-6f99-46b8-b63a-d4469626c613 admin admin] Authenticating user token {{(pid=80245) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:31:29.907011 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-944652ca-6f99-46b8-b63a-d4469626c613 admin admin] Validating token access rules against request {{(pid=80245) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:31:29.976320 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-720a013a-8a3b-4833-9e3c-32cdb84de447 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '42d15fc0515945acacbbffcf2356c7a5', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80245) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:31:29.977012 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-720a013a-8a3b-4833-9e3c-32cdb84de447 admin admin] REQUEST_METHOD: `POST` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:31:29.977325 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-720a013a-8a3b-4833-9e3c-32cdb84de447 admin admin] SCRIPT_NAME: `/identity` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:31:29.977607 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-720a013a-8a3b-4833-9e3c-32cdb84de447 admin admin] PATH_INFO: `/v3/users` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:31:29.978319 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-720a013a-8a3b-4833-9e3c-32cdb84de447 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:31:29.980807 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-720a013a-8a3b-4833-9e3c-32cdb84de447 admin admin] RBAC: Authorization granted {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:31:30.004530 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: [pid: 80245|app: 0|req: 1057/2116] 10.222.0.47 () {60 vars in 1143 bytes} [Thu Jul 21 08:31:29 2022] POST /identity/v3/users => generated 294 bytes in 108 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 08:31:30.009635 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b59e70e6-a980-46e7-a4fe-5c3b5d9b79a5 admin admin] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:31:30.024895 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b59e70e6-a980-46e7-a4fe-5c3b5d9b79a5 admin admin] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:31:30.030995 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1793bc5-6030-4631-ab53-bb3ef34db016 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '42d15fc0515945acacbbffcf2356c7a5', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:31:30.031717 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1793bc5-6030-4631-ab53-bb3ef34db016 admin admin] REQUEST_METHOD: `GET` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:31:30.032009 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1793bc5-6030-4631-ab53-bb3ef34db016 admin admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:31:30.032341 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1793bc5-6030-4631-ab53-bb3ef34db016 admin admin] PATH_INFO: `/v3/roles` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:31:30.033649 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1793bc5-6030-4631-ab53-bb3ef34db016 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:31:30.034840 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1793bc5-6030-4631-ab53-bb3ef34db016 admin admin] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:31:30.053956 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 1060/2117] 10.222.0.47 () {58 vars in 1121 bytes} [Thu Jul 21 08:31:30 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 08:31:30.054071 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-720a013a-8a3b-4833-9e3c-32cdb84de447 admin admin] Authenticating user token {{(pid=80245) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:31:30.063349 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-720a013a-8a3b-4833-9e3c-32cdb84de447 admin admin] Validating token access rules against request {{(pid=80245) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:31:30.068276 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-517f16fe-94a9-48dd-8612-848ff63d7f13 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '42d15fc0515945acacbbffcf2356c7a5', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80245) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:31:30.068986 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-517f16fe-94a9-48dd-8612-848ff63d7f13 admin admin] REQUEST_METHOD: `GET` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:31:30.069408 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-517f16fe-94a9-48dd-8612-848ff63d7f13 admin admin] SCRIPT_NAME: `/identity` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:31:30.069749 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-517f16fe-94a9-48dd-8612-848ff63d7f13 admin admin] PATH_INFO: `/v3/roles` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:31:30.070496 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-517f16fe-94a9-48dd-8612-848ff63d7f13 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:31:30.072165 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-517f16fe-94a9-48dd-8612-848ff63d7f13 admin admin] RBAC: Authorization granted {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:31:30.085152 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: [pid: 80245|app: 0|req: 1058/2118] 10.222.0.47 () {58 vars in 1121 bytes} [Thu Jul 21 08:31:30 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 08:31:30.088391 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1793bc5-6030-4631-ab53-bb3ef34db016 admin admin] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:31:30.098987 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1793bc5-6030-4631-ab53-bb3ef34db016 admin admin] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:31:30.104804 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7df271b7-ae6c-463e-b7c9-7530470c1d1f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '42d15fc0515945acacbbffcf2356c7a5', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:31:30.106220 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7df271b7-ae6c-463e-b7c9-7530470c1d1f admin admin] REQUEST_METHOD: `PUT` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:31:30.106528 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7df271b7-ae6c-463e-b7c9-7530470c1d1f admin admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:31:30.106792 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7df271b7-ae6c-463e-b7c9-7530470c1d1f admin admin] PATH_INFO: `/v3/projects/d3da643296d0420897bcfb8500f2c7a8/users/c0c589f80a654af3b9011321bc6a8dd7/roles/3a074738221e4601b5205715ba87d240` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:31:30.127379 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7df271b7-ae6c-463e-b7c9-7530470c1d1f admin admin] RBAC: Authorizing `identity:create_grant(project_id=d3da643296d0420897bcfb8500f2c7a8, user_id=c0c589f80a654af3b9011321bc6a8dd7, role_id=3a074738221e4601b5205715ba87d240)` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:31:30.130399 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7df271b7-ae6c-463e-b7c9-7530470c1d1f admin admin] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:31:30.143682 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 1061/2119] 10.222.0.47 () {60 vars in 1482 bytes} [Thu Jul 21 08:31:30 2022] PUT /identity/v3/projects/d3da643296d0420897bcfb8500f2c7a8/users/c0c589f80a654af3b9011321bc6a8dd7/roles/3a074738221e4601b5205715ba87d240 => generated 0 bytes in 56 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 21 08:31:30.154593 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c884dc9-403f-4b8a-9e0c-55c0d00b2a3d None None] REQUEST_METHOD: `POST` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:31:30.154899 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c884dc9-403f-4b8a-9e0c-55c0d00b2a3d None None] SCRIPT_NAME: `/identity` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:31:30.155192 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c884dc9-403f-4b8a-9e0c-55c0d00b2a3d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:31:30.182196 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.auth.core [None req-6c884dc9-403f-4b8a-9e0c-55c0d00b2a3d None None] MFA Rules not processed for user `c0c589f80a654af3b9011321bc6a8dd7`. Rule list: `[]` (Enabled: `True`). {{(pid=80245) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 08:31:30.184057 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.fernet_utils [None req-6c884dc9-403f-4b8a-9e0c-55c0d00b2a3d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80245) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 08:31:30.251201 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: [pid: 80245|app: 0|req: 1059/2120] 10.222.0.47 () {58 vars in 957 bytes} [Thu Jul 21 08:31:30 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 08:31:30.255947 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7df271b7-ae6c-463e-b7c9-7530470c1d1f admin admin] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:31:30.321487 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG dbcounter [-] [80244] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80244) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:31:30.325468 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7df271b7-ae6c-463e-b7c9-7530470c1d1f admin admin] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:31:30.331841 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b88e031a-cbee-4eff-bba2-b36ad6fa26e1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '42d15fc0515945acacbbffcf2356c7a5', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:31:30.332525 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b88e031a-cbee-4eff-bba2-b36ad6fa26e1 admin admin] REQUEST_METHOD: `POST` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:31:30.333187 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b88e031a-cbee-4eff-bba2-b36ad6fa26e1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:31:30.333649 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b88e031a-cbee-4eff-bba2-b36ad6fa26e1 admin admin] PATH_INFO: `/v3/projects` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:31:30.334380 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b88e031a-cbee-4eff-bba2-b36ad6fa26e1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:31:30.336477 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b88e031a-cbee-4eff-bba2-b36ad6fa26e1 admin admin] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:31:30.364648 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 1062/2121] 10.222.0.47 () {60 vars in 1152 bytes} [Thu Jul 21 08:31:30 2022] POST /identity/v3/projects => generated 385 bytes in 109 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 08:31:30.368852 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c884dc9-403f-4b8a-9e0c-55c0d00b2a3d None None] Authenticating user token {{(pid=80245) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:31:30.379897 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c884dc9-403f-4b8a-9e0c-55c0d00b2a3d None None] Validating token access rules against request {{(pid=80245) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:31:30.393006 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG dbcounter [-] [80245] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80245) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:31:30.439706 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53045e9b-ee2b-4cb4-9d0d-b77cf878fdeb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '42d15fc0515945acacbbffcf2356c7a5', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80245) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:31:30.440384 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53045e9b-ee2b-4cb4-9d0d-b77cf878fdeb admin admin] REQUEST_METHOD: `POST` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:31:30.440672 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53045e9b-ee2b-4cb4-9d0d-b77cf878fdeb admin admin] SCRIPT_NAME: `/identity` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:31:30.440951 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53045e9b-ee2b-4cb4-9d0d-b77cf878fdeb admin admin] PATH_INFO: `/v3/users` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:31:30.441662 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53045e9b-ee2b-4cb4-9d0d-b77cf878fdeb admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:31:30.443619 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53045e9b-ee2b-4cb4-9d0d-b77cf878fdeb admin admin] RBAC: Authorization granted {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:31:30.465080 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: [pid: 80245|app: 0|req: 1060/2122] 10.222.0.47 () {60 vars in 1143 bytes} [Thu Jul 21 08:31:30 2022] POST /identity/v3/users => generated 293 bytes in 96 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 08:31:30.469116 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b88e031a-cbee-4eff-bba2-b36ad6fa26e1 admin admin] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:31:30.479399 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b88e031a-cbee-4eff-bba2-b36ad6fa26e1 admin admin] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:31:30.484708 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bd54cf0-828a-45e5-a4f8-ea318ad719f0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '42d15fc0515945acacbbffcf2356c7a5', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:31:30.485368 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bd54cf0-828a-45e5-a4f8-ea318ad719f0 admin admin] REQUEST_METHOD: `GET` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:31:30.485652 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bd54cf0-828a-45e5-a4f8-ea318ad719f0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:31:30.485922 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bd54cf0-828a-45e5-a4f8-ea318ad719f0 admin admin] PATH_INFO: `/v3/roles` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:31:30.486648 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bd54cf0-828a-45e5-a4f8-ea318ad719f0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:31:30.488295 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bd54cf0-828a-45e5-a4f8-ea318ad719f0 admin admin] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:31:30.500130 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 1063/2123] 10.222.0.47 () {58 vars in 1121 bytes} [Thu Jul 21 08:31:30 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 08:31:30.505995 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53045e9b-ee2b-4cb4-9d0d-b77cf878fdeb admin admin] Authenticating user token {{(pid=80245) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:31:30.516609 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53045e9b-ee2b-4cb4-9d0d-b77cf878fdeb admin admin] Validating token access rules against request {{(pid=80245) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:31:30.521440 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92156788-af79-4092-a4e2-bc2210f7cce1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '42d15fc0515945acacbbffcf2356c7a5', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80245) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:31:30.522110 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92156788-af79-4092-a4e2-bc2210f7cce1 admin admin] REQUEST_METHOD: `GET` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:31:30.522403 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92156788-af79-4092-a4e2-bc2210f7cce1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:31:30.522681 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92156788-af79-4092-a4e2-bc2210f7cce1 admin admin] PATH_INFO: `/v3/roles` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:31:30.523482 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92156788-af79-4092-a4e2-bc2210f7cce1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:31:30.525088 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92156788-af79-4092-a4e2-bc2210f7cce1 admin admin] RBAC: Authorization granted {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:31:30.537113 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: [pid: 80245|app: 0|req: 1061/2124] 10.222.0.47 () {58 vars in 1121 bytes} [Thu Jul 21 08:31:30 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 08:31:30.541350 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bd54cf0-828a-45e5-a4f8-ea318ad719f0 admin admin] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:31:30.553084 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bd54cf0-828a-45e5-a4f8-ea318ad719f0 admin admin] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:31:30.558059 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48778d4d-5f64-4a50-b680-1af407c0cd47 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '42d15fc0515945acacbbffcf2356c7a5', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:31:30.558996 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48778d4d-5f64-4a50-b680-1af407c0cd47 admin admin] REQUEST_METHOD: `PUT` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:31:30.559288 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48778d4d-5f64-4a50-b680-1af407c0cd47 admin admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:31:30.559598 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48778d4d-5f64-4a50-b680-1af407c0cd47 admin admin] PATH_INFO: `/v3/projects/844015ffe3af4fe891a9d367a0376fbe/users/4a0ef7b7d1314f889655a46a5b570e31/roles/3a074738221e4601b5205715ba87d240` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:31:30.574734 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48778d4d-5f64-4a50-b680-1af407c0cd47 admin admin] RBAC: Authorizing `identity:create_grant(project_id=844015ffe3af4fe891a9d367a0376fbe, user_id=4a0ef7b7d1314f889655a46a5b570e31, role_id=3a074738221e4601b5205715ba87d240)` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:31:30.577573 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48778d4d-5f64-4a50-b680-1af407c0cd47 admin admin] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:31:30.590828 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 1064/2125] 10.222.0.47 () {60 vars in 1482 bytes} [Thu Jul 21 08:31:30 2022] PUT /identity/v3/projects/844015ffe3af4fe891a9d367a0376fbe/users/4a0ef7b7d1314f889655a46a5b570e31/roles/3a074738221e4601b5205715ba87d240 => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 08:31:30.602127 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08d9e181-fdb9-4dd7-bd84-8407ff326101 None None] REQUEST_METHOD: `POST` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:31:30.602432 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08d9e181-fdb9-4dd7-bd84-8407ff326101 None None] SCRIPT_NAME: `/identity` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:31:30.602719 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08d9e181-fdb9-4dd7-bd84-8407ff326101 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:31:30.638915 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.auth.core [None req-08d9e181-fdb9-4dd7-bd84-8407ff326101 None None] MFA Rules not processed for user `4a0ef7b7d1314f889655a46a5b570e31`. Rule list: `[]` (Enabled: `True`). {{(pid=80245) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 08:31:30.640234 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.fernet_utils [None req-08d9e181-fdb9-4dd7-bd84-8407ff326101 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80245) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 08:31:30.709232 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: [pid: 80245|app: 0|req: 1062/2126] 10.222.0.47 () {58 vars in 957 bytes} [Thu Jul 21 08:31:30 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 08:31:30.713760 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48778d4d-5f64-4a50-b680-1af407c0cd47 admin admin] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:31:30.777772 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48778d4d-5f64-4a50-b680-1af407c0cd47 admin admin] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:31:30.783426 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b639ba00-692a-4ba7-8edf-9ede6d21afd8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '42d15fc0515945acacbbffcf2356c7a5', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:31:30.783426 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b639ba00-692a-4ba7-8edf-9ede6d21afd8 admin admin] REQUEST_METHOD: `POST` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:31:30.783426 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b639ba00-692a-4ba7-8edf-9ede6d21afd8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:31:30.783793 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b639ba00-692a-4ba7-8edf-9ede6d21afd8 admin admin] PATH_INFO: `/v3/projects` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:31:30.784319 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b639ba00-692a-4ba7-8edf-9ede6d21afd8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:31:30.786246 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b639ba00-692a-4ba7-8edf-9ede6d21afd8 admin admin] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:31:30.817160 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 1065/2127] 10.222.0.47 () {60 vars in 1152 bytes} [Thu Jul 21 08:31:30 2022] POST /identity/v3/projects => generated 385 bytes in 104 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 08:31:30.821533 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08d9e181-fdb9-4dd7-bd84-8407ff326101 None None] Authenticating user token {{(pid=80245) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:31:30.834070 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08d9e181-fdb9-4dd7-bd84-8407ff326101 None None] Validating token access rules against request {{(pid=80245) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:31:30.907298 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bb6656f-7b2d-4d81-9ea8-09e6b322e70e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '42d15fc0515945acacbbffcf2356c7a5', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80245) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:31:30.907997 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bb6656f-7b2d-4d81-9ea8-09e6b322e70e admin admin] REQUEST_METHOD: `POST` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:31:30.908279 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bb6656f-7b2d-4d81-9ea8-09e6b322e70e admin admin] SCRIPT_NAME: `/identity` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:31:30.908686 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bb6656f-7b2d-4d81-9ea8-09e6b322e70e admin admin] PATH_INFO: `/v3/users` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:31:30.909459 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bb6656f-7b2d-4d81-9ea8-09e6b322e70e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:31:30.911737 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bb6656f-7b2d-4d81-9ea8-09e6b322e70e admin admin] RBAC: Authorization granted {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:31:30.934830 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: [pid: 80245|app: 0|req: 1063/2128] 10.222.0.47 () {60 vars in 1143 bytes} [Thu Jul 21 08:31:30 2022] POST /identity/v3/users => generated 299 bytes in 111 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 21 08:31:30.936017 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b639ba00-692a-4ba7-8edf-9ede6d21afd8 admin admin] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:31:30.946551 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b639ba00-692a-4ba7-8edf-9ede6d21afd8 admin admin] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:31:30.952665 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f539c16a-2091-4830-90c6-5053387cb124 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '42d15fc0515945acacbbffcf2356c7a5', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:31:30.953331 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f539c16a-2091-4830-90c6-5053387cb124 admin admin] REQUEST_METHOD: `GET` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:31:30.953625 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f539c16a-2091-4830-90c6-5053387cb124 admin admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:31:30.953901 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f539c16a-2091-4830-90c6-5053387cb124 admin admin] PATH_INFO: `/v3/roles` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:31:30.954633 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f539c16a-2091-4830-90c6-5053387cb124 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:31:30.956293 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f539c16a-2091-4830-90c6-5053387cb124 admin admin] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:31:30.973694 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 1066/2129] 10.222.0.47 () {58 vars in 1121 bytes} [Thu Jul 21 08:31:30 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 21 08:31:30.978012 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bb6656f-7b2d-4d81-9ea8-09e6b322e70e admin admin] Authenticating user token {{(pid=80245) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:31:30.992544 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bb6656f-7b2d-4d81-9ea8-09e6b322e70e admin admin] Validating token access rules against request {{(pid=80245) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:31:30.997572 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-654b1a9d-4b62-4f0e-880a-342f7f701a63 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '42d15fc0515945acacbbffcf2356c7a5', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80245) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:31:30.998747 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-654b1a9d-4b62-4f0e-880a-342f7f701a63 admin admin] REQUEST_METHOD: `PUT` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:31:30.999034 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-654b1a9d-4b62-4f0e-880a-342f7f701a63 admin admin] SCRIPT_NAME: `/identity` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:31:30.999315 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-654b1a9d-4b62-4f0e-880a-342f7f701a63 admin admin] PATH_INFO: `/v3/projects/a4a8929428914b9c9c14906b361e4609/users/f5ee6d78b3ff443ebf34708cbac33585/roles/e29870f6e9e2411a9db0d5bf45f32ca5` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:31:31.013739 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG dbcounter [-] [80245] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=80245) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:31:31.017887 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-654b1a9d-4b62-4f0e-880a-342f7f701a63 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a4a8929428914b9c9c14906b361e4609, user_id=f5ee6d78b3ff443ebf34708cbac33585, role_id=e29870f6e9e2411a9db0d5bf45f32ca5)` {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:31:31.020911 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-654b1a9d-4b62-4f0e-880a-342f7f701a63 admin admin] RBAC: Authorization granted {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:31:31.043133 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: [pid: 80245|app: 0|req: 1064/2130] 10.222.0.47 () {60 vars in 1482 bytes} [Thu Jul 21 08:31:30 2022] PUT /identity/v3/projects/a4a8929428914b9c9c14906b361e4609/users/f5ee6d78b3ff443ebf34708cbac33585/roles/e29870f6e9e2411a9db0d5bf45f32ca5 => generated 0 bytes in 66 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 08:31:31.047064 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f539c16a-2091-4830-90c6-5053387cb124 admin admin] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:31:31.060028 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f539c16a-2091-4830-90c6-5053387cb124 admin admin] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:31:31.123508 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25d87a13-8bfb-4bef-9611-356fca4537ab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '42d15fc0515945acacbbffcf2356c7a5', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:31:31.124326 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25d87a13-8bfb-4bef-9611-356fca4537ab admin admin] REQUEST_METHOD: `GET` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:31:31.124706 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25d87a13-8bfb-4bef-9611-356fca4537ab admin admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:31:31.125015 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25d87a13-8bfb-4bef-9611-356fca4537ab admin admin] PATH_INFO: `/v3/roles` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:31:31.125744 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25d87a13-8bfb-4bef-9611-356fca4537ab admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:31:31.127600 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25d87a13-8bfb-4bef-9611-356fca4537ab admin admin] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:31:31.142488 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 1067/2131] 10.222.0.47 () {58 vars in 1121 bytes} [Thu Jul 21 08:31:31 2022] GET /identity/v3/roles => generated 1420 bytes in 96 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 08:31:31.148146 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-654b1a9d-4b62-4f0e-880a-342f7f701a63 admin admin] Authenticating user token {{(pid=80245) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:31:31.159179 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-654b1a9d-4b62-4f0e-880a-342f7f701a63 admin admin] Validating token access rules against request {{(pid=80245) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:31:31.165414 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-406438f2-82b3-4658-baf4-ce0290570c99 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '42d15fc0515945acacbbffcf2356c7a5', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80245) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:31:31.166322 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-406438f2-82b3-4658-baf4-ce0290570c99 admin admin] REQUEST_METHOD: `PUT` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:31:31.166608 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-406438f2-82b3-4658-baf4-ce0290570c99 admin admin] SCRIPT_NAME: `/identity` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:31:31.166889 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-406438f2-82b3-4658-baf4-ce0290570c99 admin admin] PATH_INFO: `/v3/projects/a4a8929428914b9c9c14906b361e4609/users/f5ee6d78b3ff443ebf34708cbac33585/roles/e29870f6e9e2411a9db0d5bf45f32ca5` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:31:31.169024 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-406438f2-82b3-4658-baf4-ce0290570c99 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a4a8929428914b9c9c14906b361e4609, user_id=f5ee6d78b3ff443ebf34708cbac33585, role_id=e29870f6e9e2411a9db0d5bf45f32ca5)` {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:31:31.172106 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-406438f2-82b3-4658-baf4-ce0290570c99 admin admin] RBAC: Authorization granted {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:31:31.185156 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: [pid: 80245|app: 0|req: 1065/2132] 10.222.0.47 () {60 vars in 1482 bytes} [Thu Jul 21 08:31:31 2022] PUT /identity/v3/projects/a4a8929428914b9c9c14906b361e4609/users/f5ee6d78b3ff443ebf34708cbac33585/roles/e29870f6e9e2411a9db0d5bf45f32ca5 => generated 0 bytes in 38 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 08:31:31.195733 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4ad2b9e-8189-4ab6-95fb-a7dbd3162047 None None] REQUEST_METHOD: `POST` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:31:31.196182 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4ad2b9e-8189-4ab6-95fb-a7dbd3162047 None None] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:31:31.196604 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4ad2b9e-8189-4ab6-95fb-a7dbd3162047 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:31:31.239173 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.auth.core [None req-d4ad2b9e-8189-4ab6-95fb-a7dbd3162047 None None] MFA Rules not processed for user `f5ee6d78b3ff443ebf34708cbac33585`. Rule list: `[]` (Enabled: `True`). {{(pid=80244) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 08:31:31.241031 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.fernet_utils [None req-d4ad2b9e-8189-4ab6-95fb-a7dbd3162047 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80244) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 08:31:31.250625 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG dbcounter [-] [80244] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80244) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:31:31.331261 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 1068/2133] 10.222.0.47 () {58 vars in 957 bytes} [Thu Jul 21 08:31:31 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 137 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 08:31:31.343154 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-406438f2-82b3-4658-baf4-ce0290570c99 admin admin] Authenticating user token {{(pid=80245) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:31:31.344959 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.fernet_utils [None req-406438f2-82b3-4658-baf4-ce0290570c99 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80245) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 08:31:31.404732 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-406438f2-82b3-4658-baf4-ce0290570c99 admin admin] Validating token access rules against request {{(pid=80245) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:31:31.408826 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24ea7814-d5d3-45ac-b8ee-4d13187bd1e6 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4f40410b7674687ab5362ac982248ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4faef13aa50e4e36b6c265fb1398fa14', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80245) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:31:31.409481 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24ea7814-d5d3-45ac-b8ee-4d13187bd1e6 service cinder] REQUEST_METHOD: `GET` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:31:31.409769 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24ea7814-d5d3-45ac-b8ee-4d13187bd1e6 service cinder] SCRIPT_NAME: `/identity` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:31:31.410048 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24ea7814-d5d3-45ac-b8ee-4d13187bd1e6 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:31:31.420448 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24ea7814-d5d3-45ac-b8ee-4d13187bd1e6 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:31:31.421887 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24ea7814-d5d3-45ac-b8ee-4d13187bd1e6 service cinder] RBAC: Authorization granted {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:31:31.431384 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: [pid: 80245|app: 0|req: 1066/2134] 10.222.0.47 () {60 vars in 1365 bytes} [Thu Jul 21 08:31:31 2022] GET /identity/v3/auth/tokens => generated 3319 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 08:31:31.940289 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4ad2b9e-8189-4ab6-95fb-a7dbd3162047 None None] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:31:31.941774 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.fernet_utils [None req-d4ad2b9e-8189-4ab6-95fb-a7dbd3162047 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80244) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 08:31:31.955154 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4ad2b9e-8189-4ab6-95fb-a7dbd3162047 None None] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:31:31.959152 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a180f98d-96cf-49aa-b077-e7a92af09ae7 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4f40410b7674687ab5362ac982248ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4faef13aa50e4e36b6c265fb1398fa14', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:31:31.960160 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a180f98d-96cf-49aa-b077-e7a92af09ae7 service cinder] REQUEST_METHOD: `GET` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:31:31.960446 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a180f98d-96cf-49aa-b077-e7a92af09ae7 service cinder] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:31:31.961122 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a180f98d-96cf-49aa-b077-e7a92af09ae7 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:31:32.038099 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a180f98d-96cf-49aa-b077-e7a92af09ae7 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:31:32.039861 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a180f98d-96cf-49aa-b077-e7a92af09ae7 service cinder] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:31:32.056676 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 1069/2135] 10.222.0.47 () {60 vars in 1365 bytes} [Thu Jul 21 08:31:31 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 117 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 08:31:40.092864 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24ea7814-d5d3-45ac-b8ee-4d13187bd1e6 service cinder] Authenticating user token {{(pid=80245) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:31:40.094299 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.fernet_utils [None req-24ea7814-d5d3-45ac-b8ee-4d13187bd1e6 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80245) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 08:31:40.177063 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24ea7814-d5d3-45ac-b8ee-4d13187bd1e6 service cinder] Validating token access rules against request {{(pid=80245) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:31:40.182224 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9b4dc22-cbae-4d59-bc92-bbb5f354efcd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '42d15fc0515945acacbbffcf2356c7a5', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80245) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:31:40.183870 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9b4dc22-cbae-4d59-bc92-bbb5f354efcd admin admin] REQUEST_METHOD: `DELETE` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:31:40.184008 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9b4dc22-cbae-4d59-bc92-bbb5f354efcd admin admin] SCRIPT_NAME: `/identity` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:31:40.184435 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9b4dc22-cbae-4d59-bc92-bbb5f354efcd admin admin] PATH_INFO: `/v3/users/b970723297fe4217ae31c4f11fa8801a` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:31:40.185596 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9b4dc22-cbae-4d59-bc92-bbb5f354efcd admin admin] RBAC: Authorizing `identity:delete_user(user_id=b970723297fe4217ae31c4f11fa8801a)` {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:31:40.188403 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9b4dc22-cbae-4d59-bc92-bbb5f354efcd admin admin] RBAC: Authorization granted {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:31:40.257921 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.notifications [None req-d9b4dc22-cbae-4d59-bc92-bbb5f354efcd admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b970723297fe4217ae31c4f11fa8801a'} {{(pid=80245) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 08:31:40.266890 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.notifications [None req-d9b4dc22-cbae-4d59-bc92-bbb5f354efcd admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b970723297fe4217ae31c4f11fa8801a'} {{(pid=80245) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 08:31:40.279797 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.notifications [None req-d9b4dc22-cbae-4d59-bc92-bbb5f354efcd admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b970723297fe4217ae31c4f11fa8801a'} {{(pid=80245) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 08:31:40.286919 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.notifications [None req-d9b4dc22-cbae-4d59-bc92-bbb5f354efcd admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b970723297fe4217ae31c4f11fa8801a'} {{(pid=80245) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 08:31:40.287638 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.notifications [None req-d9b4dc22-cbae-4d59-bc92-bbb5f354efcd admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b970723297fe4217ae31c4f11fa8801a'} {{(pid=80245) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 08:31:40.290053 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG dbcounter [-] [80245] Writing DB stats keystone:SELECT=87,keystone:INSERT=3,keystone:DELETE=10 {{(pid=80245) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:31:40.295655 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: [pid: 80245|app: 0|req: 1067/2136] 10.222.0.47 () {58 vars in 1223 bytes} [Thu Jul 21 08:31:40 2022] DELETE /identity/v3/users/b970723297fe4217ae31c4f11fa8801a => generated 0 bytes in 203 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 08:31:40.300568 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e889c50a-9750-4e32-9572-cc48b362c92c None None] REQUEST_METHOD: `POST` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:31:40.301482 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e889c50a-9750-4e32-9572-cc48b362c92c None None] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:31:40.301759 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e889c50a-9750-4e32-9572-cc48b362c92c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:31:40.321044 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.auth.core [None req-e889c50a-9750-4e32-9572-cc48b362c92c None None] MFA Rules not processed for user `2dce40b8d74c41ec94cc36e6ff05d412`. Rule list: `[]` (Enabled: `True`). {{(pid=80244) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 08:31:40.322539 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.fernet_utils [None req-e889c50a-9750-4e32-9572-cc48b362c92c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80244) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 08:31:40.385375 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 1070/2137] 10.222.0.47 () {58 vars in 957 bytes} [Thu Jul 21 08:31:40 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 08:31:40.395607 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9b4dc22-cbae-4d59-bc92-bbb5f354efcd admin admin] Authenticating user token {{(pid=80245) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:31:40.397038 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.fernet_utils [None req-d9b4dc22-cbae-4d59-bc92-bbb5f354efcd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80245) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 08:31:40.453812 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9b4dc22-cbae-4d59-bc92-bbb5f354efcd admin admin] Validating token access rules against request {{(pid=80245) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:31:40.457576 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-add34171-052a-40fd-83aa-574807fccf66 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20ca3c3bd3774691b3374cfbfec6e494', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4faef13aa50e4e36b6c265fb1398fa14', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80245) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:31:40.458229 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-add34171-052a-40fd-83aa-574807fccf66 service neutron] REQUEST_METHOD: `GET` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:31:40.458523 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-add34171-052a-40fd-83aa-574807fccf66 service neutron] SCRIPT_NAME: `/identity` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:31:40.458819 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-add34171-052a-40fd-83aa-574807fccf66 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:31:40.468020 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-add34171-052a-40fd-83aa-574807fccf66 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:31:40.469475 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-add34171-052a-40fd-83aa-574807fccf66 service neutron] RBAC: Authorization granted {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:31:40.480409 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: [pid: 80245|app: 0|req: 1068/2138] 10.222.0.47 () {60 vars in 1365 bytes} [Thu Jul 21 08:31:40 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 08:31:40.807114 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e889c50a-9750-4e32-9572-cc48b362c92c None None] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:31:40.809642 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.fernet_utils [None req-e889c50a-9750-4e32-9572-cc48b362c92c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80244) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 08:31:40.829123 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e889c50a-9750-4e32-9572-cc48b362c92c None None] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:31:40.834421 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7625877f-2086-4bc3-9a19-00600d9e6238 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '42d15fc0515945acacbbffcf2356c7a5', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:31:40.834931 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7625877f-2086-4bc3-9a19-00600d9e6238 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:31:40.835221 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7625877f-2086-4bc3-9a19-00600d9e6238 admin admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:31:40.835523 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7625877f-2086-4bc3-9a19-00600d9e6238 admin admin] PATH_INFO: `/v3/projects/5642b20ea95e4ddba1e2f38f5db9ef80` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:31:40.836887 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7625877f-2086-4bc3-9a19-00600d9e6238 admin admin] RBAC: Authorizing `identity:delete_project(project_id=5642b20ea95e4ddba1e2f38f5db9ef80)` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:31:40.838828 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7625877f-2086-4bc3-9a19-00600d9e6238 admin admin] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:31:40.885428 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG dbcounter [-] [80244] Writing DB stats keystone:SELECT=97,keystone:DELETE=3 {{(pid=80244) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:31:40.895460 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.notifications [None req-7625877f-2086-4bc3-9a19-00600d9e6238 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5642b20ea95e4ddba1e2f38f5db9ef80', 'request_id': 'req-7625877f-2086-4bc3-9a19-00600d9e6238'} {{(pid=80244) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 08:31:40.903953 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.notifications [None req-7625877f-2086-4bc3-9a19-00600d9e6238 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5642b20ea95e4ddba1e2f38f5db9ef80', 'request_id': 'req-7625877f-2086-4bc3-9a19-00600d9e6238'} {{(pid=80244) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 08:31:40.911552 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.notifications [None req-7625877f-2086-4bc3-9a19-00600d9e6238 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5642b20ea95e4ddba1e2f38f5db9ef80', 'request_id': 'req-7625877f-2086-4bc3-9a19-00600d9e6238'} {{(pid=80244) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 08:31:40.925743 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.notifications [None req-7625877f-2086-4bc3-9a19-00600d9e6238 admin admin] The token cache is being invalidate because project 5642b20ea95e4ddba1e2f38f5db9ef80 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80244) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 21 08:31:40.926033 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.notifications [None req-7625877f-2086-4bc3-9a19-00600d9e6238 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80244) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 08:31:40.926700 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.notifications [None req-7625877f-2086-4bc3-9a19-00600d9e6238 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80244) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 08:31:40.928509 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 1071/2139] 10.222.0.47 () {58 vars in 1232 bytes} [Thu Jul 21 08:31:40 2022] DELETE /identity/v3/projects/5642b20ea95e4ddba1e2f38f5db9ef80 => generated 0 bytes in 122 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 08:31:40.932518 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-add34171-052a-40fd-83aa-574807fccf66 service neutron] Authenticating user token {{(pid=80245) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:31:40.942774 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-add34171-052a-40fd-83aa-574807fccf66 service neutron] Validating token access rules against request {{(pid=80245) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:31:41.018932 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-604fc8af-c91a-47b2-b381-6e7c7d2d4572 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '42d15fc0515945acacbbffcf2356c7a5', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80245) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:31:41.019871 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-604fc8af-c91a-47b2-b381-6e7c7d2d4572 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:31:41.020166 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-604fc8af-c91a-47b2-b381-6e7c7d2d4572 admin admin] SCRIPT_NAME: `/identity` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:31:41.020437 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-604fc8af-c91a-47b2-b381-6e7c7d2d4572 admin admin] PATH_INFO: `/v3/users/5c620a3a71e44e89a2d4f0a529a4898b` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:31:41.024509 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-604fc8af-c91a-47b2-b381-6e7c7d2d4572 admin admin] RBAC: Authorizing `identity:delete_user(user_id=5c620a3a71e44e89a2d4f0a529a4898b)` {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:31:41.028463 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-604fc8af-c91a-47b2-b381-6e7c7d2d4572 admin admin] RBAC: Authorization granted {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:31:41.095140 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.notifications [None req-604fc8af-c91a-47b2-b381-6e7c7d2d4572 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5c620a3a71e44e89a2d4f0a529a4898b'} {{(pid=80245) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 08:31:41.104297 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.notifications [None req-604fc8af-c91a-47b2-b381-6e7c7d2d4572 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5c620a3a71e44e89a2d4f0a529a4898b'} {{(pid=80245) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 08:31:41.118060 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.notifications [None req-604fc8af-c91a-47b2-b381-6e7c7d2d4572 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5c620a3a71e44e89a2d4f0a529a4898b'} {{(pid=80245) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 08:31:41.126975 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.notifications [None req-604fc8af-c91a-47b2-b381-6e7c7d2d4572 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5c620a3a71e44e89a2d4f0a529a4898b'} {{(pid=80245) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 08:31:41.127738 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.notifications [None req-604fc8af-c91a-47b2-b381-6e7c7d2d4572 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5c620a3a71e44e89a2d4f0a529a4898b'} {{(pid=80245) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 08:31:41.135474 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: [pid: 80245|app: 0|req: 1069/2140] 10.222.0.47 () {58 vars in 1223 bytes} [Thu Jul 21 08:31:40 2022] DELETE /identity/v3/users/5c620a3a71e44e89a2d4f0a529a4898b => generated 0 bytes in 203 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 08:31:41.442769 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7625877f-2086-4bc3-9a19-00600d9e6238 admin admin] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:31:41.444477 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.fernet_utils [None req-7625877f-2086-4bc3-9a19-00600d9e6238 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80244) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 08:31:41.494814 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-604fc8af-c91a-47b2-b381-6e7c7d2d4572 admin admin] Authenticating user token {{(pid=80245) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:31:41.502575 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.fernet_utils [None req-604fc8af-c91a-47b2-b381-6e7c7d2d4572 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80245) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 08:31:41.521468 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7625877f-2086-4bc3-9a19-00600d9e6238 admin admin] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:31:41.567846 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG dbcounter [-] [80245] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80245) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:31:41.592030 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df36ca8e-ceb1-490d-945c-a4bbfb8bef5b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4f40410b7674687ab5362ac982248ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4faef13aa50e4e36b6c265fb1398fa14', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:31:41.592784 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df36ca8e-ceb1-490d-945c-a4bbfb8bef5b service cinder] REQUEST_METHOD: `GET` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:31:41.593128 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df36ca8e-ceb1-490d-945c-a4bbfb8bef5b service cinder] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:31:41.593472 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df36ca8e-ceb1-490d-945c-a4bbfb8bef5b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:31:41.595597 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.fernet_utils [None req-df36ca8e-ceb1-490d-945c-a4bbfb8bef5b service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80244) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 08:31:41.610963 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-604fc8af-c91a-47b2-b381-6e7c7d2d4572 admin admin] Validating token access rules against request {{(pid=80245) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:31:41.616133 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22da7e2c-dcf0-43ae-ae7e-ad8f2f1ee0ed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '42d15fc0515945acacbbffcf2356c7a5', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80245) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:31:41.617001 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22da7e2c-dcf0-43ae-ae7e-ad8f2f1ee0ed admin admin] REQUEST_METHOD: `DELETE` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:31:41.617287 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22da7e2c-dcf0-43ae-ae7e-ad8f2f1ee0ed admin admin] SCRIPT_NAME: `/identity` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:31:41.617567 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22da7e2c-dcf0-43ae-ae7e-ad8f2f1ee0ed admin admin] PATH_INFO: `/v3/projects/fdf6ddfb5bd744c5ac2d9b12434229eb` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:31:41.618757 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22da7e2c-dcf0-43ae-ae7e-ad8f2f1ee0ed admin admin] RBAC: Authorizing `identity:delete_project(project_id=fdf6ddfb5bd744c5ac2d9b12434229eb)` {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:31:41.620741 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22da7e2c-dcf0-43ae-ae7e-ad8f2f1ee0ed admin admin] RBAC: Authorization granted {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:31:41.656768 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df36ca8e-ceb1-490d-945c-a4bbfb8bef5b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:31:41.658439 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df36ca8e-ceb1-490d-945c-a4bbfb8bef5b service cinder] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:31:41.669137 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 1072/2141] 10.222.0.47 () {60 vars in 1365 bytes} [Thu Jul 21 08:31:41 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 228 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 08:31:41.675172 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.notifications [None req-22da7e2c-dcf0-43ae-ae7e-ad8f2f1ee0ed admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'fdf6ddfb5bd744c5ac2d9b12434229eb', 'request_id': 'req-22da7e2c-dcf0-43ae-ae7e-ad8f2f1ee0ed'} {{(pid=80245) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 08:31:41.681784 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.notifications [None req-22da7e2c-dcf0-43ae-ae7e-ad8f2f1ee0ed admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'fdf6ddfb5bd744c5ac2d9b12434229eb', 'request_id': 'req-22da7e2c-dcf0-43ae-ae7e-ad8f2f1ee0ed'} {{(pid=80245) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 08:31:41.688581 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.notifications [None req-22da7e2c-dcf0-43ae-ae7e-ad8f2f1ee0ed admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'fdf6ddfb5bd744c5ac2d9b12434229eb', 'request_id': 'req-22da7e2c-dcf0-43ae-ae7e-ad8f2f1ee0ed'} {{(pid=80245) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 08:31:41.703158 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.notifications [None req-22da7e2c-dcf0-43ae-ae7e-ad8f2f1ee0ed admin admin] The token cache is being invalidate because project fdf6ddfb5bd744c5ac2d9b12434229eb was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80245) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 21 08:31:41.703505 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.notifications [None req-22da7e2c-dcf0-43ae-ae7e-ad8f2f1ee0ed admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80245) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 08:31:41.704174 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.notifications [None req-22da7e2c-dcf0-43ae-ae7e-ad8f2f1ee0ed admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80245) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 08:31:41.709438 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: [pid: 80245|app: 0|req: 1070/2142] 10.222.0.47 () {58 vars in 1232 bytes} [Thu Jul 21 08:31:41 2022] DELETE /identity/v3/projects/fdf6ddfb5bd744c5ac2d9b12434229eb => generated 0 bytes in 212 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 08:31:41.711300 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77a8b7df-4309-4985-8e94-194432417c6f None None] REQUEST_METHOD: `POST` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:31:41.711462 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77a8b7df-4309-4985-8e94-194432417c6f None None] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:31:41.711840 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77a8b7df-4309-4985-8e94-194432417c6f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:31:41.748897 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.auth.core [None req-77a8b7df-4309-4985-8e94-194432417c6f None None] MFA Rules not processed for user `2dce40b8d74c41ec94cc36e6ff05d412`. Rule list: `[]` (Enabled: `True`). {{(pid=80244) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 08:31:41.750128 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.fernet_utils [None req-77a8b7df-4309-4985-8e94-194432417c6f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80244) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 08:31:41.791455 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG dbcounter [-] [80244] Writing DB stats keystone:SELECT=95,keystone:DELETE=4,keystone:INSERT=1 {{(pid=80244) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:31:41.832292 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 1073/2143] 10.222.0.47 () {58 vars in 957 bytes} [Thu Jul 21 08:31:41 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 08:31:41.841339 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01d59706-309d-4bc0-82a6-ca04eba8244e None None] REQUEST_METHOD: `POST` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:31:41.841632 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01d59706-309d-4bc0-82a6-ca04eba8244e None None] SCRIPT_NAME: `/identity` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:31:41.842014 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01d59706-309d-4bc0-82a6-ca04eba8244e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:31:41.862784 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.auth.core [None req-01d59706-309d-4bc0-82a6-ca04eba8244e None None] MFA Rules not processed for user `2dce40b8d74c41ec94cc36e6ff05d412`. Rule list: `[]` (Enabled: `True`). {{(pid=80245) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 08:31:41.866062 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.fernet_utils [None req-01d59706-309d-4bc0-82a6-ca04eba8244e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80245) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 08:31:41.874534 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: [pid: 80245|app: 0|req: 1071/2144] 10.222.0.47 () {58 vars in 957 bytes} [Thu Jul 21 08:31:41 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 08:31:41.879236 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77a8b7df-4309-4985-8e94-194432417c6f None None] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:31:41.891182 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77a8b7df-4309-4985-8e94-194432417c6f None None] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:31:41.896757 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84df4b6f-34fb-47ec-8118-19f6af676c2a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '42d15fc0515945acacbbffcf2356c7a5', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:31:41.897766 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84df4b6f-34fb-47ec-8118-19f6af676c2a admin admin] REQUEST_METHOD: `GET` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:31:41.898050 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84df4b6f-34fb-47ec-8118-19f6af676c2a admin admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:31:41.898395 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84df4b6f-34fb-47ec-8118-19f6af676c2a admin admin] PATH_INFO: `/v3/domains` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:31:41.899692 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84df4b6f-34fb-47ec-8118-19f6af676c2a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80244) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 08:31:41.900145 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84df4b6f-34fb-47ec-8118-19f6af676c2a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:31:41.902494 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84df4b6f-34fb-47ec-8118-19f6af676c2a admin admin] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:31:41.916430 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 1074/2145] 10.222.0.47 () {58 vars in 1152 bytes} [Thu Jul 21 08:31:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 21 08:31:41.920686 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01d59706-309d-4bc0-82a6-ca04eba8244e None None] Authenticating user token {{(pid=80245) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:31:41.931176 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01d59706-309d-4bc0-82a6-ca04eba8244e None None] Validating token access rules against request {{(pid=80245) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:31:41.945060 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6045687e-473d-4b29-9839-5db609e66ef7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '42d15fc0515945acacbbffcf2356c7a5', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80245) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:31:41.945731 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6045687e-473d-4b29-9839-5db609e66ef7 admin admin] REQUEST_METHOD: `POST` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:31:41.946020 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6045687e-473d-4b29-9839-5db609e66ef7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:31:41.946297 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6045687e-473d-4b29-9839-5db609e66ef7 admin admin] PATH_INFO: `/v3/projects` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:31:41.947004 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6045687e-473d-4b29-9839-5db609e66ef7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:31:41.949483 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6045687e-473d-4b29-9839-5db609e66ef7 admin admin] RBAC: Authorization granted {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:31:41.979178 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: [pid: 80245|app: 0|req: 1072/2146] 10.222.0.47 () {60 vars in 1152 bytes} [Thu Jul 21 08:31:41 2022] POST /identity/v3/projects => generated 377 bytes in 59 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 08:31:41.983158 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84df4b6f-34fb-47ec-8118-19f6af676c2a admin admin] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:31:41.993190 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84df4b6f-34fb-47ec-8118-19f6af676c2a admin admin] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:31:42.054967 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81d49d2f-5cc0-4bb8-a2b7-bcc963bc1385 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '42d15fc0515945acacbbffcf2356c7a5', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:31:42.055773 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81d49d2f-5cc0-4bb8-a2b7-bcc963bc1385 admin admin] REQUEST_METHOD: `POST` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:31:42.056063 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81d49d2f-5cc0-4bb8-a2b7-bcc963bc1385 admin admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:31:42.056343 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81d49d2f-5cc0-4bb8-a2b7-bcc963bc1385 admin admin] PATH_INFO: `/v3/users` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:31:42.057037 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81d49d2f-5cc0-4bb8-a2b7-bcc963bc1385 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:31:42.058998 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81d49d2f-5cc0-4bb8-a2b7-bcc963bc1385 admin admin] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:31:42.083101 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 1075/2147] 10.222.0.47 () {60 vars in 1143 bytes} [Thu Jul 21 08:31:41 2022] POST /identity/v3/users => generated 289 bytes in 100 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 21 08:31:42.087446 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6045687e-473d-4b29-9839-5db609e66ef7 admin admin] Authenticating user token {{(pid=80245) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:31:42.100969 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6045687e-473d-4b29-9839-5db609e66ef7 admin admin] Validating token access rules against request {{(pid=80245) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:31:42.106122 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ec9913d-721b-423a-918d-f297ea851fdc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '42d15fc0515945acacbbffcf2356c7a5', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80245) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:31:42.106945 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ec9913d-721b-423a-918d-f297ea851fdc admin admin] REQUEST_METHOD: `GET` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:31:42.107358 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ec9913d-721b-423a-918d-f297ea851fdc admin admin] SCRIPT_NAME: `/identity` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:31:42.107748 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ec9913d-721b-423a-918d-f297ea851fdc admin admin] PATH_INFO: `/v3/roles` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:31:42.108571 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ec9913d-721b-423a-918d-f297ea851fdc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:31:42.110238 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ec9913d-721b-423a-918d-f297ea851fdc admin admin] RBAC: Authorization granted {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:31:42.122413 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: [pid: 80245|app: 0|req: 1073/2148] 10.222.0.47 () {58 vars in 1121 bytes} [Thu Jul 21 08:31:42 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 08:31:42.126716 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81d49d2f-5cc0-4bb8-a2b7-bcc963bc1385 admin admin] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:31:42.139491 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81d49d2f-5cc0-4bb8-a2b7-bcc963bc1385 admin admin] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:31:42.146777 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d9242bf-ec40-4bee-ac62-b4bec956a789 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '42d15fc0515945acacbbffcf2356c7a5', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:31:42.147843 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d9242bf-ec40-4bee-ac62-b4bec956a789 admin admin] REQUEST_METHOD: `GET` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:31:42.148293 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d9242bf-ec40-4bee-ac62-b4bec956a789 admin admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:31:42.148721 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d9242bf-ec40-4bee-ac62-b4bec956a789 admin admin] PATH_INFO: `/v3/roles` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:31:42.149607 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d9242bf-ec40-4bee-ac62-b4bec956a789 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:31:42.151741 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d9242bf-ec40-4bee-ac62-b4bec956a789 admin admin] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:31:42.166989 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 1076/2149] 10.222.0.47 () {58 vars in 1121 bytes} [Thu Jul 21 08:31:42 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 08:31:42.168323 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ec9913d-721b-423a-918d-f297ea851fdc admin admin] Authenticating user token {{(pid=80245) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:31:42.181581 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ec9913d-721b-423a-918d-f297ea851fdc admin admin] Validating token access rules against request {{(pid=80245) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:31:42.187475 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-029e90ac-bff7-4271-adfb-a31ccfca002b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '42d15fc0515945acacbbffcf2356c7a5', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80245) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:31:42.188477 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-029e90ac-bff7-4271-adfb-a31ccfca002b admin admin] REQUEST_METHOD: `PUT` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:31:42.188836 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-029e90ac-bff7-4271-adfb-a31ccfca002b admin admin] SCRIPT_NAME: `/identity` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:31:42.189175 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-029e90ac-bff7-4271-adfb-a31ccfca002b admin admin] PATH_INFO: `/v3/projects/70fd29b7ea724938bcf4e1ef0597aee3/users/65ae2660fced4eed9a61c9779c28aff9/roles/3a074738221e4601b5205715ba87d240` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:31:42.211352 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-029e90ac-bff7-4271-adfb-a31ccfca002b admin admin] RBAC: Authorizing `identity:create_grant(project_id=70fd29b7ea724938bcf4e1ef0597aee3, user_id=65ae2660fced4eed9a61c9779c28aff9, role_id=3a074738221e4601b5205715ba87d240)` {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:31:42.215877 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-029e90ac-bff7-4271-adfb-a31ccfca002b admin admin] RBAC: Authorization granted {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:31:42.232241 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: [pid: 80245|app: 0|req: 1074/2150] 10.222.0.47 () {60 vars in 1482 bytes} [Thu Jul 21 08:31:42 2022] PUT /identity/v3/projects/70fd29b7ea724938bcf4e1ef0597aee3/users/65ae2660fced4eed9a61c9779c28aff9/roles/3a074738221e4601b5205715ba87d240 => generated 0 bytes in 65 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 21 08:31:42.245212 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-692002a4-9922-48e1-8130-59e547e99a2a None None] REQUEST_METHOD: `POST` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:31:42.245578 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-692002a4-9922-48e1-8130-59e547e99a2a None None] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:31:42.245991 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-692002a4-9922-48e1-8130-59e547e99a2a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:31:42.282115 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.auth.core [None req-692002a4-9922-48e1-8130-59e547e99a2a None None] MFA Rules not processed for user `65ae2660fced4eed9a61c9779c28aff9`. Rule list: `[]` (Enabled: `True`). {{(pid=80244) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 08:31:42.283766 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.fernet_utils [None req-692002a4-9922-48e1-8130-59e547e99a2a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80244) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 08:31:42.394001 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 1077/2151] 10.222.0.47 () {58 vars in 957 bytes} [Thu Jul 21 08:31:42 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 150 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 08:31:42.399090 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-029e90ac-bff7-4271-adfb-a31ccfca002b admin admin] Authenticating user token {{(pid=80245) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:31:42.473412 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG dbcounter [-] [80245] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80245) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:31:42.490960 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-029e90ac-bff7-4271-adfb-a31ccfca002b admin admin] Validating token access rules against request {{(pid=80245) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:31:42.497003 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e03ffbe3-6a1d-46cf-92c1-352cd340546f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '42d15fc0515945acacbbffcf2356c7a5', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80245) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:31:42.497680 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e03ffbe3-6a1d-46cf-92c1-352cd340546f admin admin] REQUEST_METHOD: `POST` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:31:42.497987 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e03ffbe3-6a1d-46cf-92c1-352cd340546f admin admin] SCRIPT_NAME: `/identity` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:31:42.498261 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e03ffbe3-6a1d-46cf-92c1-352cd340546f admin admin] PATH_INFO: `/v3/projects` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:31:42.498971 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e03ffbe3-6a1d-46cf-92c1-352cd340546f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:31:42.501803 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e03ffbe3-6a1d-46cf-92c1-352cd340546f admin admin] RBAC: Authorization granted {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:31:42.529450 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: [pid: 80245|app: 0|req: 1075/2152] 10.222.0.47 () {60 vars in 1152 bytes} [Thu Jul 21 08:31:42 2022] POST /identity/v3/projects => generated 377 bytes in 131 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 21 08:31:42.533610 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-692002a4-9922-48e1-8130-59e547e99a2a None None] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:31:42.544791 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-692002a4-9922-48e1-8130-59e547e99a2a None None] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:31:42.575011 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG dbcounter [-] [80244] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80244) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:31:42.605860 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc4fb5a0-0d03-4ae7-b0f1-f01833ec692b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '42d15fc0515945acacbbffcf2356c7a5', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:31:42.606678 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc4fb5a0-0d03-4ae7-b0f1-f01833ec692b admin admin] REQUEST_METHOD: `POST` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:31:42.606969 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc4fb5a0-0d03-4ae7-b0f1-f01833ec692b admin admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:31:42.607248 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc4fb5a0-0d03-4ae7-b0f1-f01833ec692b admin admin] PATH_INFO: `/v3/users` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:31:42.607982 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc4fb5a0-0d03-4ae7-b0f1-f01833ec692b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:31:42.609916 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc4fb5a0-0d03-4ae7-b0f1-f01833ec692b admin admin] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:31:42.635618 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 1078/2153] 10.222.0.47 () {60 vars in 1143 bytes} [Thu Jul 21 08:31:42 2022] POST /identity/v3/users => generated 289 bytes in 103 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 08:31:42.640945 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e03ffbe3-6a1d-46cf-92c1-352cd340546f admin admin] Authenticating user token {{(pid=80245) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:31:42.652056 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e03ffbe3-6a1d-46cf-92c1-352cd340546f admin admin] Validating token access rules against request {{(pid=80245) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:31:42.658657 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36ec25d5-c21d-4723-a80e-2d8033ff1558 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '42d15fc0515945acacbbffcf2356c7a5', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80245) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:31:42.659701 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36ec25d5-c21d-4723-a80e-2d8033ff1558 admin admin] REQUEST_METHOD: `GET` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:31:42.660053 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36ec25d5-c21d-4723-a80e-2d8033ff1558 admin admin] SCRIPT_NAME: `/identity` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:31:42.660272 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36ec25d5-c21d-4723-a80e-2d8033ff1558 admin admin] PATH_INFO: `/v3/roles` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:31:42.661283 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36ec25d5-c21d-4723-a80e-2d8033ff1558 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:31:42.663172 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36ec25d5-c21d-4723-a80e-2d8033ff1558 admin admin] RBAC: Authorization granted {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:31:42.682443 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: [pid: 80245|app: 0|req: 1076/2154] 10.222.0.47 () {58 vars in 1121 bytes} [Thu Jul 21 08:31:42 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 08:31:42.687667 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc4fb5a0-0d03-4ae7-b0f1-f01833ec692b admin admin] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:31:42.698932 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc4fb5a0-0d03-4ae7-b0f1-f01833ec692b admin admin] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:31:42.703968 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d426c622-0199-4993-ae29-b4701f720719 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '42d15fc0515945acacbbffcf2356c7a5', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:31:42.704655 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d426c622-0199-4993-ae29-b4701f720719 admin admin] REQUEST_METHOD: `GET` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:31:42.704939 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d426c622-0199-4993-ae29-b4701f720719 admin admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:31:42.705388 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d426c622-0199-4993-ae29-b4701f720719 admin admin] PATH_INFO: `/v3/roles` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:31:42.706422 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d426c622-0199-4993-ae29-b4701f720719 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:31:42.708101 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d426c622-0199-4993-ae29-b4701f720719 admin admin] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:31:42.718942 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 1079/2155] 10.222.0.47 () {58 vars in 1121 bytes} [Thu Jul 21 08:31:42 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 08:31:42.724152 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36ec25d5-c21d-4723-a80e-2d8033ff1558 admin admin] Authenticating user token {{(pid=80245) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:31:42.735685 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36ec25d5-c21d-4723-a80e-2d8033ff1558 admin admin] Validating token access rules against request {{(pid=80245) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:31:42.741009 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18d7dae5-9e3d-46f6-b54f-786e2ee54138 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '42d15fc0515945acacbbffcf2356c7a5', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80245) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:31:42.741911 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18d7dae5-9e3d-46f6-b54f-786e2ee54138 admin admin] REQUEST_METHOD: `PUT` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:31:42.742277 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18d7dae5-9e3d-46f6-b54f-786e2ee54138 admin admin] SCRIPT_NAME: `/identity` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:31:42.742456 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18d7dae5-9e3d-46f6-b54f-786e2ee54138 admin admin] PATH_INFO: `/v3/projects/60be14b4d3c04785a20e502f0a05216d/users/fac290d79709487baf46603654317731/roles/3a074738221e4601b5205715ba87d240` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:31:42.761423 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18d7dae5-9e3d-46f6-b54f-786e2ee54138 admin admin] RBAC: Authorizing `identity:create_grant(project_id=60be14b4d3c04785a20e502f0a05216d, user_id=fac290d79709487baf46603654317731, role_id=3a074738221e4601b5205715ba87d240)` {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:31:42.764237 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18d7dae5-9e3d-46f6-b54f-786e2ee54138 admin admin] RBAC: Authorization granted {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:31:42.779706 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: [pid: 80245|app: 0|req: 1077/2156] 10.222.0.47 () {60 vars in 1482 bytes} [Thu Jul 21 08:31:42 2022] PUT /identity/v3/projects/60be14b4d3c04785a20e502f0a05216d/users/fac290d79709487baf46603654317731/roles/3a074738221e4601b5205715ba87d240 => generated 0 bytes in 56 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 08:31:42.789750 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc3eaa3e-c861-48ff-9590-cc6b312afeda None None] REQUEST_METHOD: `POST` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:31:42.790042 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc3eaa3e-c861-48ff-9590-cc6b312afeda None None] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:31:42.790323 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc3eaa3e-c861-48ff-9590-cc6b312afeda None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:31:42.820658 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.auth.core [None req-cc3eaa3e-c861-48ff-9590-cc6b312afeda None None] MFA Rules not processed for user `fac290d79709487baf46603654317731`. Rule list: `[]` (Enabled: `True`). {{(pid=80244) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 08:31:42.821947 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.fernet_utils [None req-cc3eaa3e-c861-48ff-9590-cc6b312afeda None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80244) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 08:31:42.886256 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 1080/2157] 10.222.0.47 () {58 vars in 957 bytes} [Thu Jul 21 08:31:42 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 08:31:42.890979 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18d7dae5-9e3d-46f6-b54f-786e2ee54138 admin admin] Authenticating user token {{(pid=80245) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:31:42.952987 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18d7dae5-9e3d-46f6-b54f-786e2ee54138 admin admin] Validating token access rules against request {{(pid=80245) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:31:42.958021 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-add2a3e8-8f51-4348-9d9c-bd3e71239d93 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '42d15fc0515945acacbbffcf2356c7a5', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80245) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:31:42.958734 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-add2a3e8-8f51-4348-9d9c-bd3e71239d93 admin admin] REQUEST_METHOD: `POST` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:31:42.958998 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-add2a3e8-8f51-4348-9d9c-bd3e71239d93 admin admin] SCRIPT_NAME: `/identity` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:31:42.959482 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-add2a3e8-8f51-4348-9d9c-bd3e71239d93 admin admin] PATH_INFO: `/v3/projects` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:31:42.960313 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-add2a3e8-8f51-4348-9d9c-bd3e71239d93 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:31:42.963972 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-add2a3e8-8f51-4348-9d9c-bd3e71239d93 admin admin] RBAC: Authorization granted {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:31:42.991192 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: [pid: 80245|app: 0|req: 1078/2158] 10.222.0.47 () {60 vars in 1152 bytes} [Thu Jul 21 08:31:42 2022] POST /identity/v3/projects => generated 379 bytes in 101 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 08:31:42.995411 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc3eaa3e-c861-48ff-9590-cc6b312afeda None None] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:31:43.007115 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc3eaa3e-c861-48ff-9590-cc6b312afeda None None] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:31:43.069650 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aa593f8-d9f9-4573-b6a2-69862a64c52f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '42d15fc0515945acacbbffcf2356c7a5', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:31:43.070356 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aa593f8-d9f9-4573-b6a2-69862a64c52f admin admin] REQUEST_METHOD: `POST` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:31:43.070626 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aa593f8-d9f9-4573-b6a2-69862a64c52f admin admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:31:43.070905 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aa593f8-d9f9-4573-b6a2-69862a64c52f admin admin] PATH_INFO: `/v3/users` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:31:43.072125 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5aa593f8-d9f9-4573-b6a2-69862a64c52f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:31:43.073631 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5aa593f8-d9f9-4573-b6a2-69862a64c52f admin admin] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:31:43.095075 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 1081/2159] 10.222.0.47 () {60 vars in 1143 bytes} [Thu Jul 21 08:31:42 2022] POST /identity/v3/users => generated 296 bytes in 100 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 08:31:43.099019 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-add2a3e8-8f51-4348-9d9c-bd3e71239d93 admin admin] Authenticating user token {{(pid=80245) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:31:43.110547 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-add2a3e8-8f51-4348-9d9c-bd3e71239d93 admin admin] Validating token access rules against request {{(pid=80245) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:31:43.115219 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ee883a8-4635-44a8-8999-842333d3a940 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '42d15fc0515945acacbbffcf2356c7a5', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80245) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:31:43.115953 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ee883a8-4635-44a8-8999-842333d3a940 admin admin] REQUEST_METHOD: `GET` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:31:43.116239 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ee883a8-4635-44a8-8999-842333d3a940 admin admin] SCRIPT_NAME: `/identity` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:31:43.116596 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ee883a8-4635-44a8-8999-842333d3a940 admin admin] PATH_INFO: `/v3/roles` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:31:43.117307 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ee883a8-4635-44a8-8999-842333d3a940 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:31:43.118900 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ee883a8-4635-44a8-8999-842333d3a940 admin admin] RBAC: Authorization granted {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:31:43.129868 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: [pid: 80245|app: 0|req: 1079/2160] 10.222.0.47 () {58 vars in 1121 bytes} [Thu Jul 21 08:31:43 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 08:31:43.134374 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aa593f8-d9f9-4573-b6a2-69862a64c52f admin admin] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:31:43.144467 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aa593f8-d9f9-4573-b6a2-69862a64c52f admin admin] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:31:43.149100 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aa02e2c-b369-47b9-a4af-a02493a9219f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '42d15fc0515945acacbbffcf2356c7a5', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:31:43.149980 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aa02e2c-b369-47b9-a4af-a02493a9219f admin admin] REQUEST_METHOD: `PUT` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:31:43.150283 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aa02e2c-b369-47b9-a4af-a02493a9219f admin admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:31:43.150542 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aa02e2c-b369-47b9-a4af-a02493a9219f admin admin] PATH_INFO: `/v3/projects/e797f003b6134c0e88a78370830609ba/users/070eceecd25b42aaa8c02fffb8d2cbd3/roles/e29870f6e9e2411a9db0d5bf45f32ca5` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:31:43.165594 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2aa02e2c-b369-47b9-a4af-a02493a9219f admin admin] RBAC: Authorizing `identity:create_grant(project_id=e797f003b6134c0e88a78370830609ba, user_id=070eceecd25b42aaa8c02fffb8d2cbd3, role_id=e29870f6e9e2411a9db0d5bf45f32ca5)` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:31:43.168935 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2aa02e2c-b369-47b9-a4af-a02493a9219f admin admin] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:31:43.182077 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 1082/2161] 10.222.0.47 () {60 vars in 1482 bytes} [Thu Jul 21 08:31:43 2022] PUT /identity/v3/projects/e797f003b6134c0e88a78370830609ba/users/070eceecd25b42aaa8c02fffb8d2cbd3/roles/e29870f6e9e2411a9db0d5bf45f32ca5 => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 08:31:43.186199 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ee883a8-4635-44a8-8999-842333d3a940 admin admin] Authenticating user token {{(pid=80245) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:31:43.196317 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ee883a8-4635-44a8-8999-842333d3a940 admin admin] Validating token access rules against request {{(pid=80245) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:31:43.252239 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f799c2d4-52b2-4ba1-9834-124d189baf59 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '42d15fc0515945acacbbffcf2356c7a5', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80245) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:31:43.252838 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f799c2d4-52b2-4ba1-9834-124d189baf59 admin admin] REQUEST_METHOD: `GET` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:31:43.253122 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f799c2d4-52b2-4ba1-9834-124d189baf59 admin admin] SCRIPT_NAME: `/identity` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:31:43.253397 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f799c2d4-52b2-4ba1-9834-124d189baf59 admin admin] PATH_INFO: `/v3/roles` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:31:43.254383 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f799c2d4-52b2-4ba1-9834-124d189baf59 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:31:43.256029 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f799c2d4-52b2-4ba1-9834-124d189baf59 admin admin] RBAC: Authorization granted {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:31:43.265841 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: [pid: 80245|app: 0|req: 1080/2162] 10.222.0.47 () {58 vars in 1121 bytes} [Thu Jul 21 08:31:43 2022] GET /identity/v3/roles => generated 1420 bytes in 80 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 08:31:43.270086 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aa02e2c-b369-47b9-a4af-a02493a9219f admin admin] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:31:43.277204 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG dbcounter [-] [80244] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=80244) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:31:43.281662 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aa02e2c-b369-47b9-a4af-a02493a9219f admin admin] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:31:43.287495 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6866d2c0-f481-402e-803a-1e5787ffb411 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '42d15fc0515945acacbbffcf2356c7a5', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:31:43.288394 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6866d2c0-f481-402e-803a-1e5787ffb411 admin admin] REQUEST_METHOD: `PUT` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:31:43.288666 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6866d2c0-f481-402e-803a-1e5787ffb411 admin admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:31:43.288940 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6866d2c0-f481-402e-803a-1e5787ffb411 admin admin] PATH_INFO: `/v3/projects/e797f003b6134c0e88a78370830609ba/users/070eceecd25b42aaa8c02fffb8d2cbd3/roles/e29870f6e9e2411a9db0d5bf45f32ca5` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:31:43.290919 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6866d2c0-f481-402e-803a-1e5787ffb411 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e797f003b6134c0e88a78370830609ba, user_id=070eceecd25b42aaa8c02fffb8d2cbd3, role_id=e29870f6e9e2411a9db0d5bf45f32ca5)` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:31:43.294569 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6866d2c0-f481-402e-803a-1e5787ffb411 admin admin] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:31:43.304859 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 1083/2163] 10.222.0.47 () {60 vars in 1482 bytes} [Thu Jul 21 08:31:43 2022] PUT /identity/v3/projects/e797f003b6134c0e88a78370830609ba/users/070eceecd25b42aaa8c02fffb8d2cbd3/roles/e29870f6e9e2411a9db0d5bf45f32ca5 => generated 0 bytes in 35 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 08:31:43.314932 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7eee3df-76b3-4fc8-8e8d-d8aa9846ea4e None None] REQUEST_METHOD: `POST` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:31:43.315222 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7eee3df-76b3-4fc8-8e8d-d8aa9846ea4e None None] SCRIPT_NAME: `/identity` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:31:43.315536 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7eee3df-76b3-4fc8-8e8d-d8aa9846ea4e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:31:43.333289 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG dbcounter [-] [80245] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80245) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:31:43.343739 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.auth.core [None req-c7eee3df-76b3-4fc8-8e8d-d8aa9846ea4e None None] MFA Rules not processed for user `070eceecd25b42aaa8c02fffb8d2cbd3`. Rule list: `[]` (Enabled: `True`). {{(pid=80245) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 08:31:43.344942 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.fernet_utils [None req-c7eee3df-76b3-4fc8-8e8d-d8aa9846ea4e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80245) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 08:31:43.425520 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: [pid: 80245|app: 0|req: 1081/2164] 10.222.0.47 () {58 vars in 957 bytes} [Thu Jul 21 08:31:43 2022] POST /identity/v3/auth/tokens => generated 3313 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 08:31:43.436314 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6866d2c0-f481-402e-803a-1e5787ffb411 admin admin] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:31:43.437980 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.fernet_utils [None req-6866d2c0-f481-402e-803a-1e5787ffb411 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80244) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 08:31:43.503116 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6866d2c0-f481-402e-803a-1e5787ffb411 admin admin] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:31:43.508751 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d15b05ca-4e63-4a7b-86ed-e72ef95dfda9 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4f40410b7674687ab5362ac982248ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4faef13aa50e4e36b6c265fb1398fa14', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:31:43.509417 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d15b05ca-4e63-4a7b-86ed-e72ef95dfda9 service cinder] REQUEST_METHOD: `GET` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:31:43.509708 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d15b05ca-4e63-4a7b-86ed-e72ef95dfda9 service cinder] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:31:43.509987 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d15b05ca-4e63-4a7b-86ed-e72ef95dfda9 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:31:43.521035 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d15b05ca-4e63-4a7b-86ed-e72ef95dfda9 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:31:43.522518 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d15b05ca-4e63-4a7b-86ed-e72ef95dfda9 service cinder] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:31:43.535006 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 1084/2165] 10.222.0.47 () {60 vars in 1365 bytes} [Thu Jul 21 08:31:43 2022] GET /identity/v3/auth/tokens => generated 3313 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 08:31:43.574970 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7eee3df-76b3-4fc8-8e8d-d8aa9846ea4e None None] Authenticating user token {{(pid=80245) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:31:43.576558 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.fernet_utils [None req-c7eee3df-76b3-4fc8-8e8d-d8aa9846ea4e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80245) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 08:31:43.592511 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7eee3df-76b3-4fc8-8e8d-d8aa9846ea4e None None] Validating token access rules against request {{(pid=80245) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:31:43.596471 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f90c80a-ea9f-4f10-b58f-447776d5d4b0 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4f40410b7674687ab5362ac982248ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4faef13aa50e4e36b6c265fb1398fa14', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80245) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:31:43.597128 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f90c80a-ea9f-4f10-b58f-447776d5d4b0 service cinder] REQUEST_METHOD: `GET` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:31:43.597416 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f90c80a-ea9f-4f10-b58f-447776d5d4b0 service cinder] SCRIPT_NAME: `/identity` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:31:43.597696 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f90c80a-ea9f-4f10-b58f-447776d5d4b0 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:31:43.665359 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f90c80a-ea9f-4f10-b58f-447776d5d4b0 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:31:43.666801 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f90c80a-ea9f-4f10-b58f-447776d5d4b0 service cinder] RBAC: Authorization granted {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:31:43.675937 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: [pid: 80245|app: 0|req: 1082/2166] 10.222.0.47 () {60 vars in 1365 bytes} [Thu Jul 21 08:31:43 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 102 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 08:31:47.208776 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d15b05ca-4e63-4a7b-86ed-e72ef95dfda9 service cinder] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:31:47.210381 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.fernet_utils [None req-d15b05ca-4e63-4a7b-86ed-e72ef95dfda9 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80244) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 08:31:47.281141 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d15b05ca-4e63-4a7b-86ed-e72ef95dfda9 service cinder] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:31:47.286086 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-057a0681-7134-4d9f-b0c7-943d2c7487fc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '42d15fc0515945acacbbffcf2356c7a5', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:31:47.286730 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-057a0681-7134-4d9f-b0c7-943d2c7487fc admin admin] REQUEST_METHOD: `DELETE` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:31:47.287019 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-057a0681-7134-4d9f-b0c7-943d2c7487fc admin admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:31:47.287300 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-057a0681-7134-4d9f-b0c7-943d2c7487fc admin admin] PATH_INFO: `/v3/users/c0c589f80a654af3b9011321bc6a8dd7` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:31:47.289617 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-057a0681-7134-4d9f-b0c7-943d2c7487fc admin admin] RBAC: Authorizing `identity:delete_user(user_id=c0c589f80a654af3b9011321bc6a8dd7)` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:31:47.290491 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-057a0681-7134-4d9f-b0c7-943d2c7487fc admin admin] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:31:47.360905 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.notifications [None req-057a0681-7134-4d9f-b0c7-943d2c7487fc admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c0c589f80a654af3b9011321bc6a8dd7'} {{(pid=80244) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 08:31:47.370261 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.notifications [None req-057a0681-7134-4d9f-b0c7-943d2c7487fc admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c0c589f80a654af3b9011321bc6a8dd7'} {{(pid=80244) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 08:31:47.386688 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.notifications [None req-057a0681-7134-4d9f-b0c7-943d2c7487fc admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c0c589f80a654af3b9011321bc6a8dd7'} {{(pid=80244) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 08:31:47.387460 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.notifications [None req-057a0681-7134-4d9f-b0c7-943d2c7487fc admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c0c589f80a654af3b9011321bc6a8dd7'} {{(pid=80244) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 08:31:47.396307 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.notifications [None req-057a0681-7134-4d9f-b0c7-943d2c7487fc admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c0c589f80a654af3b9011321bc6a8dd7'} {{(pid=80244) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 08:31:47.404270 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 1085/2167] 10.222.0.47 () {58 vars in 1223 bytes} [Thu Jul 21 08:31:47 2022] DELETE /identity/v3/users/c0c589f80a654af3b9011321bc6a8dd7 => generated 0 bytes in 198 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 08:31:47.409107 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b4c34dc-e2f1-410e-b13d-8f17ff3914aa None None] REQUEST_METHOD: `POST` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:31:47.410001 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b4c34dc-e2f1-410e-b13d-8f17ff3914aa None None] SCRIPT_NAME: `/identity` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:31:47.410584 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b4c34dc-e2f1-410e-b13d-8f17ff3914aa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:31:47.431983 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.auth.core [None req-8b4c34dc-e2f1-410e-b13d-8f17ff3914aa None None] MFA Rules not processed for user `2dce40b8d74c41ec94cc36e6ff05d412`. Rule list: `[]` (Enabled: `True`). {{(pid=80245) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 08:31:47.435598 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.fernet_utils [None req-8b4c34dc-e2f1-410e-b13d-8f17ff3914aa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80245) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 08:31:47.515485 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: [pid: 80245|app: 0|req: 1083/2168] 10.222.0.47 () {58 vars in 957 bytes} [Thu Jul 21 08:31:47 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 08:31:47.524240 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-057a0681-7134-4d9f-b0c7-943d2c7487fc admin admin] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:31:47.525687 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.fernet_utils [None req-057a0681-7134-4d9f-b0c7-943d2c7487fc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80244) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 08:31:47.542819 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG dbcounter [-] [80244] Writing DB stats keystone:SELECT=88,keystone:INSERT=2,keystone:DELETE=10 {{(pid=80244) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:31:47.590802 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-057a0681-7134-4d9f-b0c7-943d2c7487fc admin admin] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:31:47.594460 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a7b744a-ca0d-44ae-8b95-e4bd4407fc54 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20ca3c3bd3774691b3374cfbfec6e494', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4faef13aa50e4e36b6c265fb1398fa14', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:31:47.595110 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a7b744a-ca0d-44ae-8b95-e4bd4407fc54 service neutron] REQUEST_METHOD: `GET` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:31:47.595475 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a7b744a-ca0d-44ae-8b95-e4bd4407fc54 service neutron] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:31:47.595766 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a7b744a-ca0d-44ae-8b95-e4bd4407fc54 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:31:47.605848 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a7b744a-ca0d-44ae-8b95-e4bd4407fc54 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:31:47.607426 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a7b744a-ca0d-44ae-8b95-e4bd4407fc54 service neutron] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:31:47.618187 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 1086/2169] 10.222.0.47 () {60 vars in 1365 bytes} [Thu Jul 21 08:31:47 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 08:31:47.660786 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b4c34dc-e2f1-410e-b13d-8f17ff3914aa None None] Authenticating user token {{(pid=80245) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:31:47.662389 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.fernet_utils [None req-8b4c34dc-e2f1-410e-b13d-8f17ff3914aa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80245) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 08:31:47.678046 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b4c34dc-e2f1-410e-b13d-8f17ff3914aa None None] Validating token access rules against request {{(pid=80245) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:31:47.684104 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b111cbb-2e09-454c-8d8b-9e5b0e5a10c8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '42d15fc0515945acacbbffcf2356c7a5', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80245) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:31:47.685549 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b111cbb-2e09-454c-8d8b-9e5b0e5a10c8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:31:47.686021 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b111cbb-2e09-454c-8d8b-9e5b0e5a10c8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:31:47.686385 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b111cbb-2e09-454c-8d8b-9e5b0e5a10c8 admin admin] PATH_INFO: `/v3/users/65ae2660fced4eed9a61c9779c28aff9` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:31:47.687713 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b111cbb-2e09-454c-8d8b-9e5b0e5a10c8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=65ae2660fced4eed9a61c9779c28aff9)` {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:31:47.689791 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b111cbb-2e09-454c-8d8b-9e5b0e5a10c8 admin admin] RBAC: Authorization granted {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:31:47.724464 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG dbcounter [-] [80245] Writing DB stats keystone:SELECT=95,keystone:DELETE=5 {{(pid=80245) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:31:47.755486 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.notifications [None req-6b111cbb-2e09-454c-8d8b-9e5b0e5a10c8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '65ae2660fced4eed9a61c9779c28aff9'} {{(pid=80245) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 08:31:47.767834 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.notifications [None req-6b111cbb-2e09-454c-8d8b-9e5b0e5a10c8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '65ae2660fced4eed9a61c9779c28aff9'} {{(pid=80245) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 08:31:47.783100 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.notifications [None req-6b111cbb-2e09-454c-8d8b-9e5b0e5a10c8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '65ae2660fced4eed9a61c9779c28aff9'} {{(pid=80245) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 08:31:47.790539 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.notifications [None req-6b111cbb-2e09-454c-8d8b-9e5b0e5a10c8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '65ae2660fced4eed9a61c9779c28aff9'} {{(pid=80245) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 08:31:47.791084 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.notifications [None req-6b111cbb-2e09-454c-8d8b-9e5b0e5a10c8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '65ae2660fced4eed9a61c9779c28aff9'} {{(pid=80245) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 08:31:47.798209 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: [pid: 80245|app: 0|req: 1084/2170] 10.222.0.47 () {58 vars in 1223 bytes} [Thu Jul 21 08:31:47 2022] DELETE /identity/v3/users/65ae2660fced4eed9a61c9779c28aff9 => generated 0 bytes in 138 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 08:31:47.802713 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bff56115-de8b-4ed5-9049-c9c4878bd4d8 None None] REQUEST_METHOD: `POST` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:31:47.803511 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bff56115-de8b-4ed5-9049-c9c4878bd4d8 None None] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:31:47.803511 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bff56115-de8b-4ed5-9049-c9c4878bd4d8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:31:47.824868 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.auth.core [None req-bff56115-de8b-4ed5-9049-c9c4878bd4d8 None None] MFA Rules not processed for user `2dce40b8d74c41ec94cc36e6ff05d412`. Rule list: `[]` (Enabled: `True`). {{(pid=80244) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 08:31:47.828008 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.fernet_utils [None req-bff56115-de8b-4ed5-9049-c9c4878bd4d8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80244) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 08:31:47.912499 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 1087/2171] 10.222.0.47 () {58 vars in 957 bytes} [Thu Jul 21 08:31:47 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 21 08:31:47.924521 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b111cbb-2e09-454c-8d8b-9e5b0e5a10c8 admin admin] Authenticating user token {{(pid=80245) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:31:47.926414 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.fernet_utils [None req-6b111cbb-2e09-454c-8d8b-9e5b0e5a10c8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80245) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 08:31:47.954561 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bff56115-de8b-4ed5-9049-c9c4878bd4d8 None None] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:31:47.956358 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.fernet_utils [None req-bff56115-de8b-4ed5-9049-c9c4878bd4d8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80244) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 08:31:47.978275 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bff56115-de8b-4ed5-9049-c9c4878bd4d8 None None] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:31:47.986663 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c88544e9-2c87-4e89-bc52-6ac917ec40c3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '42d15fc0515945acacbbffcf2356c7a5', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:31:47.988021 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c88544e9-2c87-4e89-bc52-6ac917ec40c3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:31:47.988312 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c88544e9-2c87-4e89-bc52-6ac917ec40c3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:31:47.988736 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c88544e9-2c87-4e89-bc52-6ac917ec40c3 admin admin] PATH_INFO: `/v3/projects/d3da643296d0420897bcfb8500f2c7a8` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:31:47.989817 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c88544e9-2c87-4e89-bc52-6ac917ec40c3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d3da643296d0420897bcfb8500f2c7a8)` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:31:47.991994 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c88544e9-2c87-4e89-bc52-6ac917ec40c3 admin admin] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:31:48.009760 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b111cbb-2e09-454c-8d8b-9e5b0e5a10c8 admin admin] Validating token access rules against request {{(pid=80245) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:31:48.014834 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-803dcd19-d8fc-47fd-9ade-d4b4a1f43796 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20ca3c3bd3774691b3374cfbfec6e494', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4faef13aa50e4e36b6c265fb1398fa14', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80245) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:31:48.015712 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-803dcd19-d8fc-47fd-9ade-d4b4a1f43796 service neutron] REQUEST_METHOD: `GET` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:31:48.016142 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-803dcd19-d8fc-47fd-9ade-d4b4a1f43796 service neutron] SCRIPT_NAME: `/identity` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:31:48.016429 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-803dcd19-d8fc-47fd-9ade-d4b4a1f43796 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:31:48.029845 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-803dcd19-d8fc-47fd-9ade-d4b4a1f43796 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:31:48.031270 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-803dcd19-d8fc-47fd-9ade-d4b4a1f43796 service neutron] RBAC: Authorization granted {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:31:48.045498 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: [pid: 80245|app: 0|req: 1085/2172] 10.222.0.47 () {60 vars in 1365 bytes} [Thu Jul 21 08:31:47 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 122 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 08:31:48.053668 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.notifications [None req-c88544e9-2c87-4e89-bc52-6ac917ec40c3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd3da643296d0420897bcfb8500f2c7a8', 'request_id': 'req-c88544e9-2c87-4e89-bc52-6ac917ec40c3'} {{(pid=80244) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 08:31:48.070534 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.notifications [None req-c88544e9-2c87-4e89-bc52-6ac917ec40c3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd3da643296d0420897bcfb8500f2c7a8', 'request_id': 'req-c88544e9-2c87-4e89-bc52-6ac917ec40c3'} {{(pid=80244) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 08:31:48.078943 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.notifications [None req-c88544e9-2c87-4e89-bc52-6ac917ec40c3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd3da643296d0420897bcfb8500f2c7a8', 'request_id': 'req-c88544e9-2c87-4e89-bc52-6ac917ec40c3'} {{(pid=80244) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 08:31:48.112168 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.notifications [None req-c88544e9-2c87-4e89-bc52-6ac917ec40c3 admin admin] The token cache is being invalidate because project d3da643296d0420897bcfb8500f2c7a8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80244) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 21 08:31:48.112468 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.notifications [None req-c88544e9-2c87-4e89-bc52-6ac917ec40c3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80244) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 08:31:48.113512 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.notifications [None req-c88544e9-2c87-4e89-bc52-6ac917ec40c3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80244) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 08:31:48.115190 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 1088/2173] 10.222.0.47 () {58 vars in 1232 bytes} [Thu Jul 21 08:31:47 2022] DELETE /identity/v3/projects/d3da643296d0420897bcfb8500f2c7a8 => generated 0 bytes in 161 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 21 08:31:48.119675 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-803dcd19-d8fc-47fd-9ade-d4b4a1f43796 service neutron] Authenticating user token {{(pid=80245) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:31:48.138466 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-803dcd19-d8fc-47fd-9ade-d4b4a1f43796 service neutron] Validating token access rules against request {{(pid=80245) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:31:48.216411 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70b06ff4-dc2f-4aef-9fa6-df64ab7de5d6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '42d15fc0515945acacbbffcf2356c7a5', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80245) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:31:48.217476 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70b06ff4-dc2f-4aef-9fa6-df64ab7de5d6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:31:48.217952 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70b06ff4-dc2f-4aef-9fa6-df64ab7de5d6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:31:48.218212 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70b06ff4-dc2f-4aef-9fa6-df64ab7de5d6 admin admin] PATH_INFO: `/v3/users/4a0ef7b7d1314f889655a46a5b570e31` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:31:48.219588 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70b06ff4-dc2f-4aef-9fa6-df64ab7de5d6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=4a0ef7b7d1314f889655a46a5b570e31)` {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:31:48.221754 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70b06ff4-dc2f-4aef-9fa6-df64ab7de5d6 admin admin] RBAC: Authorization granted {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:31:48.343831 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.notifications [None req-70b06ff4-dc2f-4aef-9fa6-df64ab7de5d6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4a0ef7b7d1314f889655a46a5b570e31'} {{(pid=80245) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 08:31:48.347212 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG dbcounter [-] [80245] Writing DB stats keystone:SELECT=86,keystone:DELETE=12,keystone:INSERT=2 {{(pid=80245) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:31:48.358575 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.notifications [None req-70b06ff4-dc2f-4aef-9fa6-df64ab7de5d6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4a0ef7b7d1314f889655a46a5b570e31'} {{(pid=80245) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 08:31:48.382558 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.notifications [None req-70b06ff4-dc2f-4aef-9fa6-df64ab7de5d6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4a0ef7b7d1314f889655a46a5b570e31'} {{(pid=80245) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 08:31:48.392365 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.notifications [None req-70b06ff4-dc2f-4aef-9fa6-df64ab7de5d6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4a0ef7b7d1314f889655a46a5b570e31'} {{(pid=80245) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 08:31:48.393398 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.notifications [None req-70b06ff4-dc2f-4aef-9fa6-df64ab7de5d6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4a0ef7b7d1314f889655a46a5b570e31'} {{(pid=80245) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 08:31:48.400818 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: [pid: 80245|app: 0|req: 1086/2174] 10.222.0.47 () {58 vars in 1223 bytes} [Thu Jul 21 08:31:48 2022] DELETE /identity/v3/users/4a0ef7b7d1314f889655a46a5b570e31 => generated 0 bytes in 282 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 21 08:31:48.548945 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c88544e9-2c87-4e89-bc52-6ac917ec40c3 admin admin] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:31:48.551350 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.fernet_utils [None req-c88544e9-2c87-4e89-bc52-6ac917ec40c3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80244) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 08:31:48.589066 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c88544e9-2c87-4e89-bc52-6ac917ec40c3 admin admin] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:31:48.671421 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG dbcounter [-] [80244] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80244) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:31:48.695651 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0ac1067-8409-4f7d-ae91-f9fd9a087bf1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '42d15fc0515945acacbbffcf2356c7a5', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:31:48.696524 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0ac1067-8409-4f7d-ae91-f9fd9a087bf1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:31:48.696831 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0ac1067-8409-4f7d-ae91-f9fd9a087bf1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:31:48.697168 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0ac1067-8409-4f7d-ae91-f9fd9a087bf1 admin admin] PATH_INFO: `/v3/projects/70fd29b7ea724938bcf4e1ef0597aee3` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:31:48.698348 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0ac1067-8409-4f7d-ae91-f9fd9a087bf1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=70fd29b7ea724938bcf4e1ef0597aee3)` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:31:48.700551 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0ac1067-8409-4f7d-ae91-f9fd9a087bf1 admin admin] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:31:48.753959 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.notifications [None req-e0ac1067-8409-4f7d-ae91-f9fd9a087bf1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '70fd29b7ea724938bcf4e1ef0597aee3', 'request_id': 'req-e0ac1067-8409-4f7d-ae91-f9fd9a087bf1'} {{(pid=80244) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 08:31:48.762165 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.notifications [None req-e0ac1067-8409-4f7d-ae91-f9fd9a087bf1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '70fd29b7ea724938bcf4e1ef0597aee3', 'request_id': 'req-e0ac1067-8409-4f7d-ae91-f9fd9a087bf1'} {{(pid=80244) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 08:31:48.769957 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.notifications [None req-e0ac1067-8409-4f7d-ae91-f9fd9a087bf1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '70fd29b7ea724938bcf4e1ef0597aee3', 'request_id': 'req-e0ac1067-8409-4f7d-ae91-f9fd9a087bf1'} {{(pid=80244) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 08:31:48.780066 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70b06ff4-dc2f-4aef-9fa6-df64ab7de5d6 admin admin] Authenticating user token {{(pid=80245) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:31:48.781714 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.fernet_utils [None req-70b06ff4-dc2f-4aef-9fa6-df64ab7de5d6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80245) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 08:31:48.782769 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.notifications [None req-e0ac1067-8409-4f7d-ae91-f9fd9a087bf1 admin admin] The token cache is being invalidate because project 70fd29b7ea724938bcf4e1ef0597aee3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80244) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 21 08:31:48.783064 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.notifications [None req-e0ac1067-8409-4f7d-ae91-f9fd9a087bf1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80244) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 08:31:48.783749 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.notifications [None req-e0ac1067-8409-4f7d-ae91-f9fd9a087bf1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80244) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 08:31:48.786588 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 1089/2175] 10.222.0.47 () {58 vars in 1232 bytes} [Thu Jul 21 08:31:48 2022] DELETE /identity/v3/projects/70fd29b7ea724938bcf4e1ef0597aee3 => generated 0 bytes in 238 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 21 08:31:48.791726 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0ac1067-8409-4f7d-ae91-f9fd9a087bf1 admin admin] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:31:48.794168 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.fernet_utils [None req-e0ac1067-8409-4f7d-ae91-f9fd9a087bf1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80244) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 08:31:48.796732 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70b06ff4-dc2f-4aef-9fa6-df64ab7de5d6 admin admin] Validating token access rules against request {{(pid=80245) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:31:48.863566 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5e13587-40d0-4668-8d47-b6e8856d852d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '42d15fc0515945acacbbffcf2356c7a5', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80245) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:31:48.864081 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0ac1067-8409-4f7d-ae91-f9fd9a087bf1 admin admin] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:31:48.864443 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5e13587-40d0-4668-8d47-b6e8856d852d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:31:48.864727 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5e13587-40d0-4668-8d47-b6e8856d852d admin admin] SCRIPT_NAME: `/identity` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:31:48.865005 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5e13587-40d0-4668-8d47-b6e8856d852d admin admin] PATH_INFO: `/v3/projects/844015ffe3af4fe891a9d367a0376fbe` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:31:48.866197 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5e13587-40d0-4668-8d47-b6e8856d852d admin admin] RBAC: Authorizing `identity:delete_project(project_id=844015ffe3af4fe891a9d367a0376fbe)` {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:31:48.868331 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5e13587-40d0-4668-8d47-b6e8856d852d admin admin] RBAC: Authorization granted {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:31:48.868873 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6528bcd8-2767-4e37-8ab2-8d52a9789cf6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '42d15fc0515945acacbbffcf2356c7a5', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:31:48.869755 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6528bcd8-2767-4e37-8ab2-8d52a9789cf6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:31:48.870066 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6528bcd8-2767-4e37-8ab2-8d52a9789cf6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:31:48.870897 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6528bcd8-2767-4e37-8ab2-8d52a9789cf6 admin admin] PATH_INFO: `/v3/users/fac290d79709487baf46603654317731` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:31:48.872149 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6528bcd8-2767-4e37-8ab2-8d52a9789cf6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=fac290d79709487baf46603654317731)` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:31:48.874079 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6528bcd8-2767-4e37-8ab2-8d52a9789cf6 admin admin] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:31:48.927228 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.notifications [None req-c5e13587-40d0-4668-8d47-b6e8856d852d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '844015ffe3af4fe891a9d367a0376fbe', 'request_id': 'req-c5e13587-40d0-4668-8d47-b6e8856d852d'} {{(pid=80245) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 08:31:48.930355 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.notifications [None req-6528bcd8-2767-4e37-8ab2-8d52a9789cf6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'fac290d79709487baf46603654317731'} {{(pid=80244) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 08:31:48.934332 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.notifications [None req-c5e13587-40d0-4668-8d47-b6e8856d852d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '844015ffe3af4fe891a9d367a0376fbe', 'request_id': 'req-c5e13587-40d0-4668-8d47-b6e8856d852d'} {{(pid=80245) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 08:31:48.939317 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.notifications [None req-6528bcd8-2767-4e37-8ab2-8d52a9789cf6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'fac290d79709487baf46603654317731'} {{(pid=80244) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 08:31:48.941323 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.notifications [None req-c5e13587-40d0-4668-8d47-b6e8856d852d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '844015ffe3af4fe891a9d367a0376fbe', 'request_id': 'req-c5e13587-40d0-4668-8d47-b6e8856d852d'} {{(pid=80245) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 08:31:48.954847 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.notifications [None req-6528bcd8-2767-4e37-8ab2-8d52a9789cf6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'fac290d79709487baf46603654317731'} {{(pid=80244) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 08:31:48.955587 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.notifications [None req-6528bcd8-2767-4e37-8ab2-8d52a9789cf6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'fac290d79709487baf46603654317731'} {{(pid=80244) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 08:31:48.956129 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.notifications [None req-c5e13587-40d0-4668-8d47-b6e8856d852d admin admin] The token cache is being invalidate because project 844015ffe3af4fe891a9d367a0376fbe was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80245) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 21 08:31:48.956452 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.notifications [None req-c5e13587-40d0-4668-8d47-b6e8856d852d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80245) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 08:31:48.956842 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG dbcounter [-] [80244] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=80244) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:31:48.957080 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.notifications [None req-c5e13587-40d0-4668-8d47-b6e8856d852d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80245) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 08:31:48.959016 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: [pid: 80245|app: 0|req: 1087/2176] 10.222.0.47 () {58 vars in 1232 bytes} [Thu Jul 21 08:31:48 2022] DELETE /identity/v3/projects/844015ffe3af4fe891a9d367a0376fbe => generated 0 bytes in 179 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 21 08:31:48.962896 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5e13587-40d0-4668-8d47-b6e8856d852d admin admin] Authenticating user token {{(pid=80245) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:31:48.964306 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.fernet_utils [None req-c5e13587-40d0-4668-8d47-b6e8856d852d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80245) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 08:31:48.965107 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.notifications [None req-6528bcd8-2767-4e37-8ab2-8d52a9789cf6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'fac290d79709487baf46603654317731'} {{(pid=80244) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 08:31:48.973374 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 1090/2177] 10.222.0.47 () {58 vars in 1223 bytes} [Thu Jul 21 08:31:48 2022] DELETE /identity/v3/users/fac290d79709487baf46603654317731 => generated 0 bytes in 182 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 21 08:31:49.034931 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG dbcounter [-] [80245] Writing DB stats keystone:DELETE=10,keystone:SELECT=89,keystone:INSERT=1 {{(pid=80245) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:31:49.038745 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5e13587-40d0-4668-8d47-b6e8856d852d admin admin] Validating token access rules against request {{(pid=80245) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:31:49.109677 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99fb5644-aab5-47b2-ab1b-31df3fda61fc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '42d15fc0515945acacbbffcf2356c7a5', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80245) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:31:49.110703 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99fb5644-aab5-47b2-ab1b-31df3fda61fc admin admin] REQUEST_METHOD: `DELETE` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:31:49.110996 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99fb5644-aab5-47b2-ab1b-31df3fda61fc admin admin] SCRIPT_NAME: `/identity` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:31:49.111270 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99fb5644-aab5-47b2-ab1b-31df3fda61fc admin admin] PATH_INFO: `/v3/users/f5ee6d78b3ff443ebf34708cbac33585` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:31:49.112604 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99fb5644-aab5-47b2-ab1b-31df3fda61fc admin admin] RBAC: Authorizing `identity:delete_user(user_id=f5ee6d78b3ff443ebf34708cbac33585)` {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:31:49.114587 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99fb5644-aab5-47b2-ab1b-31df3fda61fc admin admin] RBAC: Authorization granted {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:31:49.182142 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.notifications [None req-99fb5644-aab5-47b2-ab1b-31df3fda61fc admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f5ee6d78b3ff443ebf34708cbac33585'} {{(pid=80245) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 08:31:49.192137 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.notifications [None req-99fb5644-aab5-47b2-ab1b-31df3fda61fc admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f5ee6d78b3ff443ebf34708cbac33585'} {{(pid=80245) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 08:31:49.206485 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.notifications [None req-99fb5644-aab5-47b2-ab1b-31df3fda61fc admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f5ee6d78b3ff443ebf34708cbac33585'} {{(pid=80245) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 08:31:49.213540 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.notifications [None req-99fb5644-aab5-47b2-ab1b-31df3fda61fc admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f5ee6d78b3ff443ebf34708cbac33585'} {{(pid=80245) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 08:31:49.214809 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.notifications [None req-99fb5644-aab5-47b2-ab1b-31df3fda61fc admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f5ee6d78b3ff443ebf34708cbac33585'} {{(pid=80245) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 08:31:49.221465 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: [pid: 80245|app: 0|req: 1088/2178] 10.222.0.47 () {58 vars in 1223 bytes} [Thu Jul 21 08:31:48 2022] DELETE /identity/v3/users/f5ee6d78b3ff443ebf34708cbac33585 => generated 0 bytes in 259 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 21 08:31:49.284971 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6528bcd8-2767-4e37-8ab2-8d52a9789cf6 admin admin] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:31:49.286740 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.fernet_utils [None req-6528bcd8-2767-4e37-8ab2-8d52a9789cf6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80244) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 08:31:49.303315 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6528bcd8-2767-4e37-8ab2-8d52a9789cf6 admin admin] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:31:49.371816 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c97cb467-a687-4612-8ea0-5b18d9757b6e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '42d15fc0515945acacbbffcf2356c7a5', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:31:49.373132 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c97cb467-a687-4612-8ea0-5b18d9757b6e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:31:49.373493 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c97cb467-a687-4612-8ea0-5b18d9757b6e admin admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:31:49.373838 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c97cb467-a687-4612-8ea0-5b18d9757b6e admin admin] PATH_INFO: `/v3/projects/60be14b4d3c04785a20e502f0a05216d` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:31:49.375636 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c97cb467-a687-4612-8ea0-5b18d9757b6e admin admin] RBAC: Authorizing `identity:delete_project(project_id=60be14b4d3c04785a20e502f0a05216d)` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:31:49.377586 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c97cb467-a687-4612-8ea0-5b18d9757b6e admin admin] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:31:49.449606 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.notifications [None req-c97cb467-a687-4612-8ea0-5b18d9757b6e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '60be14b4d3c04785a20e502f0a05216d', 'request_id': 'req-c97cb467-a687-4612-8ea0-5b18d9757b6e'} {{(pid=80244) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 08:31:49.461054 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.notifications [None req-c97cb467-a687-4612-8ea0-5b18d9757b6e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '60be14b4d3c04785a20e502f0a05216d', 'request_id': 'req-c97cb467-a687-4612-8ea0-5b18d9757b6e'} {{(pid=80244) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 08:31:49.470308 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.notifications [None req-c97cb467-a687-4612-8ea0-5b18d9757b6e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '60be14b4d3c04785a20e502f0a05216d', 'request_id': 'req-c97cb467-a687-4612-8ea0-5b18d9757b6e'} {{(pid=80244) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 08:31:49.487470 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.notifications [None req-c97cb467-a687-4612-8ea0-5b18d9757b6e admin admin] The token cache is being invalidate because project 60be14b4d3c04785a20e502f0a05216d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80244) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 21 08:31:49.487767 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.notifications [None req-c97cb467-a687-4612-8ea0-5b18d9757b6e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80244) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 08:31:49.488772 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.notifications [None req-c97cb467-a687-4612-8ea0-5b18d9757b6e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80244) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 08:31:49.490637 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 1091/2179] 10.222.0.47 () {58 vars in 1232 bytes} [Thu Jul 21 08:31:49 2022] DELETE /identity/v3/projects/60be14b4d3c04785a20e502f0a05216d => generated 0 bytes in 206 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 08:31:49.494714 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99fb5644-aab5-47b2-ab1b-31df3fda61fc admin admin] Authenticating user token {{(pid=80245) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:31:49.496396 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.fernet_utils [None req-99fb5644-aab5-47b2-ab1b-31df3fda61fc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80245) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 08:31:49.511570 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99fb5644-aab5-47b2-ab1b-31df3fda61fc admin admin] Validating token access rules against request {{(pid=80245) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:31:49.581524 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daf746b5-96e4-4f7a-bafe-e3e1138d5cb0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '42d15fc0515945acacbbffcf2356c7a5', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80245) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:31:49.582575 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daf746b5-96e4-4f7a-bafe-e3e1138d5cb0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:31:49.583106 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daf746b5-96e4-4f7a-bafe-e3e1138d5cb0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:31:49.583220 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daf746b5-96e4-4f7a-bafe-e3e1138d5cb0 admin admin] PATH_INFO: `/v3/users/070eceecd25b42aaa8c02fffb8d2cbd3` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:31:49.584429 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daf746b5-96e4-4f7a-bafe-e3e1138d5cb0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=070eceecd25b42aaa8c02fffb8d2cbd3)` {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:31:49.586378 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daf746b5-96e4-4f7a-bafe-e3e1138d5cb0 admin admin] RBAC: Authorization granted {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:31:49.621402 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG dbcounter [-] [80245] Writing DB stats keystone:SELECT=85,keystone:DELETE=14,keystone:INSERT=1 {{(pid=80245) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:31:49.676870 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.notifications [None req-daf746b5-96e4-4f7a-bafe-e3e1138d5cb0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '070eceecd25b42aaa8c02fffb8d2cbd3'} {{(pid=80245) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 08:31:49.684541 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c97cb467-a687-4612-8ea0-5b18d9757b6e admin admin] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:31:49.686930 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.fernet_utils [None req-c97cb467-a687-4612-8ea0-5b18d9757b6e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80244) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 08:31:49.689543 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.notifications [None req-daf746b5-96e4-4f7a-bafe-e3e1138d5cb0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '070eceecd25b42aaa8c02fffb8d2cbd3'} {{(pid=80245) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 08:31:49.713896 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c97cb467-a687-4612-8ea0-5b18d9757b6e admin admin] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:31:49.723148 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.notifications [None req-daf746b5-96e4-4f7a-bafe-e3e1138d5cb0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '070eceecd25b42aaa8c02fffb8d2cbd3'} {{(pid=80245) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 08:31:49.728653 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e83071f-b937-42ea-ac78-939ea47a0145 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '42d15fc0515945acacbbffcf2356c7a5', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:31:49.730101 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e83071f-b937-42ea-ac78-939ea47a0145 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:31:49.730389 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e83071f-b937-42ea-ac78-939ea47a0145 admin admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:31:49.730831 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e83071f-b937-42ea-ac78-939ea47a0145 admin admin] PATH_INFO: `/v3/projects/a4a8929428914b9c9c14906b361e4609` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:31:49.732846 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e83071f-b937-42ea-ac78-939ea47a0145 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a4a8929428914b9c9c14906b361e4609)` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:31:49.735789 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.notifications [None req-daf746b5-96e4-4f7a-bafe-e3e1138d5cb0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '070eceecd25b42aaa8c02fffb8d2cbd3'} {{(pid=80245) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 08:31:49.736752 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e83071f-b937-42ea-ac78-939ea47a0145 admin admin] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:31:49.737615 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.notifications [None req-daf746b5-96e4-4f7a-bafe-e3e1138d5cb0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '070eceecd25b42aaa8c02fffb8d2cbd3'} {{(pid=80245) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 08:31:49.749076 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: [pid: 80245|app: 0|req: 1089/2180] 10.222.0.47 () {58 vars in 1223 bytes} [Thu Jul 21 08:31:49 2022] DELETE /identity/v3/users/070eceecd25b42aaa8c02fffb8d2cbd3 => generated 0 bytes in 252 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 08:31:49.816803 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.notifications [None req-4e83071f-b937-42ea-ac78-939ea47a0145 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a4a8929428914b9c9c14906b361e4609', 'request_id': 'req-4e83071f-b937-42ea-ac78-939ea47a0145'} {{(pid=80244) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 08:31:49.826269 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.notifications [None req-4e83071f-b937-42ea-ac78-939ea47a0145 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a4a8929428914b9c9c14906b361e4609', 'request_id': 'req-4e83071f-b937-42ea-ac78-939ea47a0145'} {{(pid=80244) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 08:31:49.836182 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.notifications [None req-4e83071f-b937-42ea-ac78-939ea47a0145 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a4a8929428914b9c9c14906b361e4609', 'request_id': 'req-4e83071f-b937-42ea-ac78-939ea47a0145'} {{(pid=80244) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 08:31:49.857574 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.notifications [None req-4e83071f-b937-42ea-ac78-939ea47a0145 admin admin] The token cache is being invalidate because project a4a8929428914b9c9c14906b361e4609 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80244) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 21 08:31:49.857990 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.notifications [None req-4e83071f-b937-42ea-ac78-939ea47a0145 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80244) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 08:31:49.859008 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.notifications [None req-4e83071f-b937-42ea-ac78-939ea47a0145 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80244) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 08:31:49.866034 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 1092/2181] 10.222.0.47 () {58 vars in 1232 bytes} [Thu Jul 21 08:31:49 2022] DELETE /identity/v3/projects/a4a8929428914b9c9c14906b361e4609 => generated 0 bytes in 178 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 21 08:31:49.868797 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b1cf9e9-122c-4aa8-9c12-a9f4c5b3bb8d None None] REQUEST_METHOD: `POST` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:31:49.869821 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b1cf9e9-122c-4aa8-9c12-a9f4c5b3bb8d None None] SCRIPT_NAME: `/identity` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:31:49.870558 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b1cf9e9-122c-4aa8-9c12-a9f4c5b3bb8d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:31:49.894875 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.auth.core [None req-9b1cf9e9-122c-4aa8-9c12-a9f4c5b3bb8d None None] MFA Rules not processed for user `2dce40b8d74c41ec94cc36e6ff05d412`. Rule list: `[]` (Enabled: `True`). {{(pid=80245) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 08:31:49.896285 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.fernet_utils [None req-9b1cf9e9-122c-4aa8-9c12-a9f4c5b3bb8d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80245) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 08:31:49.972828 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: [pid: 80245|app: 0|req: 1090/2182] 10.222.0.47 () {58 vars in 957 bytes} [Thu Jul 21 08:31:49 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 08:31:49.982535 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dafd92b4-008c-43b8-8584-b81bbd1c5108 None None] REQUEST_METHOD: `POST` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:31:49.982995 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dafd92b4-008c-43b8-8584-b81bbd1c5108 None None] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:31:49.983464 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dafd92b4-008c-43b8-8584-b81bbd1c5108 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:31:49.992757 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG dbcounter [-] [80244] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=80244) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:31:50.008020 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.auth.core [None req-dafd92b4-008c-43b8-8584-b81bbd1c5108 None None] MFA Rules not processed for user `2dce40b8d74c41ec94cc36e6ff05d412`. Rule list: `[]` (Enabled: `True`). {{(pid=80244) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 08:31:50.010524 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.fernet_utils [None req-dafd92b4-008c-43b8-8584-b81bbd1c5108 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80244) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 08:31:50.018705 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 1093/2183] 10.222.0.47 () {58 vars in 957 bytes} [Thu Jul 21 08:31:49 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 08:31:50.023093 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b1cf9e9-122c-4aa8-9c12-a9f4c5b3bb8d None None] Authenticating user token {{(pid=80245) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:31:50.033275 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b1cf9e9-122c-4aa8-9c12-a9f4c5b3bb8d None None] Validating token access rules against request {{(pid=80245) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:31:50.037996 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa416ed9-33d0-4e40-bb8b-7fa88c70a916 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '42d15fc0515945acacbbffcf2356c7a5', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80245) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:31:50.038666 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa416ed9-33d0-4e40-bb8b-7fa88c70a916 admin admin] REQUEST_METHOD: `GET` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:31:50.038956 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa416ed9-33d0-4e40-bb8b-7fa88c70a916 admin admin] SCRIPT_NAME: `/identity` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:31:50.039236 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa416ed9-33d0-4e40-bb8b-7fa88c70a916 admin admin] PATH_INFO: `/v3/domains` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:31:50.039998 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa416ed9-33d0-4e40-bb8b-7fa88c70a916 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80245) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 08:31:50.040360 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa416ed9-33d0-4e40-bb8b-7fa88c70a916 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:31:50.042922 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa416ed9-33d0-4e40-bb8b-7fa88c70a916 admin admin] RBAC: Authorization granted {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:31:50.056298 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: [pid: 80245|app: 0|req: 1091/2184] 10.222.0.47 () {58 vars in 1152 bytes} [Thu Jul 21 08:31:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 08:31:50.060363 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dafd92b4-008c-43b8-8584-b81bbd1c5108 None None] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:31:50.070739 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dafd92b4-008c-43b8-8584-b81bbd1c5108 None None] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:31:50.079363 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c3c8724-3cfc-4b52-be13-54a552b1c46b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '42d15fc0515945acacbbffcf2356c7a5', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:31:50.079363 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c3c8724-3cfc-4b52-be13-54a552b1c46b admin admin] REQUEST_METHOD: `POST` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:31:50.079363 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c3c8724-3cfc-4b52-be13-54a552b1c46b admin admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:31:50.079363 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c3c8724-3cfc-4b52-be13-54a552b1c46b admin admin] PATH_INFO: `/v3/projects` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:31:50.079809 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c3c8724-3cfc-4b52-be13-54a552b1c46b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:31:50.081590 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c3c8724-3cfc-4b52-be13-54a552b1c46b admin admin] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:31:50.110387 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa416ed9-33d0-4e40-bb8b-7fa88c70a916 admin admin] Authenticating user token {{(pid=80245) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:31:50.112434 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.fernet_utils [None req-fa416ed9-33d0-4e40-bb8b-7fa88c70a916 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80245) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 08:31:50.116672 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 1094/2185] 10.222.0.47 () {60 vars in 1152 bytes} [Thu Jul 21 08:31:50 2022] POST /identity/v3/projects => generated 381 bytes in 57 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 08:31:50.123868 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c3c8724-3cfc-4b52-be13-54a552b1c46b admin admin] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:31:50.142399 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa416ed9-33d0-4e40-bb8b-7fa88c70a916 admin admin] Validating token access rules against request {{(pid=80245) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:31:50.193501 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c3c8724-3cfc-4b52-be13-54a552b1c46b admin admin] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:31:50.199116 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96955689-b563-46c4-9b79-038210b75d0b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '42d15fc0515945acacbbffcf2356c7a5', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:31:50.199865 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96955689-b563-46c4-9b79-038210b75d0b admin admin] REQUEST_METHOD: `POST` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:31:50.200190 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96955689-b563-46c4-9b79-038210b75d0b admin admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:31:50.200574 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96955689-b563-46c4-9b79-038210b75d0b admin admin] PATH_INFO: `/v3/users` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:31:50.201334 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96955689-b563-46c4-9b79-038210b75d0b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:31:50.203462 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96955689-b563-46c4-9b79-038210b75d0b admin admin] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:31:50.219170 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29e48c14-dafa-4e9e-9baa-3713dedb906c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '42d15fc0515945acacbbffcf2356c7a5', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80245) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:31:50.220122 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29e48c14-dafa-4e9e-9baa-3713dedb906c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:31:50.220409 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29e48c14-dafa-4e9e-9baa-3713dedb906c admin admin] SCRIPT_NAME: `/identity` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:31:50.220688 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29e48c14-dafa-4e9e-9baa-3713dedb906c admin admin] PATH_INFO: `/v3/projects/e797f003b6134c0e88a78370830609ba` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:31:50.222201 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29e48c14-dafa-4e9e-9baa-3713dedb906c admin admin] RBAC: Authorizing `identity:delete_project(project_id=e797f003b6134c0e88a78370830609ba)` {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:31:50.224761 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29e48c14-dafa-4e9e-9baa-3713dedb906c admin admin] RBAC: Authorization granted {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:31:50.228826 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 1095/2186] 10.222.0.47 () {60 vars in 1143 bytes} [Thu Jul 21 08:31:50 2022] POST /identity/v3/users => generated 291 bytes in 105 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 08:31:50.232771 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96955689-b563-46c4-9b79-038210b75d0b admin admin] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:31:50.244541 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96955689-b563-46c4-9b79-038210b75d0b admin admin] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:31:50.251499 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b17629bb-3036-44fe-a260-f43a7edbadff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '42d15fc0515945acacbbffcf2356c7a5', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:31:50.252432 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b17629bb-3036-44fe-a260-f43a7edbadff admin admin] REQUEST_METHOD: `GET` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:31:50.252791 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b17629bb-3036-44fe-a260-f43a7edbadff admin admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:31:50.253142 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b17629bb-3036-44fe-a260-f43a7edbadff admin admin] PATH_INFO: `/v3/roles` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:31:50.258875 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b17629bb-3036-44fe-a260-f43a7edbadff admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:31:50.258875 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b17629bb-3036-44fe-a260-f43a7edbadff admin admin] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:31:50.273710 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 1096/2187] 10.222.0.47 () {58 vars in 1121 bytes} [Thu Jul 21 08:31:50 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 21 08:31:50.278854 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b17629bb-3036-44fe-a260-f43a7edbadff admin admin] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:31:50.279468 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG dbcounter [-] [80245] Writing DB stats keystone:DELETE=8,keystone:SELECT=91,keystone:INSERT=1 {{(pid=80245) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:31:50.290713 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b17629bb-3036-44fe-a260-f43a7edbadff admin admin] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:31:50.301428 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.notifications [None req-29e48c14-dafa-4e9e-9baa-3713dedb906c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e797f003b6134c0e88a78370830609ba', 'request_id': 'req-29e48c14-dafa-4e9e-9baa-3713dedb906c'} {{(pid=80245) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 08:31:50.308334 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.notifications [None req-29e48c14-dafa-4e9e-9baa-3713dedb906c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e797f003b6134c0e88a78370830609ba', 'request_id': 'req-29e48c14-dafa-4e9e-9baa-3713dedb906c'} {{(pid=80245) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 08:31:50.315051 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.notifications [None req-29e48c14-dafa-4e9e-9baa-3713dedb906c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e797f003b6134c0e88a78370830609ba', 'request_id': 'req-29e48c14-dafa-4e9e-9baa-3713dedb906c'} {{(pid=80245) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 08:31:50.329460 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.notifications [None req-29e48c14-dafa-4e9e-9baa-3713dedb906c admin admin] The token cache is being invalidate because project e797f003b6134c0e88a78370830609ba was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80245) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 21 08:31:50.329793 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.notifications [None req-29e48c14-dafa-4e9e-9baa-3713dedb906c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80245) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 08:31:50.330430 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.notifications [None req-29e48c14-dafa-4e9e-9baa-3713dedb906c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80245) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 08:31:50.332034 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: [pid: 80245|app: 0|req: 1092/2188] 10.222.0.47 () {58 vars in 1232 bytes} [Thu Jul 21 08:31:50 2022] DELETE /identity/v3/projects/e797f003b6134c0e88a78370830609ba => generated 0 bytes in 222 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 08:31:50.338501 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a511a2d1-af68-4d84-8137-9187a23d9fbd None None] REQUEST_METHOD: `POST` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:31:50.338841 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a511a2d1-af68-4d84-8137-9187a23d9fbd None None] SCRIPT_NAME: `/identity` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:31:50.339639 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a511a2d1-af68-4d84-8137-9187a23d9fbd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:31:50.359896 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.auth.core [None req-a511a2d1-af68-4d84-8137-9187a23d9fbd None None] MFA Rules not processed for user `2dce40b8d74c41ec94cc36e6ff05d412`. Rule list: `[]` (Enabled: `True`). {{(pid=80245) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 08:31:50.361333 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.fernet_utils [None req-a511a2d1-af68-4d84-8137-9187a23d9fbd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80245) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 08:31:50.362184 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fee0d11-594f-4758-ad29-0e2d8ffeb5e3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '42d15fc0515945acacbbffcf2356c7a5', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:31:50.362865 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fee0d11-594f-4758-ad29-0e2d8ffeb5e3 admin admin] REQUEST_METHOD: `GET` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:31:50.363148 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fee0d11-594f-4758-ad29-0e2d8ffeb5e3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:31:50.363455 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fee0d11-594f-4758-ad29-0e2d8ffeb5e3 admin admin] PATH_INFO: `/v3/roles` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:31:50.364181 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fee0d11-594f-4758-ad29-0e2d8ffeb5e3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:31:50.365937 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fee0d11-594f-4758-ad29-0e2d8ffeb5e3 admin admin] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:31:50.369804 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: [pid: 80245|app: 0|req: 1093/2189] 10.222.0.47 () {58 vars in 957 bytes} [Thu Jul 21 08:31:50 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 21 08:31:50.378273 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 1097/2190] 10.222.0.47 () {58 vars in 1121 bytes} [Thu Jul 21 08:31:50 2022] GET /identity/v3/roles => generated 1420 bytes in 98 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 21 08:31:50.380124 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47d8c486-b2be-441b-b7f7-785155405c72 None None] REQUEST_METHOD: `POST` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:31:50.380429 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47d8c486-b2be-441b-b7f7-785155405c72 None None] SCRIPT_NAME: `/identity` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:31:50.380712 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47d8c486-b2be-441b-b7f7-785155405c72 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:31:50.380809 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fee0d11-594f-4758-ad29-0e2d8ffeb5e3 admin admin] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:31:50.382463 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.fernet_utils [None req-5fee0d11-594f-4758-ad29-0e2d8ffeb5e3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80244) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 08:31:50.396302 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fee0d11-594f-4758-ad29-0e2d8ffeb5e3 admin admin] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:31:50.398946 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.auth.core [None req-47d8c486-b2be-441b-b7f7-785155405c72 None None] MFA Rules not processed for user `2dce40b8d74c41ec94cc36e6ff05d412`. Rule list: `[]` (Enabled: `True`). {{(pid=80245) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 08:31:50.400181 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.fernet_utils [None req-47d8c486-b2be-441b-b7f7-785155405c72 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80245) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 08:31:50.401006 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbf344cf-983e-4d58-b2e8-224cef27e011 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '42d15fc0515945acacbbffcf2356c7a5', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:31:50.402493 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbf344cf-983e-4d58-b2e8-224cef27e011 admin admin] REQUEST_METHOD: `PUT` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:31:50.402842 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbf344cf-983e-4d58-b2e8-224cef27e011 admin admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:31:50.403051 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbf344cf-983e-4d58-b2e8-224cef27e011 admin admin] PATH_INFO: `/v3/projects/b76ac06d76d8470dae64c89763e11b9f/users/61e3499425174cb586a0f8895f0591b6/roles/3a074738221e4601b5205715ba87d240` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:31:50.408788 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: [pid: 80245|app: 0|req: 1094/2191] 10.222.0.47 () {58 vars in 957 bytes} [Thu Jul 21 08:31:50 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 08:31:50.413129 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47d8c486-b2be-441b-b7f7-785155405c72 None None] Authenticating user token {{(pid=80245) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:31:50.418215 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbf344cf-983e-4d58-b2e8-224cef27e011 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b76ac06d76d8470dae64c89763e11b9f, user_id=61e3499425174cb586a0f8895f0591b6, role_id=3a074738221e4601b5205715ba87d240)` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:31:50.420893 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbf344cf-983e-4d58-b2e8-224cef27e011 admin admin] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:31:50.422132 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47d8c486-b2be-441b-b7f7-785155405c72 None None] Validating token access rules against request {{(pid=80245) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:31:50.427258 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aae44127-783c-4839-95a0-dd2468ac21ab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '42d15fc0515945acacbbffcf2356c7a5', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80245) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:31:50.427997 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aae44127-783c-4839-95a0-dd2468ac21ab admin admin] REQUEST_METHOD: `GET` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:31:50.428349 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aae44127-783c-4839-95a0-dd2468ac21ab admin admin] SCRIPT_NAME: `/identity` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:31:50.428651 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aae44127-783c-4839-95a0-dd2468ac21ab admin admin] PATH_INFO: `/v3/domains` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:31:50.429384 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aae44127-783c-4839-95a0-dd2468ac21ab admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80245) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 08:31:50.429692 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aae44127-783c-4839-95a0-dd2468ac21ab admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:31:50.431463 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aae44127-783c-4839-95a0-dd2468ac21ab admin admin] RBAC: Authorization granted {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:31:50.435806 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 1098/2192] 10.222.0.47 () {60 vars in 1482 bytes} [Thu Jul 21 08:31:50 2022] PUT /identity/v3/projects/b76ac06d76d8470dae64c89763e11b9f/users/61e3499425174cb586a0f8895f0591b6/roles/3a074738221e4601b5205715ba87d240 => generated 0 bytes in 55 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 21 08:31:50.449033 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: [pid: 80245|app: 0|req: 1095/2193] 10.222.0.47 () {58 vars in 1152 bytes} [Thu Jul 21 08:31:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 21 08:31:50.449611 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80fe1318-9527-4dec-8022-93257b6f8300 None None] REQUEST_METHOD: `POST` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:31:50.449902 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80fe1318-9527-4dec-8022-93257b6f8300 None None] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:31:50.450172 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80fe1318-9527-4dec-8022-93257b6f8300 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:31:50.450555 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aae44127-783c-4839-95a0-dd2468ac21ab admin admin] Authenticating user token {{(pid=80245) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:31:50.460562 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aae44127-783c-4839-95a0-dd2468ac21ab admin admin] Validating token access rules against request {{(pid=80245) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:31:50.476954 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.auth.core [None req-80fe1318-9527-4dec-8022-93257b6f8300 None None] MFA Rules not processed for user `61e3499425174cb586a0f8895f0591b6`. Rule list: `[]` (Enabled: `True`). {{(pid=80244) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 08:31:50.478173 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.fernet_utils [None req-80fe1318-9527-4dec-8022-93257b6f8300 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80244) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 08:31:50.490812 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG dbcounter [-] [80244] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80244) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:31:50.532361 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91ecd8a0-eda0-40a5-84bd-97cdae0b1b77 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '42d15fc0515945acacbbffcf2356c7a5', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80245) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:31:50.533163 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91ecd8a0-eda0-40a5-84bd-97cdae0b1b77 admin admin] REQUEST_METHOD: `POST` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:31:50.533458 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91ecd8a0-eda0-40a5-84bd-97cdae0b1b77 admin admin] SCRIPT_NAME: `/identity` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:31:50.533745 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91ecd8a0-eda0-40a5-84bd-97cdae0b1b77 admin admin] PATH_INFO: `/v3/projects` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:31:50.534484 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91ecd8a0-eda0-40a5-84bd-97cdae0b1b77 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:31:50.536450 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91ecd8a0-eda0-40a5-84bd-97cdae0b1b77 admin admin] RBAC: Authorization granted {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:31:50.555907 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 1099/2194] 10.222.0.47 () {58 vars in 957 bytes} [Thu Jul 21 08:31:50 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 08:31:50.564438 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80fe1318-9527-4dec-8022-93257b6f8300 None None] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:31:50.567014 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.fernet_utils [None req-80fe1318-9527-4dec-8022-93257b6f8300 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80244) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 08:31:50.570501 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: [pid: 80245|app: 0|req: 1096/2195] 10.222.0.47 () {60 vars in 1152 bytes} [Thu Jul 21 08:31:50 2022] POST /identity/v3/projects => generated 367 bytes in 120 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 21 08:31:50.576305 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91ecd8a0-eda0-40a5-84bd-97cdae0b1b77 admin admin] Authenticating user token {{(pid=80245) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:31:50.657667 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91ecd8a0-eda0-40a5-84bd-97cdae0b1b77 admin admin] Validating token access rules against request {{(pid=80245) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:31:50.658200 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80fe1318-9527-4dec-8022-93257b6f8300 None None] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:31:50.663967 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce864270-4faa-499c-9822-7f4ecb5c45f9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '42d15fc0515945acacbbffcf2356c7a5', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80245) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:31:50.664652 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce864270-4faa-499c-9822-7f4ecb5c45f9 admin admin] REQUEST_METHOD: `POST` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:31:50.664973 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce864270-4faa-499c-9822-7f4ecb5c45f9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:31:50.665318 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce864270-4faa-499c-9822-7f4ecb5c45f9 admin admin] PATH_INFO: `/v3/users` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:31:50.666214 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce864270-4faa-499c-9822-7f4ecb5c45f9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:31:50.668216 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce864270-4faa-499c-9822-7f4ecb5c45f9 admin admin] RBAC: Authorization granted {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:31:50.697027 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: [pid: 80245|app: 0|req: 1097/2196] 10.222.0.47 () {60 vars in 1143 bytes} [Thu Jul 21 08:31:50 2022] POST /identity/v3/users => generated 284 bytes in 122 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 21 08:31:50.698028 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce864270-4faa-499c-9822-7f4ecb5c45f9 admin admin] Authenticating user token {{(pid=80245) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:31:50.709927 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce864270-4faa-499c-9822-7f4ecb5c45f9 admin admin] Validating token access rules against request {{(pid=80245) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:31:50.717304 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bacda634-2663-41b2-ae31-f5274df89d50 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '42d15fc0515945acacbbffcf2356c7a5', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80245) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:31:50.717993 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bacda634-2663-41b2-ae31-f5274df89d50 admin admin] REQUEST_METHOD: `GET` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:31:50.718281 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bacda634-2663-41b2-ae31-f5274df89d50 admin admin] SCRIPT_NAME: `/identity` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:31:50.718561 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bacda634-2663-41b2-ae31-f5274df89d50 admin admin] PATH_INFO: `/v3/roles` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:31:50.719306 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bacda634-2663-41b2-ae31-f5274df89d50 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:31:50.721084 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bacda634-2663-41b2-ae31-f5274df89d50 admin admin] RBAC: Authorization granted {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:31:50.722585 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6430965-a43b-4d0b-9025-7e7d474ec6c0 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4f40410b7674687ab5362ac982248ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4faef13aa50e4e36b6c265fb1398fa14', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:31:50.723375 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6430965-a43b-4d0b-9025-7e7d474ec6c0 service cinder] REQUEST_METHOD: `GET` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:31:50.723849 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6430965-a43b-4d0b-9025-7e7d474ec6c0 service cinder] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:31:50.724135 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6430965-a43b-4d0b-9025-7e7d474ec6c0 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:31:50.725144 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG dbcounter [-] [80245] Writing DB stats keystone:SELECT=90,keystone:DELETE=5,keystone:INSERT=5 {{(pid=80245) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:31:50.740764 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: [pid: 80245|app: 0|req: 1098/2197] 10.222.0.47 () {58 vars in 1121 bytes} [Thu Jul 21 08:31:50 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 08:31:50.741928 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bacda634-2663-41b2-ae31-f5274df89d50 admin admin] Authenticating user token {{(pid=80245) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:31:50.759678 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bacda634-2663-41b2-ae31-f5274df89d50 admin admin] Validating token access rules against request {{(pid=80245) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:31:50.772956 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02779174-1d02-4ec1-818f-9e34c1509c91 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '42d15fc0515945acacbbffcf2356c7a5', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80245) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:31:50.775494 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02779174-1d02-4ec1-818f-9e34c1509c91 admin admin] REQUEST_METHOD: `GET` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:31:50.775822 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02779174-1d02-4ec1-818f-9e34c1509c91 admin admin] SCRIPT_NAME: `/identity` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:31:50.776336 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02779174-1d02-4ec1-818f-9e34c1509c91 admin admin] PATH_INFO: `/v3/roles` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:31:50.777356 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02779174-1d02-4ec1-818f-9e34c1509c91 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:31:50.779052 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02779174-1d02-4ec1-818f-9e34c1509c91 admin admin] RBAC: Authorization granted {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:31:50.794369 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: [pid: 80245|app: 0|req: 1099/2198] 10.222.0.47 () {58 vars in 1121 bytes} [Thu Jul 21 08:31:50 2022] GET /identity/v3/roles => generated 1420 bytes in 53 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 08:31:50.799891 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02779174-1d02-4ec1-818f-9e34c1509c91 admin admin] Authenticating user token {{(pid=80245) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:31:50.812154 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02779174-1d02-4ec1-818f-9e34c1509c91 admin admin] Validating token access rules against request {{(pid=80245) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:31:50.813457 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6430965-a43b-4d0b-9025-7e7d474ec6c0 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:31:50.814939 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6430965-a43b-4d0b-9025-7e7d474ec6c0 service cinder] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:31:50.818274 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9f07c34-adfc-4282-9f9d-1fede69c0619 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '42d15fc0515945acacbbffcf2356c7a5', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80245) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:31:50.819195 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9f07c34-adfc-4282-9f9d-1fede69c0619 admin admin] REQUEST_METHOD: `PUT` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:31:50.819496 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9f07c34-adfc-4282-9f9d-1fede69c0619 admin admin] SCRIPT_NAME: `/identity` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:31:50.819765 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9f07c34-adfc-4282-9f9d-1fede69c0619 admin admin] PATH_INFO: `/v3/projects/bf11a7e194124ec8b57bfd2138d09a4f/users/acfab497869f427ab16f26bdec11f7ed/roles/3a074738221e4601b5205715ba87d240` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:31:50.825965 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 1100/2199] 10.222.0.47 () {60 vars in 1365 bytes} [Thu Jul 21 08:31:50 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 262 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 21 08:31:50.851246 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9f07c34-adfc-4282-9f9d-1fede69c0619 admin admin] RBAC: Authorizing `identity:create_grant(project_id=bf11a7e194124ec8b57bfd2138d09a4f, user_id=acfab497869f427ab16f26bdec11f7ed, role_id=3a074738221e4601b5205715ba87d240)` {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:31:50.851920 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6430965-a43b-4d0b-9025-7e7d474ec6c0 service cinder] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:31:50.860630 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9f07c34-adfc-4282-9f9d-1fede69c0619 admin admin] RBAC: Authorization granted {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:31:50.869236 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6430965-a43b-4d0b-9025-7e7d474ec6c0 service cinder] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:31:50.876982 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78d297d9-e603-4d86-a2bf-b705a90cf1c2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '42d15fc0515945acacbbffcf2356c7a5', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:31:50.878213 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78d297d9-e603-4d86-a2bf-b705a90cf1c2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:31:50.878595 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78d297d9-e603-4d86-a2bf-b705a90cf1c2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:31:50.879267 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78d297d9-e603-4d86-a2bf-b705a90cf1c2 admin admin] PATH_INFO: `/v3/users/61e3499425174cb586a0f8895f0591b6` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:31:50.880788 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78d297d9-e603-4d86-a2bf-b705a90cf1c2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=61e3499425174cb586a0f8895f0591b6)` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:31:50.882814 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78d297d9-e603-4d86-a2bf-b705a90cf1c2 admin admin] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:31:50.883856 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: [pid: 80245|app: 0|req: 1100/2200] 10.222.0.47 () {60 vars in 1482 bytes} [Thu Jul 21 08:31:50 2022] PUT /identity/v3/projects/bf11a7e194124ec8b57bfd2138d09a4f/users/acfab497869f427ab16f26bdec11f7ed/roles/3a074738221e4601b5205715ba87d240 => generated 0 bytes in 86 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 21 08:31:50.897021 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7460d19-5d38-4855-b4c2-152fcfd4555c None None] REQUEST_METHOD: `POST` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:31:50.897310 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7460d19-5d38-4855-b4c2-152fcfd4555c None None] SCRIPT_NAME: `/identity` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:31:50.897589 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7460d19-5d38-4855-b4c2-152fcfd4555c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:31:50.918753 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG dbcounter [-] [80244] Writing DB stats keystone:SELECT=98,keystone:DELETE=2 {{(pid=80244) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:31:50.930026 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.auth.core [None req-e7460d19-5d38-4855-b4c2-152fcfd4555c None None] MFA Rules not processed for user `acfab497869f427ab16f26bdec11f7ed`. Rule list: `[]` (Enabled: `True`). {{(pid=80245) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 08:31:50.931430 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.fernet_utils [None req-e7460d19-5d38-4855-b4c2-152fcfd4555c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80245) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 08:31:50.966826 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.notifications [None req-78d297d9-e603-4d86-a2bf-b705a90cf1c2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '61e3499425174cb586a0f8895f0591b6'} {{(pid=80244) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 08:31:50.977251 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.notifications [None req-78d297d9-e603-4d86-a2bf-b705a90cf1c2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '61e3499425174cb586a0f8895f0591b6'} {{(pid=80244) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 08:31:51.003080 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.notifications [None req-78d297d9-e603-4d86-a2bf-b705a90cf1c2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '61e3499425174cb586a0f8895f0591b6'} {{(pid=80244) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 08:31:51.003897 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.notifications [None req-78d297d9-e603-4d86-a2bf-b705a90cf1c2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '61e3499425174cb586a0f8895f0591b6'} {{(pid=80244) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 08:31:51.012957 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.notifications [None req-78d297d9-e603-4d86-a2bf-b705a90cf1c2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '61e3499425174cb586a0f8895f0591b6'} {{(pid=80244) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 08:31:51.020178 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: [pid: 80245|app: 0|req: 1101/2201] 10.222.0.47 () {58 vars in 957 bytes} [Thu Jul 21 08:31:50 2022] POST /identity/v3/auth/tokens => generated 3234 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 08:31:51.020178 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7460d19-5d38-4855-b4c2-152fcfd4555c None None] Authenticating user token {{(pid=80245) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:31:51.020178 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.fernet_utils [None req-e7460d19-5d38-4855-b4c2-152fcfd4555c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80245) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 08:31:51.023820 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 1101/2202] 10.222.0.47 () {58 vars in 1223 bytes} [Thu Jul 21 08:31:50 2022] DELETE /identity/v3/users/61e3499425174cb586a0f8895f0591b6 => generated 0 bytes in 172 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 08:31:51.027275 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78d297d9-e603-4d86-a2bf-b705a90cf1c2 admin admin] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:31:51.032016 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.fernet_utils [None req-78d297d9-e603-4d86-a2bf-b705a90cf1c2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80244) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 08:31:51.124549 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78d297d9-e603-4d86-a2bf-b705a90cf1c2 admin admin] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:31:51.125126 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7460d19-5d38-4855-b4c2-152fcfd4555c None None] Validating token access rules against request {{(pid=80245) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:31:51.129485 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0c9f60e-c13f-4b17-ae57-96c01cb66ebb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '42d15fc0515945acacbbffcf2356c7a5', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:31:51.130159 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0c9f60e-c13f-4b17-ae57-96c01cb66ebb admin admin] REQUEST_METHOD: `POST` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:31:51.130272 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01a49ae8-01eb-42e8-b7e3-8d32b3fde583 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '42d15fc0515945acacbbffcf2356c7a5', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80245) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:31:51.130441 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0c9f60e-c13f-4b17-ae57-96c01cb66ebb admin admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:31:51.130720 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0c9f60e-c13f-4b17-ae57-96c01cb66ebb admin admin] PATH_INFO: `/v3/projects` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:31:51.131453 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0c9f60e-c13f-4b17-ae57-96c01cb66ebb admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:31:51.132358 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01a49ae8-01eb-42e8-b7e3-8d32b3fde583 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:31:51.132653 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01a49ae8-01eb-42e8-b7e3-8d32b3fde583 admin admin] SCRIPT_NAME: `/identity` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:31:51.133333 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01a49ae8-01eb-42e8-b7e3-8d32b3fde583 admin admin] PATH_INFO: `/v3/users/c42cb2d145c8453283d3b4a291e3663e` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:31:51.133452 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0c9f60e-c13f-4b17-ae57-96c01cb66ebb admin admin] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:31:51.134347 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01a49ae8-01eb-42e8-b7e3-8d32b3fde583 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c42cb2d145c8453283d3b4a291e3663e)` {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:31:51.136448 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01a49ae8-01eb-42e8-b7e3-8d32b3fde583 admin admin] RBAC: Authorization granted {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:31:51.165899 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 1102/2203] 10.222.0.47 () {60 vars in 1152 bytes} [Thu Jul 21 08:31:51 2022] POST /identity/v3/projects => generated 367 bytes in 142 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 08:31:51.169388 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8205eb5-980e-49b6-98d6-b2baeb9a0f96 None None] REQUEST_METHOD: `POST` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:31:51.169831 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8205eb5-980e-49b6-98d6-b2baeb9a0f96 None None] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:31:51.170120 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8205eb5-980e-49b6-98d6-b2baeb9a0f96 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:31:51.188906 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.auth.core [None req-c8205eb5-980e-49b6-98d6-b2baeb9a0f96 None None] MFA Rules not processed for user `2dce40b8d74c41ec94cc36e6ff05d412`. Rule list: `[]` (Enabled: `True`). {{(pid=80244) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 08:31:51.190043 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.fernet_utils [None req-c8205eb5-980e-49b6-98d6-b2baeb9a0f96 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80244) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 08:31:51.198139 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.notifications [None req-01a49ae8-01eb-42e8-b7e3-8d32b3fde583 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c42cb2d145c8453283d3b4a291e3663e'} {{(pid=80245) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 08:31:51.207230 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.notifications [None req-01a49ae8-01eb-42e8-b7e3-8d32b3fde583 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c42cb2d145c8453283d3b4a291e3663e'} {{(pid=80245) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 08:31:51.209670 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG dbcounter [-] [80245] Writing DB stats keystone:SELECT=90,keystone:INSERT=2,keystone:DELETE=8 {{(pid=80245) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:31:51.233964 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.notifications [None req-01a49ae8-01eb-42e8-b7e3-8d32b3fde583 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c42cb2d145c8453283d3b4a291e3663e'} {{(pid=80245) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 08:31:51.242498 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.notifications [None req-01a49ae8-01eb-42e8-b7e3-8d32b3fde583 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c42cb2d145c8453283d3b4a291e3663e'} {{(pid=80245) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 08:31:51.243442 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.notifications [None req-01a49ae8-01eb-42e8-b7e3-8d32b3fde583 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c42cb2d145c8453283d3b4a291e3663e'} {{(pid=80245) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 08:31:51.255141 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: [pid: 80245|app: 0|req: 1102/2204] 10.222.0.47 () {58 vars in 1223 bytes} [Thu Jul 21 08:31:51 2022] DELETE /identity/v3/users/c42cb2d145c8453283d3b4a291e3663e => generated 0 bytes in 238 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 08:31:51.256279 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01a49ae8-01eb-42e8-b7e3-8d32b3fde583 admin admin] Authenticating user token {{(pid=80245) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:31:51.259293 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.fernet_utils [None req-01a49ae8-01eb-42e8-b7e3-8d32b3fde583 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80245) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 08:31:51.272983 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 1103/2205] 10.222.0.47 () {58 vars in 957 bytes} [Thu Jul 21 08:31:51 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 08:31:51.274275 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de99030c-9519-45e1-8cb6-1e8d7de21930 None None] REQUEST_METHOD: `POST` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:31:51.274569 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de99030c-9519-45e1-8cb6-1e8d7de21930 None None] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:31:51.274902 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de99030c-9519-45e1-8cb6-1e8d7de21930 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:31:51.304698 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.auth.core [None req-de99030c-9519-45e1-8cb6-1e8d7de21930 None None] MFA Rules not processed for user `2dce40b8d74c41ec94cc36e6ff05d412`. Rule list: `[]` (Enabled: `True`). {{(pid=80244) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 08:31:51.306135 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.fernet_utils [None req-de99030c-9519-45e1-8cb6-1e8d7de21930 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80244) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 08:31:51.328355 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG dbcounter [-] [80244] Writing DB stats keystone:DELETE=8,keystone:SELECT=90,keystone:INSERT=2 {{(pid=80244) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:31:51.376154 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 1104/2206] 10.222.0.47 () {58 vars in 957 bytes} [Thu Jul 21 08:31:51 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 08:31:51.377808 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de99030c-9519-45e1-8cb6-1e8d7de21930 None None] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:31:51.386574 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.fernet_utils [None req-de99030c-9519-45e1-8cb6-1e8d7de21930 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80244) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 08:31:51.392194 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01a49ae8-01eb-42e8-b7e3-8d32b3fde583 admin admin] Validating token access rules against request {{(pid=80245) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:31:51.399299 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6c433f7-8672-42cb-8f0f-0e277c755b64 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '42d15fc0515945acacbbffcf2356c7a5', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80245) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:31:51.400074 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6c433f7-8672-42cb-8f0f-0e277c755b64 admin admin] REQUEST_METHOD: `POST` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:31:51.400364 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6c433f7-8672-42cb-8f0f-0e277c755b64 admin admin] SCRIPT_NAME: `/identity` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:31:51.400646 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6c433f7-8672-42cb-8f0f-0e277c755b64 admin admin] PATH_INFO: `/v3/users` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:31:51.401961 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6c433f7-8672-42cb-8f0f-0e277c755b64 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:31:51.403998 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6c433f7-8672-42cb-8f0f-0e277c755b64 admin admin] RBAC: Authorization granted {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:31:51.456876 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: [pid: 80245|app: 0|req: 1103/2207] 10.222.0.47 () {60 vars in 1143 bytes} [Thu Jul 21 08:31:51 2022] POST /identity/v3/users => generated 290 bytes in 199 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 08:31:51.456876 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6c433f7-8672-42cb-8f0f-0e277c755b64 admin admin] Authenticating user token {{(pid=80245) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:31:51.461723 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.fernet_utils [None req-e6c433f7-8672-42cb-8f0f-0e277c755b64 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80245) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 08:31:51.491475 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de99030c-9519-45e1-8cb6-1e8d7de21930 None None] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:31:51.495935 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eaf8d30-15de-494e-94f4-48b365c7c217 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20ca3c3bd3774691b3374cfbfec6e494', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4faef13aa50e4e36b6c265fb1398fa14', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:31:51.497225 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eaf8d30-15de-494e-94f4-48b365c7c217 service neutron] REQUEST_METHOD: `GET` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:31:51.497816 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eaf8d30-15de-494e-94f4-48b365c7c217 service neutron] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:31:51.498406 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eaf8d30-15de-494e-94f4-48b365c7c217 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:31:51.519123 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3eaf8d30-15de-494e-94f4-48b365c7c217 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:31:51.520720 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3eaf8d30-15de-494e-94f4-48b365c7c217 service neutron] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:31:51.537887 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 1105/2208] 10.222.0.47 () {60 vars in 1365 bytes} [Thu Jul 21 08:31:51 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 161 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 08:31:51.549403 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6c433f7-8672-42cb-8f0f-0e277c755b64 admin admin] Validating token access rules against request {{(pid=80245) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:31:51.553326 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eaf8d30-15de-494e-94f4-48b365c7c217 service neutron] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:31:51.554029 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7c793fb-7bc9-4d8a-a395-bc9a87b6bb34 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20ca3c3bd3774691b3374cfbfec6e494', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4faef13aa50e4e36b6c265fb1398fa14', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80245) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:31:51.554770 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7c793fb-7bc9-4d8a-a395-bc9a87b6bb34 service neutron] REQUEST_METHOD: `GET` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:31:51.555149 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7c793fb-7bc9-4d8a-a395-bc9a87b6bb34 service neutron] SCRIPT_NAME: `/identity` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:31:51.555531 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7c793fb-7bc9-4d8a-a395-bc9a87b6bb34 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:31:51.570225 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7c793fb-7bc9-4d8a-a395-bc9a87b6bb34 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:31:51.571976 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7c793fb-7bc9-4d8a-a395-bc9a87b6bb34 service neutron] RBAC: Authorization granted {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:31:51.573426 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eaf8d30-15de-494e-94f4-48b365c7c217 service neutron] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:31:51.585433 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: [pid: 80245|app: 0|req: 1104/2209] 10.222.0.47 () {60 vars in 1365 bytes} [Thu Jul 21 08:31:51 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 130 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 08:31:51.585694 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c194370-3a56-4e20-9832-896c7fe4ed15 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '42d15fc0515945acacbbffcf2356c7a5', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:31:51.586406 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c194370-3a56-4e20-9832-896c7fe4ed15 admin admin] REQUEST_METHOD: `GET` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:31:51.586691 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c194370-3a56-4e20-9832-896c7fe4ed15 admin admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:31:51.586970 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c194370-3a56-4e20-9832-896c7fe4ed15 admin admin] PATH_INFO: `/v3/roles` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:31:51.588023 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c194370-3a56-4e20-9832-896c7fe4ed15 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:31:51.590022 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c194370-3a56-4e20-9832-896c7fe4ed15 admin admin] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:31:51.605627 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 1106/2210] 10.222.0.47 () {58 vars in 1121 bytes} [Thu Jul 21 08:31:51 2022] GET /identity/v3/roles => generated 1420 bytes in 49 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 08:31:51.608059 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7c793fb-7bc9-4d8a-a395-bc9a87b6bb34 service neutron] Authenticating user token {{(pid=80245) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:31:51.621536 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7c793fb-7bc9-4d8a-a395-bc9a87b6bb34 service neutron] Validating token access rules against request {{(pid=80245) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:31:51.627778 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e77619f-7a17-4b18-9772-9e2f2335f4c4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '42d15fc0515945acacbbffcf2356c7a5', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80245) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:31:51.628744 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e77619f-7a17-4b18-9772-9e2f2335f4c4 admin admin] REQUEST_METHOD: `PUT` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:31:51.629497 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e77619f-7a17-4b18-9772-9e2f2335f4c4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:31:51.629906 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e77619f-7a17-4b18-9772-9e2f2335f4c4 admin admin] PATH_INFO: `/v3/projects/9d0ee62a91f642abbeed3199a08b8623/users/fadfd7afe48545499050bda5b323005e/roles/e29870f6e9e2411a9db0d5bf45f32ca5` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:31:51.656539 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e77619f-7a17-4b18-9772-9e2f2335f4c4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9d0ee62a91f642abbeed3199a08b8623, user_id=fadfd7afe48545499050bda5b323005e, role_id=e29870f6e9e2411a9db0d5bf45f32ca5)` {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:31:51.660489 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e77619f-7a17-4b18-9772-9e2f2335f4c4 admin admin] RBAC: Authorization granted {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:31:51.683847 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: [pid: 80245|app: 0|req: 1105/2211] 10.222.0.47 () {60 vars in 1482 bytes} [Thu Jul 21 08:31:51 2022] PUT /identity/v3/projects/9d0ee62a91f642abbeed3199a08b8623/users/fadfd7afe48545499050bda5b323005e/roles/e29870f6e9e2411a9db0d5bf45f32ca5 => generated 0 bytes in 77 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 08:31:51.688233 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c194370-3a56-4e20-9832-896c7fe4ed15 admin admin] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:31:51.703779 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c194370-3a56-4e20-9832-896c7fe4ed15 admin admin] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:31:51.789579 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f939c7b3-21f2-4a10-a02e-8ca8d5e6f105 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '42d15fc0515945acacbbffcf2356c7a5', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:31:51.790491 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f939c7b3-21f2-4a10-a02e-8ca8d5e6f105 admin admin] REQUEST_METHOD: `GET` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:31:51.790885 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f939c7b3-21f2-4a10-a02e-8ca8d5e6f105 admin admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:31:51.791540 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f939c7b3-21f2-4a10-a02e-8ca8d5e6f105 admin admin] PATH_INFO: `/v3/roles` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:31:51.793284 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f939c7b3-21f2-4a10-a02e-8ca8d5e6f105 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:31:51.795183 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f939c7b3-21f2-4a10-a02e-8ca8d5e6f105 admin admin] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:31:51.816233 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e77619f-7a17-4b18-9772-9e2f2335f4c4 admin admin] Authenticating user token {{(pid=80245) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:31:51.823660 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 1107/2212] 10.222.0.47 () {58 vars in 1121 bytes} [Thu Jul 21 08:31:51 2022] GET /identity/v3/roles => generated 1420 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 08:31:51.840182 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e77619f-7a17-4b18-9772-9e2f2335f4c4 admin admin] Validating token access rules against request {{(pid=80245) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:31:51.846596 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1de4ad99-396c-44e0-9b30-66dddd628fd4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '42d15fc0515945acacbbffcf2356c7a5', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80245) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:31:51.847848 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1de4ad99-396c-44e0-9b30-66dddd628fd4 admin admin] REQUEST_METHOD: `PUT` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:31:51.848322 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1de4ad99-396c-44e0-9b30-66dddd628fd4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:31:51.848784 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1de4ad99-396c-44e0-9b30-66dddd628fd4 admin admin] PATH_INFO: `/v3/projects/9d0ee62a91f642abbeed3199a08b8623/users/fadfd7afe48545499050bda5b323005e/roles/e29870f6e9e2411a9db0d5bf45f32ca5` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:31:51.852440 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1de4ad99-396c-44e0-9b30-66dddd628fd4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9d0ee62a91f642abbeed3199a08b8623, user_id=fadfd7afe48545499050bda5b323005e, role_id=e29870f6e9e2411a9db0d5bf45f32ca5)` {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:31:51.870848 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1de4ad99-396c-44e0-9b30-66dddd628fd4 admin admin] RBAC: Authorization granted {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:31:51.899086 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bffea952-66ee-4abd-b93e-6d4c0b04a3cd None None] REQUEST_METHOD: `POST` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:31:51.899277 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bffea952-66ee-4abd-b93e-6d4c0b04a3cd None None] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:31:51.899960 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bffea952-66ee-4abd-b93e-6d4c0b04a3cd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:31:51.901785 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: [pid: 80245|app: 0|req: 1106/2213] 10.222.0.47 () {60 vars in 1482 bytes} [Thu Jul 21 08:31:51 2022] PUT /identity/v3/projects/9d0ee62a91f642abbeed3199a08b8623/users/fadfd7afe48545499050bda5b323005e/roles/e29870f6e9e2411a9db0d5bf45f32ca5 => generated 0 bytes in 73 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 08:31:51.936956 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.auth.core [None req-bffea952-66ee-4abd-b93e-6d4c0b04a3cd None None] MFA Rules not processed for user `fadfd7afe48545499050bda5b323005e`. Rule list: `[]` (Enabled: `True`). {{(pid=80244) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 08:31:51.938908 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.fernet_utils [None req-bffea952-66ee-4abd-b93e-6d4c0b04a3cd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80244) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 08:31:52.004628 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG dbcounter [-] [80244] Writing DB stats keystone:SELECT=100 {{(pid=80244) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:31:52.047590 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 1108/2214] 10.222.0.47 () {58 vars in 957 bytes} [Thu Jul 21 08:31:51 2022] POST /identity/v3/auth/tokens => generated 3301 bytes in 145 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 08:31:52.062300 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1de4ad99-396c-44e0-9b30-66dddd628fd4 admin admin] Authenticating user token {{(pid=80245) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:31:52.062300 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.fernet_utils [None req-1de4ad99-396c-44e0-9b30-66dddd628fd4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80245) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 08:31:52.106453 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG dbcounter [-] [80245] Writing DB stats keystone:SELECT=93,keystone:DELETE=2,keystone:INSERT=5 {{(pid=80245) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:31:52.162191 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bffea952-66ee-4abd-b93e-6d4c0b04a3cd None None] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:31:52.164074 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1de4ad99-396c-44e0-9b30-66dddd628fd4 admin admin] Validating token access rules against request {{(pid=80245) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:31:52.173853 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.fernet_utils [None req-bffea952-66ee-4abd-b93e-6d4c0b04a3cd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80244) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 08:31:52.183912 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46ca2fe4-49e4-4e6a-95c3-28b0bf541e0e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4f40410b7674687ab5362ac982248ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4faef13aa50e4e36b6c265fb1398fa14', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80245) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:31:52.184614 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46ca2fe4-49e4-4e6a-95c3-28b0bf541e0e service cinder] REQUEST_METHOD: `GET` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:31:52.186621 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46ca2fe4-49e4-4e6a-95c3-28b0bf541e0e service cinder] SCRIPT_NAME: `/identity` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:31:52.186956 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46ca2fe4-49e4-4e6a-95c3-28b0bf541e0e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:31:52.207311 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46ca2fe4-49e4-4e6a-95c3-28b0bf541e0e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:31:52.209010 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46ca2fe4-49e4-4e6a-95c3-28b0bf541e0e service cinder] RBAC: Authorization granted {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:31:52.222675 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: [pid: 80245|app: 0|req: 1107/2215] 10.222.0.47 () {60 vars in 1365 bytes} [Thu Jul 21 08:31:52 2022] GET /identity/v3/auth/tokens => generated 3301 bytes in 165 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 08:31:52.223632 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46ca2fe4-49e4-4e6a-95c3-28b0bf541e0e service cinder] Authenticating user token {{(pid=80245) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:31:52.224997 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.fernet_utils [None req-46ca2fe4-49e4-4e6a-95c3-28b0bf541e0e service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80245) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 08:31:52.287607 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bffea952-66ee-4abd-b93e-6d4c0b04a3cd None None] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:31:52.297535 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0b1104a-b1dd-49a6-af92-72f6c8d0c925 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '42d15fc0515945acacbbffcf2356c7a5', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:31:52.299127 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0b1104a-b1dd-49a6-af92-72f6c8d0c925 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:31:52.299594 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0b1104a-b1dd-49a6-af92-72f6c8d0c925 admin admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:31:52.299830 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0b1104a-b1dd-49a6-af92-72f6c8d0c925 admin admin] PATH_INFO: `/v3/projects/75b9dc5bbdce44e8acb76c36390f5946` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:31:52.303367 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0b1104a-b1dd-49a6-af92-72f6c8d0c925 admin admin] RBAC: Authorizing `identity:delete_project(project_id=75b9dc5bbdce44e8acb76c36390f5946)` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:31:52.308383 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0b1104a-b1dd-49a6-af92-72f6c8d0c925 admin admin] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:31:52.360283 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46ca2fe4-49e4-4e6a-95c3-28b0bf541e0e service cinder] Validating token access rules against request {{(pid=80245) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:31:52.367232 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67a4a53c-2614-43ed-8e18-05a5588517dc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '42d15fc0515945acacbbffcf2356c7a5', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80245) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:31:52.368279 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67a4a53c-2614-43ed-8e18-05a5588517dc admin admin] REQUEST_METHOD: `DELETE` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:31:52.368656 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67a4a53c-2614-43ed-8e18-05a5588517dc admin admin] SCRIPT_NAME: `/identity` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:31:52.369007 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67a4a53c-2614-43ed-8e18-05a5588517dc admin admin] PATH_INFO: `/v3/projects/b76ac06d76d8470dae64c89763e11b9f` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:31:52.370829 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67a4a53c-2614-43ed-8e18-05a5588517dc admin admin] RBAC: Authorizing `identity:delete_project(project_id=b76ac06d76d8470dae64c89763e11b9f)` {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:31:52.373014 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67a4a53c-2614-43ed-8e18-05a5588517dc admin admin] RBAC: Authorization granted {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:31:52.383229 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.notifications [None req-d0b1104a-b1dd-49a6-af92-72f6c8d0c925 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '75b9dc5bbdce44e8acb76c36390f5946', 'request_id': 'req-d0b1104a-b1dd-49a6-af92-72f6c8d0c925'} {{(pid=80244) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 08:31:52.393522 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.notifications [None req-d0b1104a-b1dd-49a6-af92-72f6c8d0c925 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '75b9dc5bbdce44e8acb76c36390f5946', 'request_id': 'req-d0b1104a-b1dd-49a6-af92-72f6c8d0c925'} {{(pid=80244) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 08:31:52.400537 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.notifications [None req-d0b1104a-b1dd-49a6-af92-72f6c8d0c925 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '75b9dc5bbdce44e8acb76c36390f5946', 'request_id': 'req-d0b1104a-b1dd-49a6-af92-72f6c8d0c925'} {{(pid=80244) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 08:31:52.412801 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.notifications [None req-d0b1104a-b1dd-49a6-af92-72f6c8d0c925 admin admin] The token cache is being invalidate because project 75b9dc5bbdce44e8acb76c36390f5946 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80244) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 21 08:31:52.413124 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.notifications [None req-d0b1104a-b1dd-49a6-af92-72f6c8d0c925 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80244) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 08:31:52.413872 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.notifications [None req-d0b1104a-b1dd-49a6-af92-72f6c8d0c925 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80244) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 08:31:52.415798 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 1109/2216] 10.222.0.47 () {58 vars in 1232 bytes} [Thu Jul 21 08:31:52 2022] DELETE /identity/v3/projects/75b9dc5bbdce44e8acb76c36390f5946 => generated 0 bytes in 256 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 08:31:52.416866 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0b1104a-b1dd-49a6-af92-72f6c8d0c925 admin admin] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:31:52.418292 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.fernet_utils [None req-d0b1104a-b1dd-49a6-af92-72f6c8d0c925 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80244) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 08:31:52.441166 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.notifications [None req-67a4a53c-2614-43ed-8e18-05a5588517dc admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b76ac06d76d8470dae64c89763e11b9f', 'request_id': 'req-67a4a53c-2614-43ed-8e18-05a5588517dc'} {{(pid=80245) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 08:31:52.449702 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.notifications [None req-67a4a53c-2614-43ed-8e18-05a5588517dc admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b76ac06d76d8470dae64c89763e11b9f', 'request_id': 'req-67a4a53c-2614-43ed-8e18-05a5588517dc'} {{(pid=80245) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 08:31:52.458487 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.notifications [None req-67a4a53c-2614-43ed-8e18-05a5588517dc admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b76ac06d76d8470dae64c89763e11b9f', 'request_id': 'req-67a4a53c-2614-43ed-8e18-05a5588517dc'} {{(pid=80245) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 08:31:52.485371 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.notifications [None req-67a4a53c-2614-43ed-8e18-05a5588517dc admin admin] The token cache is being invalidate because project b76ac06d76d8470dae64c89763e11b9f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80245) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 21 08:31:52.486384 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.notifications [None req-67a4a53c-2614-43ed-8e18-05a5588517dc admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80245) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 08:31:52.491947 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.notifications [None req-67a4a53c-2614-43ed-8e18-05a5588517dc admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80245) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 08:31:52.496185 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: [pid: 80245|app: 0|req: 1108/2217] 10.222.0.47 () {58 vars in 1232 bytes} [Thu Jul 21 08:31:52 2022] DELETE /identity/v3/projects/b76ac06d76d8470dae64c89763e11b9f => generated 0 bytes in 273 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 08:31:52.498695 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daa130b0-b882-419f-a8b4-ea07183dba62 None None] REQUEST_METHOD: `POST` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:31:52.499445 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daa130b0-b882-419f-a8b4-ea07183dba62 None None] SCRIPT_NAME: `/identity` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:31:52.500100 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daa130b0-b882-419f-a8b4-ea07183dba62 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:31:52.527225 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.auth.core [None req-daa130b0-b882-419f-a8b4-ea07183dba62 None None] MFA Rules not processed for user `2dce40b8d74c41ec94cc36e6ff05d412`. Rule list: `[]` (Enabled: `True`). {{(pid=80245) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 08:31:52.528749 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.fernet_utils [None req-daa130b0-b882-419f-a8b4-ea07183dba62 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80245) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 08:31:52.529217 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0b1104a-b1dd-49a6-af92-72f6c8d0c925 admin admin] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:31:52.560608 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG dbcounter [-] [80244] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80244) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:31:52.596398 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG dbcounter [-] [80245] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80245) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:31:52.606844 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: [pid: 80245|app: 0|req: 1109/2218] 10.222.0.47 () {58 vars in 957 bytes} [Thu Jul 21 08:31:52 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 08:31:52.607180 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f94ddc2e-e196-4ab8-bb94-01f94342c1e0 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4f40410b7674687ab5362ac982248ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4faef13aa50e4e36b6c265fb1398fa14', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:31:52.607952 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f94ddc2e-e196-4ab8-bb94-01f94342c1e0 service cinder] REQUEST_METHOD: `GET` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:31:52.608329 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f94ddc2e-e196-4ab8-bb94-01f94342c1e0 service cinder] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:31:52.608655 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f94ddc2e-e196-4ab8-bb94-01f94342c1e0 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:31:52.608758 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f69bf17e-959a-485c-9b52-2e054c93169c None None] REQUEST_METHOD: `POST` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:31:52.609103 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f69bf17e-959a-485c-9b52-2e054c93169c None None] SCRIPT_NAME: `/identity` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:31:52.609596 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f69bf17e-959a-485c-9b52-2e054c93169c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:31:52.610852 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.fernet_utils [None req-f94ddc2e-e196-4ab8-bb94-01f94342c1e0 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80244) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 08:31:52.632591 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.auth.core [None req-f69bf17e-959a-485c-9b52-2e054c93169c None None] MFA Rules not processed for user `2dce40b8d74c41ec94cc36e6ff05d412`. Rule list: `[]` (Enabled: `True`). {{(pid=80245) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 08:31:52.633963 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.fernet_utils [None req-f69bf17e-959a-485c-9b52-2e054c93169c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80245) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 08:31:52.646278 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: [pid: 80245|app: 0|req: 1110/2219] 10.222.0.47 () {58 vars in 957 bytes} [Thu Jul 21 08:31:52 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 08:31:52.648149 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eb710cc-6d6b-4f7b-86b0-d33b4c118f47 None None] REQUEST_METHOD: `POST` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:31:52.648496 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eb710cc-6d6b-4f7b-86b0-d33b4c118f47 None None] SCRIPT_NAME: `/identity` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:31:52.648834 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eb710cc-6d6b-4f7b-86b0-d33b4c118f47 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:31:52.680807 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.auth.core [None req-3eb710cc-6d6b-4f7b-86b0-d33b4c118f47 None None] MFA Rules not processed for user `2dce40b8d74c41ec94cc36e6ff05d412`. Rule list: `[]` (Enabled: `True`). {{(pid=80245) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 08:31:52.682127 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.fernet_utils [None req-3eb710cc-6d6b-4f7b-86b0-d33b4c118f47 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80245) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 08:31:52.691250 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: [pid: 80245|app: 0|req: 1111/2220] 10.222.0.47 () {58 vars in 957 bytes} [Thu Jul 21 08:31:52 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 44 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 08:31:52.692549 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3e4494c-99ab-4f45-8e78-fd18217ec503 None None] REQUEST_METHOD: `POST` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:31:52.692944 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3e4494c-99ab-4f45-8e78-fd18217ec503 None None] SCRIPT_NAME: `/identity` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:31:52.693328 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3e4494c-99ab-4f45-8e78-fd18217ec503 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:31:52.707962 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f94ddc2e-e196-4ab8-bb94-01f94342c1e0 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:31:52.709339 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f94ddc2e-e196-4ab8-bb94-01f94342c1e0 service cinder] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:31:52.714786 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.auth.core [None req-d3e4494c-99ab-4f45-8e78-fd18217ec503 None None] MFA Rules not processed for user `2dce40b8d74c41ec94cc36e6ff05d412`. Rule list: `[]` (Enabled: `True`). {{(pid=80245) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 08:31:52.716198 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.fernet_utils [None req-d3e4494c-99ab-4f45-8e78-fd18217ec503 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80245) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 08:31:52.720948 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 1110/2221] 10.222.0.47 () {60 vars in 1365 bytes} [Thu Jul 21 08:31:52 2022] GET /identity/v3/auth/tokens => generated 3234 bytes in 304 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 08:31:52.727858 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: [pid: 80245|app: 0|req: 1112/2222] 10.222.0.47 () {58 vars in 957 bytes} [Thu Jul 21 08:31:52 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 08:31:52.728778 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f94ddc2e-e196-4ab8-bb94-01f94342c1e0 service cinder] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:31:52.730145 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3e4494c-99ab-4f45-8e78-fd18217ec503 None None] Authenticating user token {{(pid=80245) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:31:52.745674 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3e4494c-99ab-4f45-8e78-fd18217ec503 None None] Validating token access rules against request {{(pid=80245) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:31:52.750429 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37f08adb-5789-4d3d-8e34-a4b22c6545b9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '42d15fc0515945acacbbffcf2356c7a5', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80245) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:31:52.751284 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37f08adb-5789-4d3d-8e34-a4b22c6545b9 admin admin] REQUEST_METHOD: `GET` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:31:52.751754 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37f08adb-5789-4d3d-8e34-a4b22c6545b9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:31:52.752207 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37f08adb-5789-4d3d-8e34-a4b22c6545b9 admin admin] PATH_INFO: `/v3/domains` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:31:52.753031 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37f08adb-5789-4d3d-8e34-a4b22c6545b9 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80245) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 08:31:52.753449 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37f08adb-5789-4d3d-8e34-a4b22c6545b9 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:31:52.755547 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37f08adb-5789-4d3d-8e34-a4b22c6545b9 admin admin] RBAC: Authorization granted {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:31:52.757734 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f94ddc2e-e196-4ab8-bb94-01f94342c1e0 service cinder] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:31:52.764193 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79828460-135c-4f12-83ef-5b68efdad83b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '42d15fc0515945acacbbffcf2356c7a5', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:31:52.764874 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79828460-135c-4f12-83ef-5b68efdad83b admin admin] REQUEST_METHOD: `GET` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:31:52.765162 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79828460-135c-4f12-83ef-5b68efdad83b admin admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:31:52.765440 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79828460-135c-4f12-83ef-5b68efdad83b admin admin] PATH_INFO: `/v3/domains` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:31:52.766181 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79828460-135c-4f12-83ef-5b68efdad83b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80244) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 08:31:52.766577 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79828460-135c-4f12-83ef-5b68efdad83b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:31:52.768315 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79828460-135c-4f12-83ef-5b68efdad83b admin admin] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:31:52.783895 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: [pid: 80245|app: 0|req: 1113/2223] 10.222.0.47 () {58 vars in 1152 bytes} [Thu Jul 21 08:31:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 54 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 08:31:52.788549 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37f08adb-5789-4d3d-8e34-a4b22c6545b9 admin admin] Authenticating user token {{(pid=80245) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:31:52.788945 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 1111/2224] 10.222.0.47 () {58 vars in 1152 bytes} [Thu Jul 21 08:31:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 66 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 08:31:52.793069 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79828460-135c-4f12-83ef-5b68efdad83b admin admin] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:31:52.800262 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37f08adb-5789-4d3d-8e34-a4b22c6545b9 admin admin] Validating token access rules against request {{(pid=80245) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:31:52.805636 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d826f3e-2e2b-425c-bb4a-07e08d37fec9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '42d15fc0515945acacbbffcf2356c7a5', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80245) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:31:52.806799 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d826f3e-2e2b-425c-bb4a-07e08d37fec9 admin admin] REQUEST_METHOD: `POST` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:31:52.807274 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d826f3e-2e2b-425c-bb4a-07e08d37fec9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:31:52.808243 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d826f3e-2e2b-425c-bb4a-07e08d37fec9 admin admin] PATH_INFO: `/v3/projects` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:31:52.808378 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79828460-135c-4f12-83ef-5b68efdad83b admin admin] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:31:52.810683 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d826f3e-2e2b-425c-bb4a-07e08d37fec9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:31:52.813125 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d826f3e-2e2b-425c-bb4a-07e08d37fec9 admin admin] RBAC: Authorization granted {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:31:52.821810 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e1e1fc5-193a-44f9-8020-6c5c79b2f5e7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '42d15fc0515945acacbbffcf2356c7a5', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:31:52.822484 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e1e1fc5-193a-44f9-8020-6c5c79b2f5e7 admin admin] REQUEST_METHOD: `POST` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:31:52.822767 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e1e1fc5-193a-44f9-8020-6c5c79b2f5e7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:31:52.823146 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e1e1fc5-193a-44f9-8020-6c5c79b2f5e7 admin admin] PATH_INFO: `/v3/projects` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:31:52.824379 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e1e1fc5-193a-44f9-8020-6c5c79b2f5e7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:31:52.826284 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e1e1fc5-193a-44f9-8020-6c5c79b2f5e7 admin admin] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:31:52.868656 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 1112/2225] 10.222.0.47 () {60 vars in 1152 bytes} [Thu Jul 21 08:31:52 2022] POST /identity/v3/projects => generated 375 bytes in 75 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 08:31:52.868656 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e1e1fc5-193a-44f9-8020-6c5c79b2f5e7 admin admin] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:31:52.870838 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.fernet_utils [None req-2e1e1fc5-193a-44f9-8020-6c5c79b2f5e7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80244) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 08:31:52.875673 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: [pid: 80245|app: 0|req: 1114/2226] 10.222.0.47 () {60 vars in 1152 bytes} [Thu Jul 21 08:31:52 2022] POST /identity/v3/projects => generated 373 bytes in 87 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 21 08:31:52.876935 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d826f3e-2e2b-425c-bb4a-07e08d37fec9 admin admin] Authenticating user token {{(pid=80245) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:31:52.960891 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e1e1fc5-193a-44f9-8020-6c5c79b2f5e7 admin admin] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:31:52.996514 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d826f3e-2e2b-425c-bb4a-07e08d37fec9 admin admin] Validating token access rules against request {{(pid=80245) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:31:53.002044 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cee26b0-1ee6-48ad-b253-f3cc884b7bd0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '42d15fc0515945acacbbffcf2356c7a5', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80245) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:31:53.002854 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cee26b0-1ee6-48ad-b253-f3cc884b7bd0 admin admin] REQUEST_METHOD: `POST` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:31:53.003824 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cee26b0-1ee6-48ad-b253-f3cc884b7bd0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:31:53.004246 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cee26b0-1ee6-48ad-b253-f3cc884b7bd0 admin admin] PATH_INFO: `/v3/users` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:31:53.005438 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cee26b0-1ee6-48ad-b253-f3cc884b7bd0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:31:53.007689 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cee26b0-1ee6-48ad-b253-f3cc884b7bd0 admin admin] RBAC: Authorization granted {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:31:53.041386 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG dbcounter [-] [80244] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80244) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:31:53.051369 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: [pid: 80245|app: 0|req: 1115/2227] 10.222.0.47 () {60 vars in 1143 bytes} [Thu Jul 21 08:31:52 2022] POST /identity/v3/users => generated 288 bytes in 175 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 08:31:53.057882 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cee26b0-1ee6-48ad-b253-f3cc884b7bd0 admin admin] Authenticating user token {{(pid=80245) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:31:53.074579 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcbc1d20-5635-44ba-9774-00513deb2423 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '42d15fc0515945acacbbffcf2356c7a5', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:31:53.075584 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcbc1d20-5635-44ba-9774-00513deb2423 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:31:53.075945 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcbc1d20-5635-44ba-9774-00513deb2423 admin admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:31:53.076676 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcbc1d20-5635-44ba-9774-00513deb2423 admin admin] PATH_INFO: `/v3/users/9a84db7484ad48089fc1303fa54b8670` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:31:53.078017 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcbc1d20-5635-44ba-9774-00513deb2423 admin admin] RBAC: Authorizing `identity:delete_user(user_id=9a84db7484ad48089fc1303fa54b8670)` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:31:53.079975 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcbc1d20-5635-44ba-9774-00513deb2423 admin admin] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:31:53.081530 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cee26b0-1ee6-48ad-b253-f3cc884b7bd0 admin admin] Validating token access rules against request {{(pid=80245) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:31:53.098444 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71451d87-6102-4d41-b659-bc9ffbbdc120 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '42d15fc0515945acacbbffcf2356c7a5', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80245) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:31:53.100967 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71451d87-6102-4d41-b659-bc9ffbbdc120 admin admin] REQUEST_METHOD: `POST` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:31:53.101633 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71451d87-6102-4d41-b659-bc9ffbbdc120 admin admin] SCRIPT_NAME: `/identity` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:31:53.102633 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71451d87-6102-4d41-b659-bc9ffbbdc120 admin admin] PATH_INFO: `/v3/users` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:31:53.103868 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71451d87-6102-4d41-b659-bc9ffbbdc120 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:31:53.106287 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71451d87-6102-4d41-b659-bc9ffbbdc120 admin admin] RBAC: Authorization granted {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:31:53.155190 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: [pid: 80245|app: 0|req: 1116/2228] 10.222.0.47 () {60 vars in 1143 bytes} [Thu Jul 21 08:31:53 2022] POST /identity/v3/users => generated 287 bytes in 100 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 08:31:53.156087 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71451d87-6102-4d41-b659-bc9ffbbdc120 admin admin] Authenticating user token {{(pid=80245) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:31:53.160996 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.notifications [None req-fcbc1d20-5635-44ba-9774-00513deb2423 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9a84db7484ad48089fc1303fa54b8670'} {{(pid=80244) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 08:31:53.171194 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.notifications [None req-fcbc1d20-5635-44ba-9774-00513deb2423 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9a84db7484ad48089fc1303fa54b8670'} {{(pid=80244) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 08:31:53.174927 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71451d87-6102-4d41-b659-bc9ffbbdc120 admin admin] Validating token access rules against request {{(pid=80245) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:31:53.177931 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93817fca-85f3-470c-bfba-8972b6ac605b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '42d15fc0515945acacbbffcf2356c7a5', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80245) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:31:53.178613 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93817fca-85f3-470c-bfba-8972b6ac605b admin admin] REQUEST_METHOD: `GET` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:31:53.178912 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93817fca-85f3-470c-bfba-8972b6ac605b admin admin] SCRIPT_NAME: `/identity` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:31:53.179571 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93817fca-85f3-470c-bfba-8972b6ac605b admin admin] PATH_INFO: `/v3/roles` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:31:53.179940 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93817fca-85f3-470c-bfba-8972b6ac605b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:31:53.182007 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93817fca-85f3-470c-bfba-8972b6ac605b admin admin] RBAC: Authorization granted {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:31:53.196305 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: [pid: 80245|app: 0|req: 1117/2229] 10.222.0.47 () {58 vars in 1121 bytes} [Thu Jul 21 08:31:53 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 08:31:53.197446 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93817fca-85f3-470c-bfba-8972b6ac605b admin admin] Authenticating user token {{(pid=80245) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:31:53.217714 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93817fca-85f3-470c-bfba-8972b6ac605b admin admin] Validating token access rules against request {{(pid=80245) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:31:53.223273 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72ec9022-06b5-4f3f-bd2d-c3b9ab1c49f5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '42d15fc0515945acacbbffcf2356c7a5', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80245) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:31:53.224267 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72ec9022-06b5-4f3f-bd2d-c3b9ab1c49f5 admin admin] REQUEST_METHOD: `GET` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:31:53.224660 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72ec9022-06b5-4f3f-bd2d-c3b9ab1c49f5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:31:53.225042 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72ec9022-06b5-4f3f-bd2d-c3b9ab1c49f5 admin admin] PATH_INFO: `/v3/roles` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:31:53.225903 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72ec9022-06b5-4f3f-bd2d-c3b9ab1c49f5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:31:53.228776 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72ec9022-06b5-4f3f-bd2d-c3b9ab1c49f5 admin admin] RBAC: Authorization granted {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:31:53.230764 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.notifications [None req-fcbc1d20-5635-44ba-9774-00513deb2423 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9a84db7484ad48089fc1303fa54b8670'} {{(pid=80244) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 08:31:53.231737 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.notifications [None req-fcbc1d20-5635-44ba-9774-00513deb2423 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9a84db7484ad48089fc1303fa54b8670'} {{(pid=80244) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 08:31:53.246033 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: [pid: 80245|app: 0|req: 1118/2230] 10.222.0.47 () {58 vars in 1121 bytes} [Thu Jul 21 08:31:53 2022] GET /identity/v3/roles => generated 1420 bytes in 49 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 08:31:53.247040 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72ec9022-06b5-4f3f-bd2d-c3b9ab1c49f5 admin admin] Authenticating user token {{(pid=80245) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:31:53.248943 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.fernet_utils [None req-72ec9022-06b5-4f3f-bd2d-c3b9ab1c49f5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80245) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 08:31:53.252980 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.notifications [None req-fcbc1d20-5635-44ba-9774-00513deb2423 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9a84db7484ad48089fc1303fa54b8670'} {{(pid=80244) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 08:31:53.265539 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72ec9022-06b5-4f3f-bd2d-c3b9ab1c49f5 admin admin] Validating token access rules against request {{(pid=80245) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:31:53.266285 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 1113/2231] 10.222.0.47 () {58 vars in 1223 bytes} [Thu Jul 21 08:31:52 2022] DELETE /identity/v3/users/9a84db7484ad48089fc1303fa54b8670 => generated 0 bytes in 398 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 08:31:53.267449 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcbc1d20-5635-44ba-9774-00513deb2423 admin admin] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:31:53.268881 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.fernet_utils [None req-fcbc1d20-5635-44ba-9774-00513deb2423 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80244) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 08:31:53.365979 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG dbcounter [-] [80245] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=80245) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:31:53.377347 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a1d3a64-8504-4e41-9933-778853bb8edb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '42d15fc0515945acacbbffcf2356c7a5', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80245) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:31:53.378468 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a1d3a64-8504-4e41-9933-778853bb8edb admin admin] REQUEST_METHOD: `GET` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:31:53.378831 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a1d3a64-8504-4e41-9933-778853bb8edb admin admin] SCRIPT_NAME: `/identity` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:31:53.379468 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a1d3a64-8504-4e41-9933-778853bb8edb admin admin] PATH_INFO: `/v3/roles` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:31:53.380299 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a1d3a64-8504-4e41-9933-778853bb8edb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:31:53.382321 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a1d3a64-8504-4e41-9933-778853bb8edb admin admin] RBAC: Authorization granted {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:31:53.383369 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcbc1d20-5635-44ba-9774-00513deb2423 admin admin] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:31:53.391173 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f71d92a-1aaa-483f-b7f5-c7d0f1a8d27b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '42d15fc0515945acacbbffcf2356c7a5', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:31:53.392684 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f71d92a-1aaa-483f-b7f5-c7d0f1a8d27b admin admin] REQUEST_METHOD: `GET` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:31:53.393138 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f71d92a-1aaa-483f-b7f5-c7d0f1a8d27b admin admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:31:53.394252 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f71d92a-1aaa-483f-b7f5-c7d0f1a8d27b admin admin] PATH_INFO: `/v3/roles` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:31:53.395513 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f71d92a-1aaa-483f-b7f5-c7d0f1a8d27b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:31:53.396446 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: [pid: 80245|app: 0|req: 1119/2232] 10.222.0.47 () {58 vars in 1121 bytes} [Thu Jul 21 08:31:53 2022] GET /identity/v3/roles => generated 1420 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 08:31:53.397681 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f71d92a-1aaa-483f-b7f5-c7d0f1a8d27b admin admin] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:31:53.398724 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5d2cfa0-04b2-43d2-9fbd-2cbc3adbb9d3 None None] REQUEST_METHOD: `POST` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:31:53.399034 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5d2cfa0-04b2-43d2-9fbd-2cbc3adbb9d3 None None] SCRIPT_NAME: `/identity` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:31:53.399588 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5d2cfa0-04b2-43d2-9fbd-2cbc3adbb9d3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:31:53.416299 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 1114/2233] 10.222.0.47 () {58 vars in 1121 bytes} [Thu Jul 21 08:31:53 2022] GET /identity/v3/roles => generated 1420 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 08:31:53.418834 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f71d92a-1aaa-483f-b7f5-c7d0f1a8d27b admin admin] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:31:53.427214 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.auth.core [None req-a5d2cfa0-04b2-43d2-9fbd-2cbc3adbb9d3 None None] MFA Rules not processed for user `2dce40b8d74c41ec94cc36e6ff05d412`. Rule list: `[]` (Enabled: `True`). {{(pid=80245) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 08:31:53.429153 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f71d92a-1aaa-483f-b7f5-c7d0f1a8d27b admin admin] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:31:53.429539 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.fernet_utils [None req-a5d2cfa0-04b2-43d2-9fbd-2cbc3adbb9d3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80245) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 08:31:53.436861 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2380e67b-78d9-4530-8868-702e207aedbb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '42d15fc0515945acacbbffcf2356c7a5', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:31:53.437815 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2380e67b-78d9-4530-8868-702e207aedbb admin admin] REQUEST_METHOD: `PUT` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:31:53.438131 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2380e67b-78d9-4530-8868-702e207aedbb admin admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:31:53.438407 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2380e67b-78d9-4530-8868-702e207aedbb admin admin] PATH_INFO: `/v3/projects/65d666f18695484aab416d8b0d902efb/users/5d5fe2099e0845eabc2eb9ce089d03cd/roles/3a074738221e4601b5205715ba87d240` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:31:53.447576 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: [pid: 80245|app: 0|req: 1120/2234] 10.222.0.47 () {58 vars in 957 bytes} [Thu Jul 21 08:31:53 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 51 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 08:31:53.448673 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5d2cfa0-04b2-43d2-9fbd-2cbc3adbb9d3 None None] Authenticating user token {{(pid=80245) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:31:53.466914 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5d2cfa0-04b2-43d2-9fbd-2cbc3adbb9d3 None None] Validating token access rules against request {{(pid=80245) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:31:53.467857 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2380e67b-78d9-4530-8868-702e207aedbb admin admin] RBAC: Authorizing `identity:create_grant(project_id=65d666f18695484aab416d8b0d902efb, user_id=5d5fe2099e0845eabc2eb9ce089d03cd, role_id=3a074738221e4601b5205715ba87d240)` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:31:53.472422 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2380e67b-78d9-4530-8868-702e207aedbb admin admin] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:31:53.478292 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98efab6f-e6cf-4540-935e-93abafa7b69c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '42d15fc0515945acacbbffcf2356c7a5', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80245) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:31:53.479768 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98efab6f-e6cf-4540-935e-93abafa7b69c admin admin] REQUEST_METHOD: `PUT` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:31:53.480075 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98efab6f-e6cf-4540-935e-93abafa7b69c admin admin] SCRIPT_NAME: `/identity` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:31:53.480350 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98efab6f-e6cf-4540-935e-93abafa7b69c admin admin] PATH_INFO: `/v3/projects/6c163cbb79544869bb823308163a4142/users/aa156584880d4449bd89a7bcdcfe403f/roles/3a074738221e4601b5205715ba87d240` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:31:53.506651 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 1115/2235] 10.222.0.47 () {60 vars in 1482 bytes} [Thu Jul 21 08:31:53 2022] PUT /identity/v3/projects/65d666f18695484aab416d8b0d902efb/users/5d5fe2099e0845eabc2eb9ce089d03cd/roles/3a074738221e4601b5205715ba87d240 => generated 0 bytes in 89 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 08:31:53.511298 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98efab6f-e6cf-4540-935e-93abafa7b69c admin admin] RBAC: Authorizing `identity:create_grant(project_id=6c163cbb79544869bb823308163a4142, user_id=aa156584880d4449bd89a7bcdcfe403f, role_id=3a074738221e4601b5205715ba87d240)` {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:31:53.513560 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2380e67b-78d9-4530-8868-702e207aedbb admin admin] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:31:53.514043 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98efab6f-e6cf-4540-935e-93abafa7b69c admin admin] RBAC: Authorization granted {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:31:53.515411 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.fernet_utils [None req-2380e67b-78d9-4530-8868-702e207aedbb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80244) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 08:31:53.532983 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: [pid: 80245|app: 0|req: 1121/2236] 10.222.0.47 () {60 vars in 1482 bytes} [Thu Jul 21 08:31:53 2022] PUT /identity/v3/projects/6c163cbb79544869bb823308163a4142/users/aa156584880d4449bd89a7bcdcfe403f/roles/3a074738221e4601b5205715ba87d240 => generated 0 bytes in 85 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 08:31:53.537439 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12008c70-80fb-46fa-af37-9021d8b6221f None None] REQUEST_METHOD: `POST` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:31:53.537889 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12008c70-80fb-46fa-af37-9021d8b6221f None None] SCRIPT_NAME: `/identity` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:31:53.538201 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12008c70-80fb-46fa-af37-9021d8b6221f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:31:53.568596 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.auth.core [None req-12008c70-80fb-46fa-af37-9021d8b6221f None None] MFA Rules not processed for user `5d5fe2099e0845eabc2eb9ce089d03cd`. Rule list: `[]` (Enabled: `True`). {{(pid=80245) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 08:31:53.569845 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.fernet_utils [None req-12008c70-80fb-46fa-af37-9021d8b6221f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80245) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 08:31:53.581363 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG dbcounter [-] [80244] Writing DB stats keystone:SELECT=88,keystone:DELETE=10,keystone:INSERT=2 {{(pid=80244) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:31:53.607130 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2380e67b-78d9-4530-8868-702e207aedbb admin admin] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:31:53.674717 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: [pid: 80245|app: 0|req: 1122/2237] 10.222.0.47 () {58 vars in 957 bytes} [Thu Jul 21 08:31:53 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 138 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 08:31:53.674717 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56ad2b43-4711-4cf4-b1f0-989030e93da5 None None] REQUEST_METHOD: `POST` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:31:53.674717 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56ad2b43-4711-4cf4-b1f0-989030e93da5 None None] SCRIPT_NAME: `/identity` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:31:53.674717 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56ad2b43-4711-4cf4-b1f0-989030e93da5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:31:53.691408 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c625580-ddaf-42bb-972f-727070943736 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20ca3c3bd3774691b3374cfbfec6e494', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4faef13aa50e4e36b6c265fb1398fa14', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:31:53.692333 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c625580-ddaf-42bb-972f-727070943736 service neutron] REQUEST_METHOD: `GET` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:31:53.692873 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c625580-ddaf-42bb-972f-727070943736 service neutron] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:31:53.693356 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c625580-ddaf-42bb-972f-727070943736 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:31:53.694096 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.auth.core [None req-56ad2b43-4711-4cf4-b1f0-989030e93da5 None None] MFA Rules not processed for user `2dce40b8d74c41ec94cc36e6ff05d412`. Rule list: `[]` (Enabled: `True`). {{(pid=80245) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 08:31:53.695450 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.fernet_utils [None req-56ad2b43-4711-4cf4-b1f0-989030e93da5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80245) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 08:31:53.779667 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: [pid: 80245|app: 0|req: 1123/2238] 10.222.0.47 () {58 vars in 957 bytes} [Thu Jul 21 08:31:53 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 08:31:53.781151 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56ad2b43-4711-4cf4-b1f0-989030e93da5 None None] Authenticating user token {{(pid=80245) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:31:53.787921 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.fernet_utils [None req-56ad2b43-4711-4cf4-b1f0-989030e93da5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80245) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 08:31:53.804737 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c625580-ddaf-42bb-972f-727070943736 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:31:53.806241 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c625580-ddaf-42bb-972f-727070943736 service neutron] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:31:53.820099 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 1116/2239] 10.222.0.47 () {60 vars in 1365 bytes} [Thu Jul 21 08:31:53 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 307 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 21 08:31:53.821565 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c625580-ddaf-42bb-972f-727070943736 service neutron] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:31:53.823219 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.fernet_utils [None req-4c625580-ddaf-42bb-972f-727070943736 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80244) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 08:31:53.837980 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c625580-ddaf-42bb-972f-727070943736 service neutron] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:31:53.845236 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef4f8808-fc73-463e-a360-836ae74e9666 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20ca3c3bd3774691b3374cfbfec6e494', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4faef13aa50e4e36b6c265fb1398fa14', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:31:53.847774 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef4f8808-fc73-463e-a360-836ae74e9666 service neutron] REQUEST_METHOD: `GET` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:31:53.848065 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef4f8808-fc73-463e-a360-836ae74e9666 service neutron] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:31:53.848346 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef4f8808-fc73-463e-a360-836ae74e9666 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:31:53.866360 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef4f8808-fc73-463e-a360-836ae74e9666 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:31:53.868175 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef4f8808-fc73-463e-a360-836ae74e9666 service neutron] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:31:53.878225 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56ad2b43-4711-4cf4-b1f0-989030e93da5 None None] Validating token access rules against request {{(pid=80245) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:31:53.887228 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fbd1632-df96-488b-b8a0-d4a112e14d63 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4f40410b7674687ab5362ac982248ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4faef13aa50e4e36b6c265fb1398fa14', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80245) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:31:53.888945 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fbd1632-df96-488b-b8a0-d4a112e14d63 service cinder] REQUEST_METHOD: `GET` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:31:53.889561 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fbd1632-df96-488b-b8a0-d4a112e14d63 service cinder] SCRIPT_NAME: `/identity` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:31:53.889871 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fbd1632-df96-488b-b8a0-d4a112e14d63 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:31:53.890603 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 1117/2240] 10.222.0.47 () {60 vars in 1365 bytes} [Thu Jul 21 08:31:53 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 69 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 08:31:53.898839 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG dbcounter [-] [80245] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80245) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:31:53.906587 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fbd1632-df96-488b-b8a0-d4a112e14d63 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:31:53.908486 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fbd1632-df96-488b-b8a0-d4a112e14d63 service cinder] RBAC: Authorization granted {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:31:53.919409 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: [pid: 80245|app: 0|req: 1124/2241] 10.222.0.47 () {60 vars in 1365 bytes} [Thu Jul 21 08:31:53 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 139 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 08:31:54.619156 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef4f8808-fc73-463e-a360-836ae74e9666 service neutron] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:31:54.621183 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.fernet_utils [None req-ef4f8808-fc73-463e-a360-836ae74e9666 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80244) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 08:31:54.650608 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef4f8808-fc73-463e-a360-836ae74e9666 service neutron] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:31:54.655003 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e800e0b-95a4-4ee3-a2b7-ac51bc7056e9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '42d15fc0515945acacbbffcf2356c7a5', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:31:54.655942 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e800e0b-95a4-4ee3-a2b7-ac51bc7056e9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:31:54.656227 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e800e0b-95a4-4ee3-a2b7-ac51bc7056e9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:31:54.656511 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e800e0b-95a4-4ee3-a2b7-ac51bc7056e9 admin admin] PATH_INFO: `/v3/projects/63233c4fbfb843c894d87f2d4af28067` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:31:54.658703 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e800e0b-95a4-4ee3-a2b7-ac51bc7056e9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=63233c4fbfb843c894d87f2d4af28067)` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:31:54.660903 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e800e0b-95a4-4ee3-a2b7-ac51bc7056e9 admin admin] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:31:54.737935 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.notifications [None req-1e800e0b-95a4-4ee3-a2b7-ac51bc7056e9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '63233c4fbfb843c894d87f2d4af28067', 'request_id': 'req-1e800e0b-95a4-4ee3-a2b7-ac51bc7056e9'} {{(pid=80244) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 08:31:54.746835 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.notifications [None req-1e800e0b-95a4-4ee3-a2b7-ac51bc7056e9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '63233c4fbfb843c894d87f2d4af28067', 'request_id': 'req-1e800e0b-95a4-4ee3-a2b7-ac51bc7056e9'} {{(pid=80244) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 08:31:54.753932 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.notifications [None req-1e800e0b-95a4-4ee3-a2b7-ac51bc7056e9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '63233c4fbfb843c894d87f2d4af28067', 'request_id': 'req-1e800e0b-95a4-4ee3-a2b7-ac51bc7056e9'} {{(pid=80244) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 08:31:54.764998 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.notifications [None req-1e800e0b-95a4-4ee3-a2b7-ac51bc7056e9 admin admin] The token cache is being invalidate because project 63233c4fbfb843c894d87f2d4af28067 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80244) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 21 08:31:54.765289 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.notifications [None req-1e800e0b-95a4-4ee3-a2b7-ac51bc7056e9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80244) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 08:31:54.765944 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.notifications [None req-1e800e0b-95a4-4ee3-a2b7-ac51bc7056e9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80244) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 08:31:54.767592 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 1118/2242] 10.222.0.47 () {58 vars in 1232 bytes} [Thu Jul 21 08:31:54 2022] DELETE /identity/v3/projects/63233c4fbfb843c894d87f2d4af28067 => generated 0 bytes in 149 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 08:31:54.772542 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c676cd15-f529-473d-808d-da018b803f5a None None] REQUEST_METHOD: `POST` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:31:54.773071 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c676cd15-f529-473d-808d-da018b803f5a None None] SCRIPT_NAME: `/identity` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:31:54.773184 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c676cd15-f529-473d-808d-da018b803f5a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:31:54.795747 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.auth.core [None req-c676cd15-f529-473d-808d-da018b803f5a None None] MFA Rules not processed for user `2dce40b8d74c41ec94cc36e6ff05d412`. Rule list: `[]` (Enabled: `True`). {{(pid=80245) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 08:31:54.797069 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.fernet_utils [None req-c676cd15-f529-473d-808d-da018b803f5a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80245) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 08:31:54.873100 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: [pid: 80245|app: 0|req: 1125/2243] 10.222.0.47 () {58 vars in 957 bytes} [Thu Jul 21 08:31:54 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 08:31:54.881965 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18e64eaf-8d61-487f-b128-8ca576ffc17a None None] REQUEST_METHOD: `POST` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:31:54.882867 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18e64eaf-8d61-487f-b128-8ca576ffc17a None None] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:31:54.882867 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18e64eaf-8d61-487f-b128-8ca576ffc17a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:31:54.907737 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.auth.core [None req-18e64eaf-8d61-487f-b128-8ca576ffc17a None None] MFA Rules not processed for user `2dce40b8d74c41ec94cc36e6ff05d412`. Rule list: `[]` (Enabled: `True`). {{(pid=80244) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 08:31:54.909072 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.fernet_utils [None req-18e64eaf-8d61-487f-b128-8ca576ffc17a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80244) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 08:31:54.918506 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 1119/2244] 10.222.0.47 () {58 vars in 957 bytes} [Thu Jul 21 08:31:54 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 08:31:54.923116 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c676cd15-f529-473d-808d-da018b803f5a None None] Authenticating user token {{(pid=80245) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:31:54.934970 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c676cd15-f529-473d-808d-da018b803f5a None None] Validating token access rules against request {{(pid=80245) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:31:54.939988 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11a345f4-ae8e-4225-81c3-8a9cec151c22 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '42d15fc0515945acacbbffcf2356c7a5', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80245) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:31:54.940671 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11a345f4-ae8e-4225-81c3-8a9cec151c22 admin admin] REQUEST_METHOD: `GET` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:31:54.940956 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11a345f4-ae8e-4225-81c3-8a9cec151c22 admin admin] SCRIPT_NAME: `/identity` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:31:54.941237 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11a345f4-ae8e-4225-81c3-8a9cec151c22 admin admin] PATH_INFO: `/v3/domains` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:31:54.942055 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11a345f4-ae8e-4225-81c3-8a9cec151c22 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80245) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 08:31:54.942368 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11a345f4-ae8e-4225-81c3-8a9cec151c22 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:31:54.944222 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11a345f4-ae8e-4225-81c3-8a9cec151c22 admin admin] RBAC: Authorization granted {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:31:54.957417 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: [pid: 80245|app: 0|req: 1126/2245] 10.222.0.47 () {58 vars in 1152 bytes} [Thu Jul 21 08:31:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 21 08:31:54.972515 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18e64eaf-8d61-487f-b128-8ca576ffc17a None None] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:31:54.993605 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18e64eaf-8d61-487f-b128-8ca576ffc17a None None] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:31:54.998991 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0f65545-ed64-4480-8368-13ffaf55c419 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '42d15fc0515945acacbbffcf2356c7a5', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:31:54.999749 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0f65545-ed64-4480-8368-13ffaf55c419 admin admin] REQUEST_METHOD: `POST` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:31:55.000039 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0f65545-ed64-4480-8368-13ffaf55c419 admin admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:31:55.000320 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0f65545-ed64-4480-8368-13ffaf55c419 admin admin] PATH_INFO: `/v3/projects` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:31:55.001055 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0f65545-ed64-4480-8368-13ffaf55c419 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:31:55.003192 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0f65545-ed64-4480-8368-13ffaf55c419 admin admin] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:31:55.008503 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG dbcounter [-] [80244] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80244) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:31:55.046092 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 1120/2246] 10.222.0.47 () {60 vars in 1152 bytes} [Thu Jul 21 08:31:54 2022] POST /identity/v3/projects => generated 381 bytes in 74 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 08:31:55.052425 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11a345f4-ae8e-4225-81c3-8a9cec151c22 admin admin] Authenticating user token {{(pid=80245) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:31:55.063760 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11a345f4-ae8e-4225-81c3-8a9cec151c22 admin admin] Validating token access rules against request {{(pid=80245) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:31:55.186542 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52a3de58-5a93-46be-b4ff-73546466815f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '42d15fc0515945acacbbffcf2356c7a5', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80245) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:31:55.187602 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52a3de58-5a93-46be-b4ff-73546466815f admin admin] REQUEST_METHOD: `POST` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:31:55.188078 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52a3de58-5a93-46be-b4ff-73546466815f admin admin] SCRIPT_NAME: `/identity` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:31:55.188542 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52a3de58-5a93-46be-b4ff-73546466815f admin admin] PATH_INFO: `/v3/users` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:31:55.189624 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52a3de58-5a93-46be-b4ff-73546466815f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:31:55.191994 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52a3de58-5a93-46be-b4ff-73546466815f admin admin] RBAC: Authorization granted {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:31:55.237931 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: [pid: 80245|app: 0|req: 1127/2247] 10.222.0.47 () {60 vars in 1143 bytes} [Thu Jul 21 08:31:55 2022] POST /identity/v3/users => generated 291 bytes in 185 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 08:31:55.239173 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0f65545-ed64-4480-8368-13ffaf55c419 admin admin] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:31:55.257740 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0f65545-ed64-4480-8368-13ffaf55c419 admin admin] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:31:55.265342 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a16b0106-1a4b-45b4-9afb-4228177af41f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '42d15fc0515945acacbbffcf2356c7a5', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:31:55.266122 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a16b0106-1a4b-45b4-9afb-4228177af41f admin admin] REQUEST_METHOD: `GET` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:31:55.266349 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a16b0106-1a4b-45b4-9afb-4228177af41f admin admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:31:55.266632 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a16b0106-1a4b-45b4-9afb-4228177af41f admin admin] PATH_INFO: `/v3/roles` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:31:55.267412 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a16b0106-1a4b-45b4-9afb-4228177af41f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:31:55.269088 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a16b0106-1a4b-45b4-9afb-4228177af41f admin admin] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:31:55.304859 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52a3de58-5a93-46be-b4ff-73546466815f admin admin] Authenticating user token {{(pid=80245) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:31:55.309881 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 1121/2248] 10.222.0.47 () {58 vars in 1121 bytes} [Thu Jul 21 08:31:55 2022] GET /identity/v3/roles => generated 1420 bytes in 71 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 08:31:55.327368 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52a3de58-5a93-46be-b4ff-73546466815f admin admin] Validating token access rules against request {{(pid=80245) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:31:55.354648 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-999d5043-6934-40d8-bcce-7585fe2642d6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '42d15fc0515945acacbbffcf2356c7a5', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80245) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:31:55.356125 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-999d5043-6934-40d8-bcce-7585fe2642d6 admin admin] REQUEST_METHOD: `GET` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:31:55.356415 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-999d5043-6934-40d8-bcce-7585fe2642d6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:31:55.356701 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-999d5043-6934-40d8-bcce-7585fe2642d6 admin admin] PATH_INFO: `/v3/roles` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:31:55.357480 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-999d5043-6934-40d8-bcce-7585fe2642d6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:31:55.373964 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-999d5043-6934-40d8-bcce-7585fe2642d6 admin admin] RBAC: Authorization granted {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:31:55.376420 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36dae6a4-db14-4083-bf2a-a89e896a7db3 None None] REQUEST_METHOD: `POST` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:31:55.376722 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36dae6a4-db14-4083-bf2a-a89e896a7db3 None None] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:31:55.377549 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36dae6a4-db14-4083-bf2a-a89e896a7db3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:31:55.396788 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: [pid: 80245|app: 0|req: 1128/2249] 10.222.0.47 () {58 vars in 1121 bytes} [Thu Jul 21 08:31:55 2022] GET /identity/v3/roles => generated 1420 bytes in 90 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 21 08:31:55.397985 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-999d5043-6934-40d8-bcce-7585fe2642d6 admin admin] Authenticating user token {{(pid=80245) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:31:55.410716 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-999d5043-6934-40d8-bcce-7585fe2642d6 admin admin] Validating token access rules against request {{(pid=80245) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:31:55.418889 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c015ca25-7e9b-45f4-9601-53e0b84ea7eb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '42d15fc0515945acacbbffcf2356c7a5', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80245) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:31:55.426591 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c015ca25-7e9b-45f4-9601-53e0b84ea7eb admin admin] REQUEST_METHOD: `PUT` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:31:55.427146 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c015ca25-7e9b-45f4-9601-53e0b84ea7eb admin admin] SCRIPT_NAME: `/identity` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:31:55.428146 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c015ca25-7e9b-45f4-9601-53e0b84ea7eb admin admin] PATH_INFO: `/v3/projects/ddde32ff934748c487b54e47f33b4125/users/c7e96b1679ae4ce9aec31209202a894a/roles/3a074738221e4601b5205715ba87d240` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:31:55.431213 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.auth.core [None req-36dae6a4-db14-4083-bf2a-a89e896a7db3 None None] MFA Rules not processed for user `aa156584880d4449bd89a7bcdcfe403f`. Rule list: `[]` (Enabled: `True`). {{(pid=80244) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 08:31:55.433053 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.fernet_utils [None req-36dae6a4-db14-4083-bf2a-a89e896a7db3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80244) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 08:31:55.481612 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c015ca25-7e9b-45f4-9601-53e0b84ea7eb admin admin] RBAC: Authorizing `identity:create_grant(project_id=ddde32ff934748c487b54e47f33b4125, user_id=c7e96b1679ae4ce9aec31209202a894a, role_id=3a074738221e4601b5205715ba87d240)` {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:31:55.485073 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c015ca25-7e9b-45f4-9601-53e0b84ea7eb admin admin] RBAC: Authorization granted {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:31:55.516720 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: [pid: 80245|app: 0|req: 1129/2250] 10.222.0.47 () {60 vars in 1482 bytes} [Thu Jul 21 08:31:55 2022] PUT /identity/v3/projects/ddde32ff934748c487b54e47f33b4125/users/c7e96b1679ae4ce9aec31209202a894a/roles/3a074738221e4601b5205715ba87d240 => generated 0 bytes in 119 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 21 08:31:55.529784 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db60e66a-f7e6-41ab-9891-66c313fbf19c None None] REQUEST_METHOD: `POST` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:31:55.530902 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db60e66a-f7e6-41ab-9891-66c313fbf19c None None] SCRIPT_NAME: `/identity` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:31:55.531211 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db60e66a-f7e6-41ab-9891-66c313fbf19c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:31:55.578352 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.auth.core [None req-db60e66a-f7e6-41ab-9891-66c313fbf19c None None] MFA Rules not processed for user `c7e96b1679ae4ce9aec31209202a894a`. Rule list: `[]` (Enabled: `True`). {{(pid=80245) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 08:31:55.579763 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.fernet_utils [None req-db60e66a-f7e6-41ab-9891-66c313fbf19c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80245) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 08:31:55.611217 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 1122/2251] 10.222.0.47 () {58 vars in 957 bytes} [Thu Jul 21 08:31:55 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 236 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 08:31:55.625614 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36dae6a4-db14-4083-bf2a-a89e896a7db3 None None] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:31:55.627274 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.fernet_utils [None req-36dae6a4-db14-4083-bf2a-a89e896a7db3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80244) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 08:31:55.647024 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG dbcounter [-] [80245] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80245) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:31:55.713261 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: [pid: 80245|app: 0|req: 1130/2252] 10.222.0.47 () {58 vars in 957 bytes} [Thu Jul 21 08:31:55 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 185 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 21 08:31:55.754343 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db60e66a-f7e6-41ab-9891-66c313fbf19c None None] Authenticating user token {{(pid=80245) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:31:55.758442 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.fernet_utils [None req-db60e66a-f7e6-41ab-9891-66c313fbf19c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80245) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 08:31:55.764162 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36dae6a4-db14-4083-bf2a-a89e896a7db3 None None] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:31:55.830504 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db60e66a-f7e6-41ab-9891-66c313fbf19c None None] Validating token access rules against request {{(pid=80245) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:31:55.835073 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-688b510d-df1e-4d4f-b361-0328a15a963e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4f40410b7674687ab5362ac982248ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4faef13aa50e4e36b6c265fb1398fa14', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80245) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:31:55.835853 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-688b510d-df1e-4d4f-b361-0328a15a963e service cinder] REQUEST_METHOD: `GET` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:31:55.836214 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-688b510d-df1e-4d4f-b361-0328a15a963e service cinder] SCRIPT_NAME: `/identity` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:31:55.836566 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-688b510d-df1e-4d4f-b361-0328a15a963e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:31:55.840557 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c370711f-dac1-42e0-b52e-1db9afed3f28 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4f40410b7674687ab5362ac982248ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4faef13aa50e4e36b6c265fb1398fa14', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:31:55.841683 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c370711f-dac1-42e0-b52e-1db9afed3f28 service cinder] REQUEST_METHOD: `GET` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:31:55.842037 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c370711f-dac1-42e0-b52e-1db9afed3f28 service cinder] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:31:55.842386 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c370711f-dac1-42e0-b52e-1db9afed3f28 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:31:55.846778 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-688b510d-df1e-4d4f-b361-0328a15a963e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:31:55.848586 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-688b510d-df1e-4d4f-b361-0328a15a963e service cinder] RBAC: Authorization granted {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:31:55.860583 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: [pid: 80245|app: 0|req: 1131/2253] 10.222.0.47 () {60 vars in 1365 bytes} [Thu Jul 21 08:31:55 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 107 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 08:31:55.899001 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG dbcounter [-] [80244] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80244) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:31:55.931536 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c370711f-dac1-42e0-b52e-1db9afed3f28 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:31:55.933609 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c370711f-dac1-42e0-b52e-1db9afed3f28 service cinder] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:31:55.945903 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 1123/2254] 10.222.0.47 () {60 vars in 1365 bytes} [Thu Jul 21 08:31:55 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 321 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 21 08:31:56.287901 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b997817-0b44-4c72-843e-128b274adc1d None None] REQUEST_METHOD: `GET` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:31:56.288373 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b997817-0b44-4c72-843e-128b274adc1d None None] SCRIPT_NAME: `/identity` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:31:56.288918 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b997817-0b44-4c72-843e-128b274adc1d None None] PATH_INFO: `/` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:31:56.290784 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: [pid: 80245|app: 0|req: 1132/2255] 10.222.0.47 () {54 vars in 921 bytes} [Thu Jul 21 08:31:56 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 08:31:56.336948 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4fdc78a-a5ac-493a-9eab-9b15a06a338c None None] REQUEST_METHOD: `POST` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:31:56.337600 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4fdc78a-a5ac-493a-9eab-9b15a06a338c None None] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:31:56.339750 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4fdc78a-a5ac-493a-9eab-9b15a06a338c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:31:56.375076 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.auth.core [None req-e4fdc78a-a5ac-493a-9eab-9b15a06a338c None None] MFA Rules not processed for user `3379b32543474d0a8e897484e85c7d04`. Rule list: `[]` (Enabled: `True`). {{(pid=80244) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 08:31:56.378066 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.fernet_utils [None req-e4fdc78a-a5ac-493a-9eab-9b15a06a338c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80244) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 08:31:56.428572 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b997817-0b44-4c72-843e-128b274adc1d None None] Authenticating user token {{(pid=80245) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:31:56.430561 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.fernet_utils [None req-1b997817-0b44-4c72-843e-128b274adc1d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80245) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 08:31:56.527716 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 1124/2256] 10.222.0.47 () {58 vars in 1018 bytes} [Thu Jul 21 08:31:56 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 192 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 08:31:56.541962 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4fdc78a-a5ac-493a-9eab-9b15a06a338c None None] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:31:56.549752 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.fernet_utils [None req-e4fdc78a-a5ac-493a-9eab-9b15a06a338c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80244) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 08:31:56.565474 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4fdc78a-a5ac-493a-9eab-9b15a06a338c None None] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:31:56.580103 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b7cc578-63dd-470f-99df-cfecc5a79c89 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3379b32543474d0a8e897484e85c7d04', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4faef13aa50e4e36b6c265fb1398fa14', 'project_domain_id': 'default', 'roles': ['service', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:31:56.580996 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b7cc578-63dd-470f-99df-cfecc5a79c89 service placement] REQUEST_METHOD: `GET` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:31:56.581550 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b7cc578-63dd-470f-99df-cfecc5a79c89 service placement] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:31:56.581942 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b7cc578-63dd-470f-99df-cfecc5a79c89 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:31:56.583668 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b997817-0b44-4c72-843e-128b274adc1d None None] Validating token access rules against request {{(pid=80245) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:31:56.592068 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a5d8b65-0959-42e6-b60d-ab08e4453738 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '42d15fc0515945acacbbffcf2356c7a5', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80245) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:31:56.593186 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a5d8b65-0959-42e6-b60d-ab08e4453738 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:31:56.595456 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b7cc578-63dd-470f-99df-cfecc5a79c89 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:31:56.595747 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b7cc578-63dd-470f-99df-cfecc5a79c89 service placement] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:31:56.600675 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a5d8b65-0959-42e6-b60d-ab08e4453738 admin admin] SCRIPT_NAME: `/identity` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:31:56.600893 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a5d8b65-0959-42e6-b60d-ab08e4453738 admin admin] PATH_INFO: `/v3/users/2465817c3ba340f4870debf937929499` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:31:56.602277 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a5d8b65-0959-42e6-b60d-ab08e4453738 admin admin] RBAC: Authorizing `identity:delete_user(user_id=2465817c3ba340f4870debf937929499)` {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:31:56.604305 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a5d8b65-0959-42e6-b60d-ab08e4453738 admin admin] RBAC: Authorization granted {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:31:56.611558 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 1125/2257] 10.222.0.47 () {60 vars in 1365 bytes} [Thu Jul 21 08:31:56 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 71 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 21 08:31:56.683289 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.notifications [None req-0a5d8b65-0959-42e6-b60d-ab08e4453738 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2465817c3ba340f4870debf937929499'} {{(pid=80245) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 08:31:56.693486 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.notifications [None req-0a5d8b65-0959-42e6-b60d-ab08e4453738 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2465817c3ba340f4870debf937929499'} {{(pid=80245) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 08:31:56.709589 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG dbcounter [-] [80245] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80245) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:31:56.712817 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.notifications [None req-0a5d8b65-0959-42e6-b60d-ab08e4453738 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2465817c3ba340f4870debf937929499'} {{(pid=80245) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 08:31:56.728182 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.notifications [None req-0a5d8b65-0959-42e6-b60d-ab08e4453738 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2465817c3ba340f4870debf937929499'} {{(pid=80245) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 08:31:56.729566 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.notifications [None req-0a5d8b65-0959-42e6-b60d-ab08e4453738 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2465817c3ba340f4870debf937929499'} {{(pid=80245) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 08:31:56.747097 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: [pid: 80245|app: 0|req: 1133/2258] 10.222.0.47 () {58 vars in 1223 bytes} [Thu Jul 21 08:31:56 2022] DELETE /identity/v3/users/2465817c3ba340f4870debf937929499 => generated 0 bytes in 319 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 08:31:56.974456 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b7cc578-63dd-470f-99df-cfecc5a79c89 service placement] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:31:56.992212 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b7cc578-63dd-470f-99df-cfecc5a79c89 service placement] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:31:57.100697 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-940d8869-2f3b-46c8-aa36-14d0997ac254 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '42d15fc0515945acacbbffcf2356c7a5', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:31:57.101932 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-940d8869-2f3b-46c8-aa36-14d0997ac254 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:31:57.102213 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-940d8869-2f3b-46c8-aa36-14d0997ac254 admin admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:31:57.102669 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-940d8869-2f3b-46c8-aa36-14d0997ac254 admin admin] PATH_INFO: `/v3/projects/90d95f75c84d445f9ec31e4a0e8d3953` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:31:57.105388 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-940d8869-2f3b-46c8-aa36-14d0997ac254 admin admin] RBAC: Authorizing `identity:delete_project(project_id=90d95f75c84d445f9ec31e4a0e8d3953)` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:31:57.107237 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-940d8869-2f3b-46c8-aa36-14d0997ac254 admin admin] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:31:57.164077 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.notifications [None req-940d8869-2f3b-46c8-aa36-14d0997ac254 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '90d95f75c84d445f9ec31e4a0e8d3953', 'request_id': 'req-940d8869-2f3b-46c8-aa36-14d0997ac254'} {{(pid=80244) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 08:31:57.172507 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.notifications [None req-940d8869-2f3b-46c8-aa36-14d0997ac254 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '90d95f75c84d445f9ec31e4a0e8d3953', 'request_id': 'req-940d8869-2f3b-46c8-aa36-14d0997ac254'} {{(pid=80244) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 08:31:57.179544 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.notifications [None req-940d8869-2f3b-46c8-aa36-14d0997ac254 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '90d95f75c84d445f9ec31e4a0e8d3953', 'request_id': 'req-940d8869-2f3b-46c8-aa36-14d0997ac254'} {{(pid=80244) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 08:31:57.180805 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG dbcounter [-] [80244] Writing DB stats keystone:SELECT=94,keystone:DELETE=5,keystone:INSERT=1 {{(pid=80244) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:31:57.199446 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.notifications [None req-940d8869-2f3b-46c8-aa36-14d0997ac254 admin admin] The token cache is being invalidate because project 90d95f75c84d445f9ec31e4a0e8d3953 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80244) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 21 08:31:57.199745 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.notifications [None req-940d8869-2f3b-46c8-aa36-14d0997ac254 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80244) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 08:31:57.200448 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.notifications [None req-940d8869-2f3b-46c8-aa36-14d0997ac254 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80244) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 08:31:57.202383 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 1126/2259] 10.222.0.47 () {58 vars in 1232 bytes} [Thu Jul 21 08:31:56 2022] DELETE /identity/v3/projects/90d95f75c84d445f9ec31e4a0e8d3953 => generated 0 bytes in 228 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 08:31:57.207197 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b22228a-dc05-4ebc-9f9e-9f28c3193960 None None] REQUEST_METHOD: `POST` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:31:57.207730 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b22228a-dc05-4ebc-9f9e-9f28c3193960 None None] SCRIPT_NAME: `/identity` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:31:57.207822 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b22228a-dc05-4ebc-9f9e-9f28c3193960 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:31:57.228493 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.auth.core [None req-1b22228a-dc05-4ebc-9f9e-9f28c3193960 None None] MFA Rules not processed for user `2dce40b8d74c41ec94cc36e6ff05d412`. Rule list: `[]` (Enabled: `True`). {{(pid=80245) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 08:31:57.229825 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.fernet_utils [None req-1b22228a-dc05-4ebc-9f9e-9f28c3193960 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80245) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 08:31:57.301338 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: [pid: 80245|app: 0|req: 1134/2260] 10.222.0.47 () {58 vars in 957 bytes} [Thu Jul 21 08:31:57 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 21 08:31:57.310309 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40c2ab33-4eed-40ed-8bdb-8cff5a0bd876 None None] REQUEST_METHOD: `POST` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:31:57.310601 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40c2ab33-4eed-40ed-8bdb-8cff5a0bd876 None None] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:31:57.310875 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40c2ab33-4eed-40ed-8bdb-8cff5a0bd876 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:31:57.331447 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.auth.core [None req-40c2ab33-4eed-40ed-8bdb-8cff5a0bd876 None None] MFA Rules not processed for user `2dce40b8d74c41ec94cc36e6ff05d412`. Rule list: `[]` (Enabled: `True`). {{(pid=80244) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 08:31:57.332653 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.fernet_utils [None req-40c2ab33-4eed-40ed-8bdb-8cff5a0bd876 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80244) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 08:31:57.341330 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 1127/2261] 10.222.0.47 () {58 vars in 957 bytes} [Thu Jul 21 08:31:57 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 21 08:31:57.345523 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b22228a-dc05-4ebc-9f9e-9f28c3193960 None None] Authenticating user token {{(pid=80245) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:31:57.357224 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b22228a-dc05-4ebc-9f9e-9f28c3193960 None None] Validating token access rules against request {{(pid=80245) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:31:57.365054 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f409e80d-8087-4cd4-907c-4852307a9576 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '42d15fc0515945acacbbffcf2356c7a5', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80245) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:31:57.366591 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f409e80d-8087-4cd4-907c-4852307a9576 admin admin] REQUEST_METHOD: `GET` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:31:57.367182 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f409e80d-8087-4cd4-907c-4852307a9576 admin admin] SCRIPT_NAME: `/identity` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:31:57.367756 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f409e80d-8087-4cd4-907c-4852307a9576 admin admin] PATH_INFO: `/v3/domains` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:31:57.368866 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f409e80d-8087-4cd4-907c-4852307a9576 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80245) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 08:31:57.369345 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f409e80d-8087-4cd4-907c-4852307a9576 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:31:57.372454 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f409e80d-8087-4cd4-907c-4852307a9576 admin admin] RBAC: Authorization granted {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:31:57.387476 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: [pid: 80245|app: 0|req: 1135/2262] 10.222.0.47 () {58 vars in 1152 bytes} [Thu Jul 21 08:31:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 08:31:57.391726 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40c2ab33-4eed-40ed-8bdb-8cff5a0bd876 None None] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:31:57.404287 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40c2ab33-4eed-40ed-8bdb-8cff5a0bd876 None None] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:31:57.409867 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c951ded6-1054-402d-a771-20d78252995c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '42d15fc0515945acacbbffcf2356c7a5', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:31:57.410605 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c951ded6-1054-402d-a771-20d78252995c admin admin] REQUEST_METHOD: `POST` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:31:57.411237 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c951ded6-1054-402d-a771-20d78252995c admin admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:31:57.411620 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c951ded6-1054-402d-a771-20d78252995c admin admin] PATH_INFO: `/v3/projects` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:31:57.412470 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c951ded6-1054-402d-a771-20d78252995c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:31:57.414781 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c951ded6-1054-402d-a771-20d78252995c admin admin] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:31:57.457910 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 1128/2263] 10.222.0.47 () {60 vars in 1152 bytes} [Thu Jul 21 08:31:57 2022] POST /identity/v3/projects => generated 385 bytes in 66 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 08:31:57.458343 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f409e80d-8087-4cd4-907c-4852307a9576 admin admin] Authenticating user token {{(pid=80245) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:31:57.469596 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f409e80d-8087-4cd4-907c-4852307a9576 admin admin] Validating token access rules against request {{(pid=80245) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:31:57.567079 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3a7e8a3-8b89-44ee-9794-29f31dc59748 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '42d15fc0515945acacbbffcf2356c7a5', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80245) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:31:57.568039 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3a7e8a3-8b89-44ee-9794-29f31dc59748 admin admin] REQUEST_METHOD: `POST` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:31:57.569223 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3a7e8a3-8b89-44ee-9794-29f31dc59748 admin admin] SCRIPT_NAME: `/identity` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:31:57.569886 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3a7e8a3-8b89-44ee-9794-29f31dc59748 admin admin] PATH_INFO: `/v3/users` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:31:57.575858 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3a7e8a3-8b89-44ee-9794-29f31dc59748 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:31:57.577817 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3a7e8a3-8b89-44ee-9794-29f31dc59748 admin admin] RBAC: Authorization granted {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:31:57.590412 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d85fc175-c858-461a-aec3-26e6a6288ae4 None None] REQUEST_METHOD: `POST` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:31:57.591576 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d85fc175-c858-461a-aec3-26e6a6288ae4 None None] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:31:57.592108 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d85fc175-c858-461a-aec3-26e6a6288ae4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:31:57.609974 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: [pid: 80245|app: 0|req: 1136/2264] 10.222.0.47 () {60 vars in 1143 bytes} [Thu Jul 21 08:31:57 2022] POST /identity/v3/users => generated 293 bytes in 152 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 21 08:31:57.610888 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3a7e8a3-8b89-44ee-9794-29f31dc59748 admin admin] Authenticating user token {{(pid=80245) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:31:57.614234 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.auth.core [None req-d85fc175-c858-461a-aec3-26e6a6288ae4 None None] MFA Rules not processed for user `aa156584880d4449bd89a7bcdcfe403f`. Rule list: `[]` (Enabled: `True`). {{(pid=80244) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 08:31:57.615702 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.fernet_utils [None req-d85fc175-c858-461a-aec3-26e6a6288ae4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80244) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 08:31:57.625150 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3a7e8a3-8b89-44ee-9794-29f31dc59748 admin admin] Validating token access rules against request {{(pid=80245) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:31:57.630238 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b333ebf4-5e06-4a12-86dd-3c2b93efe112 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '42d15fc0515945acacbbffcf2356c7a5', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80245) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:31:57.630914 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b333ebf4-5e06-4a12-86dd-3c2b93efe112 admin admin] REQUEST_METHOD: `GET` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:31:57.631198 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b333ebf4-5e06-4a12-86dd-3c2b93efe112 admin admin] SCRIPT_NAME: `/identity` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:31:57.631510 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b333ebf4-5e06-4a12-86dd-3c2b93efe112 admin admin] PATH_INFO: `/v3/roles` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:31:57.632238 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b333ebf4-5e06-4a12-86dd-3c2b93efe112 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:31:57.634037 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b333ebf4-5e06-4a12-86dd-3c2b93efe112 admin admin] RBAC: Authorization granted {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:31:57.651582 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: [pid: 80245|app: 0|req: 1137/2265] 10.222.0.47 () {58 vars in 1121 bytes} [Thu Jul 21 08:31:57 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 08:31:57.653272 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b333ebf4-5e06-4a12-86dd-3c2b93efe112 admin admin] Authenticating user token {{(pid=80245) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:31:57.670533 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b333ebf4-5e06-4a12-86dd-3c2b93efe112 admin admin] Validating token access rules against request {{(pid=80245) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:31:57.678529 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bd38125-26b7-4db7-b8b3-8378d86068b0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '42d15fc0515945acacbbffcf2356c7a5', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80245) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:31:57.679466 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bd38125-26b7-4db7-b8b3-8378d86068b0 admin admin] REQUEST_METHOD: `GET` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:31:57.679859 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bd38125-26b7-4db7-b8b3-8378d86068b0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:31:57.680392 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bd38125-26b7-4db7-b8b3-8378d86068b0 admin admin] PATH_INFO: `/v3/roles` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:31:57.681394 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bd38125-26b7-4db7-b8b3-8378d86068b0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:31:57.684289 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bd38125-26b7-4db7-b8b3-8378d86068b0 admin admin] RBAC: Authorization granted {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:31:57.695763 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 1129/2266] 10.222.0.47 () {58 vars in 957 bytes} [Thu Jul 21 08:31:57 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 21 08:31:57.697828 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: [pid: 80245|app: 0|req: 1138/2267] 10.222.0.47 () {58 vars in 1121 bytes} [Thu Jul 21 08:31:57 2022] GET /identity/v3/roles => generated 1420 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 08:31:57.704884 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d85fc175-c858-461a-aec3-26e6a6288ae4 None None] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:31:57.716363 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d85fc175-c858-461a-aec3-26e6a6288ae4 None None] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:31:57.722077 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa4572e6-1951-4d2c-a324-628d59cb214e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '42d15fc0515945acacbbffcf2356c7a5', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:31:57.723171 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa4572e6-1951-4d2c-a324-628d59cb214e admin admin] REQUEST_METHOD: `PUT` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:31:57.723550 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa4572e6-1951-4d2c-a324-628d59cb214e admin admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:31:57.723899 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa4572e6-1951-4d2c-a324-628d59cb214e admin admin] PATH_INFO: `/v3/projects/93539ede044a41b696986aaf5b86da65/users/253aa2f7704e48b59ce4cf39d6ab50ca/roles/3a074738221e4601b5205715ba87d240` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:31:57.743957 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa4572e6-1951-4d2c-a324-628d59cb214e admin admin] RBAC: Authorizing `identity:create_grant(project_id=93539ede044a41b696986aaf5b86da65, user_id=253aa2f7704e48b59ce4cf39d6ab50ca, role_id=3a074738221e4601b5205715ba87d240)` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:31:57.748108 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa4572e6-1951-4d2c-a324-628d59cb214e admin admin] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:31:57.775427 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 1130/2268] 10.222.0.47 () {60 vars in 1482 bytes} [Thu Jul 21 08:31:57 2022] PUT /identity/v3/projects/93539ede044a41b696986aaf5b86da65/users/253aa2f7704e48b59ce4cf39d6ab50ca/roles/3a074738221e4601b5205715ba87d240 => generated 0 bytes in 68 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 08:31:57.783096 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3541ed09-e2cf-49a1-85bb-eb2ce4b7af42 None None] REQUEST_METHOD: `POST` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:31:57.783500 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3541ed09-e2cf-49a1-85bb-eb2ce4b7af42 None None] SCRIPT_NAME: `/identity` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:31:57.783764 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3541ed09-e2cf-49a1-85bb-eb2ce4b7af42 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:31:57.814627 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.auth.core [None req-3541ed09-e2cf-49a1-85bb-eb2ce4b7af42 None None] MFA Rules not processed for user `253aa2f7704e48b59ce4cf39d6ab50ca`. Rule list: `[]` (Enabled: `True`). {{(pid=80245) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 08:31:57.815820 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.fernet_utils [None req-3541ed09-e2cf-49a1-85bb-eb2ce4b7af42 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80245) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 08:31:57.846517 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG dbcounter [-] [80245] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80245) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:31:57.891268 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: [pid: 80245|app: 0|req: 1139/2269] 10.222.0.47 () {58 vars in 957 bytes} [Thu Jul 21 08:31:57 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 21 08:31:57.902953 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa4572e6-1951-4d2c-a324-628d59cb214e admin admin] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:31:57.905044 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.fernet_utils [None req-fa4572e6-1951-4d2c-a324-628d59cb214e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80244) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 08:31:57.984086 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa4572e6-1951-4d2c-a324-628d59cb214e admin admin] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:31:57.991765 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bb0b5e9-af95-4ce3-8c00-34d4b2184bd6 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2b5b99a36442148fe743269054e991', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4faef13aa50e4e36b6c265fb1398fa14', 'project_domain_id': 'default', 'roles': ['member', 'service', 'ResellerAdmin', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:31:57.992465 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bb0b5e9-af95-4ce3-8c00-34d4b2184bd6 service nova] REQUEST_METHOD: `GET` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:31:57.992754 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bb0b5e9-af95-4ce3-8c00-34d4b2184bd6 service nova] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:31:57.993036 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bb0b5e9-af95-4ce3-8c00-34d4b2184bd6 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:31:58.004978 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bb0b5e9-af95-4ce3-8c00-34d4b2184bd6 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:31:58.006596 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bb0b5e9-af95-4ce3-8c00-34d4b2184bd6 service nova] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:31:58.016684 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 1131/2270] 10.222.0.47 () {60 vars in 1365 bytes} [Thu Jul 21 08:31:57 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 114 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 21 08:31:58.118441 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3541ed09-e2cf-49a1-85bb-eb2ce4b7af42 None None] Authenticating user token {{(pid=80245) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:31:58.120028 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.fernet_utils [None req-3541ed09-e2cf-49a1-85bb-eb2ce4b7af42 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80245) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 08:31:58.203602 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3541ed09-e2cf-49a1-85bb-eb2ce4b7af42 None None] Validating token access rules against request {{(pid=80245) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:31:58.208996 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-392278d5-b6aa-41eb-8798-a329be08252d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '42d15fc0515945acacbbffcf2356c7a5', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80245) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:31:58.209908 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-392278d5-b6aa-41eb-8798-a329be08252d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:31:58.210189 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-392278d5-b6aa-41eb-8798-a329be08252d admin admin] SCRIPT_NAME: `/identity` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:31:58.210466 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-392278d5-b6aa-41eb-8798-a329be08252d admin admin] PATH_INFO: `/v3/users/5d5fe2099e0845eabc2eb9ce089d03cd` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:31:58.211941 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-392278d5-b6aa-41eb-8798-a329be08252d admin admin] RBAC: Authorizing `identity:delete_user(user_id=5d5fe2099e0845eabc2eb9ce089d03cd)` {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:31:58.214698 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-392278d5-b6aa-41eb-8798-a329be08252d admin admin] RBAC: Authorization granted {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:31:58.280690 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.notifications [None req-392278d5-b6aa-41eb-8798-a329be08252d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5d5fe2099e0845eabc2eb9ce089d03cd'} {{(pid=80245) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 08:31:58.295672 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.notifications [None req-392278d5-b6aa-41eb-8798-a329be08252d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5d5fe2099e0845eabc2eb9ce089d03cd'} {{(pid=80245) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 08:31:58.323109 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.notifications [None req-392278d5-b6aa-41eb-8798-a329be08252d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5d5fe2099e0845eabc2eb9ce089d03cd'} {{(pid=80245) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 08:31:58.341001 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.notifications [None req-392278d5-b6aa-41eb-8798-a329be08252d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5d5fe2099e0845eabc2eb9ce089d03cd'} {{(pid=80245) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 08:31:58.342735 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.notifications [None req-392278d5-b6aa-41eb-8798-a329be08252d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5d5fe2099e0845eabc2eb9ce089d03cd'} {{(pid=80245) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 08:31:58.370496 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: [pid: 80245|app: 0|req: 1140/2271] 10.222.0.47 () {58 vars in 1223 bytes} [Thu Jul 21 08:31:58 2022] DELETE /identity/v3/users/5d5fe2099e0845eabc2eb9ce089d03cd => generated 0 bytes in 253 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 08:31:58.375233 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-872d5262-a129-4605-9a3a-0ccd2e4de372 None None] REQUEST_METHOD: `POST` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:31:58.375558 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-872d5262-a129-4605-9a3a-0ccd2e4de372 None None] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:31:58.375902 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-872d5262-a129-4605-9a3a-0ccd2e4de372 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:31:58.402721 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.auth.core [None req-872d5262-a129-4605-9a3a-0ccd2e4de372 None None] MFA Rules not processed for user `2dce40b8d74c41ec94cc36e6ff05d412`. Rule list: `[]` (Enabled: `True`). {{(pid=80244) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 08:31:58.404228 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.fernet_utils [None req-872d5262-a129-4605-9a3a-0ccd2e4de372 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80244) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 08:31:58.439452 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG dbcounter [-] [80244] Writing DB stats keystone:DELETE=2,keystone:SELECT=96,keystone:INSERT=2 {{(pid=80244) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:31:58.519030 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 1132/2272] 10.222.0.47 () {58 vars in 957 bytes} [Thu Jul 21 08:31:58 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 145 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 08:31:58.529179 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-392278d5-b6aa-41eb-8798-a329be08252d admin admin] Authenticating user token {{(pid=80245) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:31:58.531059 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.fernet_utils [None req-392278d5-b6aa-41eb-8798-a329be08252d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80245) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 08:31:58.598595 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-392278d5-b6aa-41eb-8798-a329be08252d admin admin] Validating token access rules against request {{(pid=80245) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:31:58.602362 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a80bf728-3d2d-4ea8-8b73-109576efcb25 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20ca3c3bd3774691b3374cfbfec6e494', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4faef13aa50e4e36b6c265fb1398fa14', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80245) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:31:58.603101 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a80bf728-3d2d-4ea8-8b73-109576efcb25 service neutron] REQUEST_METHOD: `GET` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:31:58.603490 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a80bf728-3d2d-4ea8-8b73-109576efcb25 service neutron] SCRIPT_NAME: `/identity` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:31:58.603914 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a80bf728-3d2d-4ea8-8b73-109576efcb25 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:31:58.610234 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG dbcounter [-] [80245] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80245) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:31:58.614633 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a80bf728-3d2d-4ea8-8b73-109576efcb25 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:31:58.616648 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a80bf728-3d2d-4ea8-8b73-109576efcb25 service neutron] RBAC: Authorization granted {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:31:58.631313 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: [pid: 80245|app: 0|req: 1141/2273] 10.222.0.47 () {60 vars in 1365 bytes} [Thu Jul 21 08:31:58 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 103 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 08:31:58.888085 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-872d5262-a129-4605-9a3a-0ccd2e4de372 None None] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:31:58.889601 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.fernet_utils [None req-872d5262-a129-4605-9a3a-0ccd2e4de372 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80244) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 08:31:58.901586 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-872d5262-a129-4605-9a3a-0ccd2e4de372 None None] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:31:58.906644 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e950294-9ffe-4d2f-a660-f2ba8b244f90 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '42d15fc0515945acacbbffcf2356c7a5', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:31:58.907556 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e950294-9ffe-4d2f-a660-f2ba8b244f90 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:31:58.907962 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e950294-9ffe-4d2f-a660-f2ba8b244f90 admin admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:31:58.908244 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e950294-9ffe-4d2f-a660-f2ba8b244f90 admin admin] PATH_INFO: `/v3/users/acfab497869f427ab16f26bdec11f7ed` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:31:58.909441 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e950294-9ffe-4d2f-a660-f2ba8b244f90 admin admin] RBAC: Authorizing `identity:delete_user(user_id=acfab497869f427ab16f26bdec11f7ed)` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:31:58.911508 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e950294-9ffe-4d2f-a660-f2ba8b244f90 admin admin] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:31:58.962758 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.notifications [None req-0e950294-9ffe-4d2f-a660-f2ba8b244f90 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'acfab497869f427ab16f26bdec11f7ed'} {{(pid=80244) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 08:31:58.971767 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.notifications [None req-0e950294-9ffe-4d2f-a660-f2ba8b244f90 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'acfab497869f427ab16f26bdec11f7ed'} {{(pid=80244) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 08:31:58.986510 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.notifications [None req-0e950294-9ffe-4d2f-a660-f2ba8b244f90 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'acfab497869f427ab16f26bdec11f7ed'} {{(pid=80244) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 08:31:58.987259 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.notifications [None req-0e950294-9ffe-4d2f-a660-f2ba8b244f90 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'acfab497869f427ab16f26bdec11f7ed'} {{(pid=80244) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 08:31:58.995754 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.notifications [None req-0e950294-9ffe-4d2f-a660-f2ba8b244f90 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'acfab497869f427ab16f26bdec11f7ed'} {{(pid=80244) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 08:31:59.000771 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a80bf728-3d2d-4ea8-8b73-109576efcb25 service neutron] Authenticating user token {{(pid=80245) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:31:59.002457 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.fernet_utils [None req-a80bf728-3d2d-4ea8-8b73-109576efcb25 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80245) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 08:31:59.003110 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 1133/2274] 10.222.0.47 () {58 vars in 1223 bytes} [Thu Jul 21 08:31:58 2022] DELETE /identity/v3/users/acfab497869f427ab16f26bdec11f7ed => generated 0 bytes in 116 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 08:31:59.008123 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4159425-f836-46b2-82ba-420476ea579b None None] REQUEST_METHOD: `POST` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:31:59.008619 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4159425-f836-46b2-82ba-420476ea579b None None] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:31:59.008904 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4159425-f836-46b2-82ba-420476ea579b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:31:59.017033 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a80bf728-3d2d-4ea8-8b73-109576efcb25 service neutron] Validating token access rules against request {{(pid=80245) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:31:59.026988 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.auth.core [None req-c4159425-f836-46b2-82ba-420476ea579b None None] MFA Rules not processed for user `2dce40b8d74c41ec94cc36e6ff05d412`. Rule list: `[]` (Enabled: `True`). {{(pid=80244) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 08:31:59.028257 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.fernet_utils [None req-c4159425-f836-46b2-82ba-420476ea579b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80244) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 08:31:59.098072 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 1134/2275] 10.222.0.47 () {58 vars in 957 bytes} [Thu Jul 21 08:31:59 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 08:31:59.099821 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dd2dc93-e903-4969-86b3-f1f4040f313f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '42d15fc0515945acacbbffcf2356c7a5', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80245) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:31:59.100776 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dd2dc93-e903-4969-86b3-f1f4040f313f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:31:59.101068 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dd2dc93-e903-4969-86b3-f1f4040f313f admin admin] SCRIPT_NAME: `/identity` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:31:59.101348 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dd2dc93-e903-4969-86b3-f1f4040f313f admin admin] PATH_INFO: `/v3/projects/65d666f18695484aab416d8b0d902efb` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:31:59.102549 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dd2dc93-e903-4969-86b3-f1f4040f313f admin admin] RBAC: Authorizing `identity:delete_project(project_id=65d666f18695484aab416d8b0d902efb)` {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:31:59.105068 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dd2dc93-e903-4969-86b3-f1f4040f313f admin admin] RBAC: Authorization granted {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:31:59.108935 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4159425-f836-46b2-82ba-420476ea579b None None] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:31:59.110339 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.fernet_utils [None req-c4159425-f836-46b2-82ba-420476ea579b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80244) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 08:31:59.154634 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.notifications [None req-4dd2dc93-e903-4969-86b3-f1f4040f313f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '65d666f18695484aab416d8b0d902efb', 'request_id': 'req-4dd2dc93-e903-4969-86b3-f1f4040f313f'} {{(pid=80245) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 08:31:59.157516 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG dbcounter [-] [80244] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80244) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:31:59.161645 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.notifications [None req-4dd2dc93-e903-4969-86b3-f1f4040f313f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '65d666f18695484aab416d8b0d902efb', 'request_id': 'req-4dd2dc93-e903-4969-86b3-f1f4040f313f'} {{(pid=80245) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 08:31:59.168524 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.notifications [None req-4dd2dc93-e903-4969-86b3-f1f4040f313f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '65d666f18695484aab416d8b0d902efb', 'request_id': 'req-4dd2dc93-e903-4969-86b3-f1f4040f313f'} {{(pid=80245) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 08:31:59.178339 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4159425-f836-46b2-82ba-420476ea579b None None] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:31:59.185366 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.notifications [None req-4dd2dc93-e903-4969-86b3-f1f4040f313f admin admin] The token cache is being invalidate because project 65d666f18695484aab416d8b0d902efb was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80245) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 21 08:31:59.185740 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.notifications [None req-4dd2dc93-e903-4969-86b3-f1f4040f313f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80245) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 08:31:59.186289 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.notifications [None req-4dd2dc93-e903-4969-86b3-f1f4040f313f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80245) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 08:31:59.187903 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: [pid: 80245|app: 0|req: 1142/2276] 10.222.0.47 () {58 vars in 1232 bytes} [Thu Jul 21 08:31:58 2022] DELETE /identity/v3/projects/65d666f18695484aab416d8b0d902efb => generated 0 bytes in 188 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 08:31:59.193734 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3583b52d-12b8-4317-8641-8f1a9d31c7ea None None] REQUEST_METHOD: `POST` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:31:59.194023 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3583b52d-12b8-4317-8641-8f1a9d31c7ea None None] SCRIPT_NAME: `/identity` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:31:59.194348 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3583b52d-12b8-4317-8641-8f1a9d31c7ea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:31:59.213254 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.auth.core [None req-3583b52d-12b8-4317-8641-8f1a9d31c7ea None None] MFA Rules not processed for user `2dce40b8d74c41ec94cc36e6ff05d412`. Rule list: `[]` (Enabled: `True`). {{(pid=80245) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 08:31:59.214404 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.fernet_utils [None req-3583b52d-12b8-4317-8641-8f1a9d31c7ea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80245) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 08:31:59.234148 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbe7966f-7ae1-4c8e-b3f1-ef04666da529 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20ca3c3bd3774691b3374cfbfec6e494', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4faef13aa50e4e36b6c265fb1398fa14', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:31:59.234836 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbe7966f-7ae1-4c8e-b3f1-ef04666da529 service neutron] REQUEST_METHOD: `GET` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:31:59.235116 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbe7966f-7ae1-4c8e-b3f1-ef04666da529 service neutron] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:31:59.235423 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbe7966f-7ae1-4c8e-b3f1-ef04666da529 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:31:59.237278 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.fernet_utils [None req-bbe7966f-7ae1-4c8e-b3f1-ef04666da529 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80244) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 08:31:59.286818 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: [pid: 80245|app: 0|req: 1143/2277] 10.222.0.47 () {58 vars in 957 bytes} [Thu Jul 21 08:31:59 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 21 08:31:59.296224 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03ddbb39-4d5b-4519-87b4-1811caab24ab None None] REQUEST_METHOD: `POST` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:31:59.296516 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03ddbb39-4d5b-4519-87b4-1811caab24ab None None] SCRIPT_NAME: `/identity` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:31:59.296801 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03ddbb39-4d5b-4519-87b4-1811caab24ab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:31:59.309576 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbe7966f-7ae1-4c8e-b3f1-ef04666da529 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:31:59.311216 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbe7966f-7ae1-4c8e-b3f1-ef04666da529 service neutron] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:31:59.314720 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.auth.core [None req-03ddbb39-4d5b-4519-87b4-1811caab24ab None None] MFA Rules not processed for user `2dce40b8d74c41ec94cc36e6ff05d412`. Rule list: `[]` (Enabled: `True`). {{(pid=80245) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 08:31:59.315976 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.fernet_utils [None req-03ddbb39-4d5b-4519-87b4-1811caab24ab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80245) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 08:31:59.321091 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 1135/2278] 10.222.0.47 () {60 vars in 1365 bytes} [Thu Jul 21 08:31:59 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 213 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 21 08:31:59.326501 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: [pid: 80245|app: 0|req: 1144/2279] 10.222.0.47 () {58 vars in 957 bytes} [Thu Jul 21 08:31:59 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 21 08:31:59.330539 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbe7966f-7ae1-4c8e-b3f1-ef04666da529 service neutron] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:31:59.340442 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbe7966f-7ae1-4c8e-b3f1-ef04666da529 service neutron] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:31:59.345049 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f635f7bb-bd01-4fb8-9496-741dad11fd56 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '42d15fc0515945acacbbffcf2356c7a5', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:31:59.345698 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f635f7bb-bd01-4fb8-9496-741dad11fd56 admin admin] REQUEST_METHOD: `GET` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:31:59.345980 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f635f7bb-bd01-4fb8-9496-741dad11fd56 admin admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:31:59.346255 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f635f7bb-bd01-4fb8-9496-741dad11fd56 admin admin] PATH_INFO: `/v3/domains` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:31:59.346972 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f635f7bb-bd01-4fb8-9496-741dad11fd56 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80244) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 08:31:59.347272 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f635f7bb-bd01-4fb8-9496-741dad11fd56 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:31:59.348941 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f635f7bb-bd01-4fb8-9496-741dad11fd56 admin admin] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:31:59.362651 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 1136/2280] 10.222.0.47 () {58 vars in 1152 bytes} [Thu Jul 21 08:31:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 08:31:59.367369 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03ddbb39-4d5b-4519-87b4-1811caab24ab None None] Authenticating user token {{(pid=80245) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:31:59.379679 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03ddbb39-4d5b-4519-87b4-1811caab24ab None None] Validating token access rules against request {{(pid=80245) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:31:59.384861 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a54f37c-7387-440c-b818-e46c4ad70707 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '42d15fc0515945acacbbffcf2356c7a5', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80245) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:31:59.385552 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a54f37c-7387-440c-b818-e46c4ad70707 admin admin] REQUEST_METHOD: `POST` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:31:59.385833 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a54f37c-7387-440c-b818-e46c4ad70707 admin admin] SCRIPT_NAME: `/identity` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:31:59.386116 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a54f37c-7387-440c-b818-e46c4ad70707 admin admin] PATH_INFO: `/v3/projects` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:31:59.386813 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a54f37c-7387-440c-b818-e46c4ad70707 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:31:59.388942 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a54f37c-7387-440c-b818-e46c4ad70707 admin admin] RBAC: Authorization granted {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:31:59.408043 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG dbcounter [-] [80245] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80245) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:31:59.425632 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: [pid: 80245|app: 0|req: 1145/2281] 10.222.0.47 () {60 vars in 1152 bytes} [Thu Jul 21 08:31:59 2022] POST /identity/v3/projects => generated 367 bytes in 59 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 21 08:31:59.430521 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f635f7bb-bd01-4fb8-9496-741dad11fd56 admin admin] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:31:59.441869 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f635f7bb-bd01-4fb8-9496-741dad11fd56 admin admin] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:31:59.537159 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-719a99a3-5ebc-493a-830c-c01ab64b96c9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '42d15fc0515945acacbbffcf2356c7a5', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:31:59.537847 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-719a99a3-5ebc-493a-830c-c01ab64b96c9 admin admin] REQUEST_METHOD: `POST` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:31:59.538131 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-719a99a3-5ebc-493a-830c-c01ab64b96c9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:31:59.538409 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-719a99a3-5ebc-493a-830c-c01ab64b96c9 admin admin] PATH_INFO: `/v3/users` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:31:59.539110 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-719a99a3-5ebc-493a-830c-c01ab64b96c9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:31:59.541049 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-719a99a3-5ebc-493a-830c-c01ab64b96c9 admin admin] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:31:59.578009 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 1137/2282] 10.222.0.47 () {60 vars in 1143 bytes} [Thu Jul 21 08:31:59 2022] POST /identity/v3/users => generated 284 bytes in 148 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 08:31:59.582330 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a54f37c-7387-440c-b818-e46c4ad70707 admin admin] Authenticating user token {{(pid=80245) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:31:59.593653 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a54f37c-7387-440c-b818-e46c4ad70707 admin admin] Validating token access rules against request {{(pid=80245) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:31:59.598180 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5d13f56-9ba6-4f13-9cad-e51a5c6490c0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '42d15fc0515945acacbbffcf2356c7a5', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80245) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:31:59.598842 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5d13f56-9ba6-4f13-9cad-e51a5c6490c0 admin admin] REQUEST_METHOD: `GET` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:31:59.599176 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5d13f56-9ba6-4f13-9cad-e51a5c6490c0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:31:59.599459 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5d13f56-9ba6-4f13-9cad-e51a5c6490c0 admin admin] PATH_INFO: `/v3/roles` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:31:59.600326 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5d13f56-9ba6-4f13-9cad-e51a5c6490c0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:31:59.601932 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5d13f56-9ba6-4f13-9cad-e51a5c6490c0 admin admin] RBAC: Authorization granted {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:31:59.613380 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: [pid: 80245|app: 0|req: 1146/2283] 10.222.0.47 () {58 vars in 1121 bytes} [Thu Jul 21 08:31:59 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 08:31:59.617382 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-719a99a3-5ebc-493a-830c-c01ab64b96c9 admin admin] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:31:59.627770 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-719a99a3-5ebc-493a-830c-c01ab64b96c9 admin admin] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:31:59.633543 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-579d7910-f776-4828-90b4-e65db1efe652 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '42d15fc0515945acacbbffcf2356c7a5', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:31:59.634296 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-579d7910-f776-4828-90b4-e65db1efe652 admin admin] REQUEST_METHOD: `GET` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:31:59.634592 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-579d7910-f776-4828-90b4-e65db1efe652 admin admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:31:59.634894 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-579d7910-f776-4828-90b4-e65db1efe652 admin admin] PATH_INFO: `/v3/roles` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:31:59.636220 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-579d7910-f776-4828-90b4-e65db1efe652 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:31:59.637821 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-579d7910-f776-4828-90b4-e65db1efe652 admin admin] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:31:59.649001 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 1138/2284] 10.222.0.47 () {58 vars in 1121 bytes} [Thu Jul 21 08:31:59 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 21 08:31:59.650212 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5d13f56-9ba6-4f13-9cad-e51a5c6490c0 admin admin] Authenticating user token {{(pid=80245) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:31:59.653762 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-579d7910-f776-4828-90b4-e65db1efe652 admin admin] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:31:59.654026 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.fernet_utils [None req-a5d13f56-9ba6-4f13-9cad-e51a5c6490c0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80245) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 08:31:59.666633 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-579d7910-f776-4828-90b4-e65db1efe652 admin admin] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:31:59.667974 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5d13f56-9ba6-4f13-9cad-e51a5c6490c0 admin admin] Validating token access rules against request {{(pid=80245) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:31:59.672518 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-974df4c3-7d8d-4bd4-8d91-08c90b4f7134 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '42d15fc0515945acacbbffcf2356c7a5', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:31:59.673486 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-974df4c3-7d8d-4bd4-8d91-08c90b4f7134 admin admin] REQUEST_METHOD: `PUT` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:31:59.673681 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a43ea6fa-5ea4-4cbd-9ace-1245703f75d5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '42d15fc0515945acacbbffcf2356c7a5', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80245) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:31:59.673913 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-974df4c3-7d8d-4bd4-8d91-08c90b4f7134 admin admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:31:59.674232 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-974df4c3-7d8d-4bd4-8d91-08c90b4f7134 admin admin] PATH_INFO: `/v3/projects/afb8589160b94b4896e10d9096177e85/users/baf6101307e54e6a9149f24db612f437/roles/3a074738221e4601b5205715ba87d240` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:31:59.674733 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a43ea6fa-5ea4-4cbd-9ace-1245703f75d5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:31:59.674986 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a43ea6fa-5ea4-4cbd-9ace-1245703f75d5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:31:59.675262 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a43ea6fa-5ea4-4cbd-9ace-1245703f75d5 admin admin] PATH_INFO: `/v3/projects/bf11a7e194124ec8b57bfd2138d09a4f` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:31:59.677547 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a43ea6fa-5ea4-4cbd-9ace-1245703f75d5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=bf11a7e194124ec8b57bfd2138d09a4f)` {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:31:59.679787 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a43ea6fa-5ea4-4cbd-9ace-1245703f75d5 admin admin] RBAC: Authorization granted {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:31:59.684719 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG dbcounter [-] [80244] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80244) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:31:59.693283 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-974df4c3-7d8d-4bd4-8d91-08c90b4f7134 admin admin] RBAC: Authorizing `identity:create_grant(project_id=afb8589160b94b4896e10d9096177e85, user_id=baf6101307e54e6a9149f24db612f437, role_id=3a074738221e4601b5205715ba87d240)` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:31:59.696013 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-974df4c3-7d8d-4bd4-8d91-08c90b4f7134 admin admin] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:31:59.711177 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 1139/2285] 10.222.0.47 () {60 vars in 1482 bytes} [Thu Jul 21 08:31:59 2022] PUT /identity/v3/projects/afb8589160b94b4896e10d9096177e85/users/baf6101307e54e6a9149f24db612f437/roles/3a074738221e4601b5205715ba87d240 => generated 0 bytes in 58 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 21 08:31:59.722104 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6a59b6c-bcd6-43e4-8e69-4bcb645295f9 None None] REQUEST_METHOD: `POST` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:31:59.722289 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6a59b6c-bcd6-43e4-8e69-4bcb645295f9 None None] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:31:59.722560 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6a59b6c-bcd6-43e4-8e69-4bcb645295f9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:31:59.730948 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.notifications [None req-a43ea6fa-5ea4-4cbd-9ace-1245703f75d5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'bf11a7e194124ec8b57bfd2138d09a4f', 'request_id': 'req-a43ea6fa-5ea4-4cbd-9ace-1245703f75d5'} {{(pid=80245) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 08:31:59.737287 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.notifications [None req-a43ea6fa-5ea4-4cbd-9ace-1245703f75d5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'bf11a7e194124ec8b57bfd2138d09a4f', 'request_id': 'req-a43ea6fa-5ea4-4cbd-9ace-1245703f75d5'} {{(pid=80245) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 08:31:59.743139 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.notifications [None req-a43ea6fa-5ea4-4cbd-9ace-1245703f75d5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'bf11a7e194124ec8b57bfd2138d09a4f', 'request_id': 'req-a43ea6fa-5ea4-4cbd-9ace-1245703f75d5'} {{(pid=80245) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 08:31:59.749397 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.auth.core [None req-f6a59b6c-bcd6-43e4-8e69-4bcb645295f9 None None] MFA Rules not processed for user `baf6101307e54e6a9149f24db612f437`. Rule list: `[]` (Enabled: `True`). {{(pid=80244) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 08:31:59.750687 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.fernet_utils [None req-f6a59b6c-bcd6-43e4-8e69-4bcb645295f9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80244) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 08:31:59.758209 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.notifications [None req-a43ea6fa-5ea4-4cbd-9ace-1245703f75d5 admin admin] The token cache is being invalidate because project bf11a7e194124ec8b57bfd2138d09a4f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80245) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 21 08:31:59.758629 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.notifications [None req-a43ea6fa-5ea4-4cbd-9ace-1245703f75d5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80245) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 08:31:59.759282 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.notifications [None req-a43ea6fa-5ea4-4cbd-9ace-1245703f75d5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80245) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 08:31:59.761437 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: [pid: 80245|app: 0|req: 1147/2286] 10.222.0.47 () {58 vars in 1232 bytes} [Thu Jul 21 08:31:59 2022] DELETE /identity/v3/projects/bf11a7e194124ec8b57bfd2138d09a4f => generated 0 bytes in 112 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 08:31:59.765238 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a43ea6fa-5ea4-4cbd-9ace-1245703f75d5 admin admin] Authenticating user token {{(pid=80245) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:31:59.766537 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.fernet_utils [None req-a43ea6fa-5ea4-4cbd-9ace-1245703f75d5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80245) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 08:31:59.816891 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 1140/2287] 10.222.0.47 () {58 vars in 957 bytes} [Thu Jul 21 08:31:59 2022] POST /identity/v3/auth/tokens => generated 3234 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 08:31:59.828697 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6a59b6c-bcd6-43e4-8e69-4bcb645295f9 None None] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:31:59.830207 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.fernet_utils [None req-f6a59b6c-bcd6-43e4-8e69-4bcb645295f9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80244) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 08:31:59.834947 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a43ea6fa-5ea4-4cbd-9ace-1245703f75d5 admin admin] Validating token access rules against request {{(pid=80245) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:31:59.840664 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fece3b3d-043b-4301-80b5-0bbbc1576ac0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '42d15fc0515945acacbbffcf2356c7a5', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80245) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:31:59.841499 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fece3b3d-043b-4301-80b5-0bbbc1576ac0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:31:59.841783 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fece3b3d-043b-4301-80b5-0bbbc1576ac0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:31:59.842397 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fece3b3d-043b-4301-80b5-0bbbc1576ac0 admin admin] PATH_INFO: `/v3/users/fadfd7afe48545499050bda5b323005e` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:31:59.843457 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fece3b3d-043b-4301-80b5-0bbbc1576ac0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=fadfd7afe48545499050bda5b323005e)` {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:31:59.845439 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fece3b3d-043b-4301-80b5-0bbbc1576ac0 admin admin] RBAC: Authorization granted {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:31:59.896668 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6a59b6c-bcd6-43e4-8e69-4bcb645295f9 None None] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:31:59.900317 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da2f38ea-8823-49a4-a014-7a5970f29c0b service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9b0524aed8c74abd8949b860d6b93936', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4faef13aa50e4e36b6c265fb1398fa14', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:31:59.901655 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da2f38ea-8823-49a4-a014-7a5970f29c0b service glance] REQUEST_METHOD: `GET` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:31:59.901972 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da2f38ea-8823-49a4-a014-7a5970f29c0b service glance] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:31:59.902224 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da2f38ea-8823-49a4-a014-7a5970f29c0b service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:31:59.903470 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.notifications [None req-fece3b3d-043b-4301-80b5-0bbbc1576ac0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'fadfd7afe48545499050bda5b323005e'} {{(pid=80245) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 08:31:59.911893 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da2f38ea-8823-49a4-a014-7a5970f29c0b service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:31:59.912483 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.notifications [None req-fece3b3d-043b-4301-80b5-0bbbc1576ac0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'fadfd7afe48545499050bda5b323005e'} {{(pid=80245) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 08:31:59.913774 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da2f38ea-8823-49a4-a014-7a5970f29c0b service glance] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:31:59.924071 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 1141/2288] 10.222.0.47 () {60 vars in 1365 bytes} [Thu Jul 21 08:31:59 2022] GET /identity/v3/auth/tokens => generated 3234 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 21 08:31:59.927412 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG dbcounter [-] [80245] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=80245) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:31:59.928766 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.notifications [None req-fece3b3d-043b-4301-80b5-0bbbc1576ac0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'fadfd7afe48545499050bda5b323005e'} {{(pid=80245) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 08:31:59.938679 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.notifications [None req-fece3b3d-043b-4301-80b5-0bbbc1576ac0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'fadfd7afe48545499050bda5b323005e'} {{(pid=80245) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 08:31:59.939481 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.notifications [None req-fece3b3d-043b-4301-80b5-0bbbc1576ac0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'fadfd7afe48545499050bda5b323005e'} {{(pid=80245) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 08:31:59.947383 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: [pid: 80245|app: 0|req: 1148/2289] 10.222.0.47 () {58 vars in 1223 bytes} [Thu Jul 21 08:31:59 2022] DELETE /identity/v3/users/fadfd7afe48545499050bda5b323005e => generated 0 bytes in 181 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 21 08:32:00.256000 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da2f38ea-8823-49a4-a014-7a5970f29c0b service glance] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:32:00.272318 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da2f38ea-8823-49a4-a014-7a5970f29c0b service glance] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:32:00.340119 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-269280e6-1e74-4577-b6a5-efeee09a1429 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '42d15fc0515945acacbbffcf2356c7a5', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:32:00.341094 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-269280e6-1e74-4577-b6a5-efeee09a1429 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:32:00.341378 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-269280e6-1e74-4577-b6a5-efeee09a1429 admin admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:32:00.341651 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-269280e6-1e74-4577-b6a5-efeee09a1429 admin admin] PATH_INFO: `/v3/projects/9d0ee62a91f642abbeed3199a08b8623` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:32:00.342943 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-269280e6-1e74-4577-b6a5-efeee09a1429 admin admin] RBAC: Authorizing `identity:delete_project(project_id=9d0ee62a91f642abbeed3199a08b8623)` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:32:00.344885 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-269280e6-1e74-4577-b6a5-efeee09a1429 admin admin] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:32:00.379747 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG dbcounter [-] [80244] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80244) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:32:00.412862 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.notifications [None req-269280e6-1e74-4577-b6a5-efeee09a1429 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9d0ee62a91f642abbeed3199a08b8623', 'request_id': 'req-269280e6-1e74-4577-b6a5-efeee09a1429'} {{(pid=80244) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 08:32:00.422968 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.notifications [None req-269280e6-1e74-4577-b6a5-efeee09a1429 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9d0ee62a91f642abbeed3199a08b8623', 'request_id': 'req-269280e6-1e74-4577-b6a5-efeee09a1429'} {{(pid=80244) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 08:32:00.430679 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.notifications [None req-269280e6-1e74-4577-b6a5-efeee09a1429 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9d0ee62a91f642abbeed3199a08b8623', 'request_id': 'req-269280e6-1e74-4577-b6a5-efeee09a1429'} {{(pid=80244) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 08:32:00.442507 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.notifications [None req-269280e6-1e74-4577-b6a5-efeee09a1429 admin admin] The token cache is being invalidate because project 9d0ee62a91f642abbeed3199a08b8623 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80244) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 21 08:32:00.442793 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.notifications [None req-269280e6-1e74-4577-b6a5-efeee09a1429 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80244) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 08:32:00.443465 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.notifications [None req-269280e6-1e74-4577-b6a5-efeee09a1429 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80244) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 08:32:00.445202 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 1142/2290] 10.222.0.47 () {58 vars in 1232 bytes} [Thu Jul 21 08:32:00 2022] DELETE /identity/v3/projects/9d0ee62a91f642abbeed3199a08b8623 => generated 0 bytes in 190 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 08:32:00.449892 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a45694e-156f-4ba0-a9f3-8128b7bbf1a7 None None] REQUEST_METHOD: `POST` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:32:00.450551 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a45694e-156f-4ba0-a9f3-8128b7bbf1a7 None None] SCRIPT_NAME: `/identity` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:32:00.450551 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a45694e-156f-4ba0-a9f3-8128b7bbf1a7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:32:00.468961 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.auth.core [None req-7a45694e-156f-4ba0-a9f3-8128b7bbf1a7 None None] MFA Rules not processed for user `2dce40b8d74c41ec94cc36e6ff05d412`. Rule list: `[]` (Enabled: `True`). {{(pid=80245) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 08:32:00.470287 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.fernet_utils [None req-7a45694e-156f-4ba0-a9f3-8128b7bbf1a7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80245) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 08:32:00.533079 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: [pid: 80245|app: 0|req: 1149/2291] 10.222.0.47 () {58 vars in 957 bytes} [Thu Jul 21 08:32:00 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 08:32:00.542206 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28e24c98-7f79-49a1-b166-c8764213eefa None None] REQUEST_METHOD: `POST` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:32:00.542472 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28e24c98-7f79-49a1-b166-c8764213eefa None None] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:32:00.542767 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28e24c98-7f79-49a1-b166-c8764213eefa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:32:00.561471 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.auth.core [None req-28e24c98-7f79-49a1-b166-c8764213eefa None None] MFA Rules not processed for user `2dce40b8d74c41ec94cc36e6ff05d412`. Rule list: `[]` (Enabled: `True`). {{(pid=80244) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 08:32:00.562658 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.fernet_utils [None req-28e24c98-7f79-49a1-b166-c8764213eefa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80244) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 08:32:00.575055 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 1143/2292] 10.222.0.47 () {58 vars in 957 bytes} [Thu Jul 21 08:32:00 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 08:32:00.581435 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a45694e-156f-4ba0-a9f3-8128b7bbf1a7 None None] Authenticating user token {{(pid=80245) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:32:00.591354 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a45694e-156f-4ba0-a9f3-8128b7bbf1a7 None None] Validating token access rules against request {{(pid=80245) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:32:00.596450 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7c17eda-973e-45d6-8273-8e065dc18beb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '42d15fc0515945acacbbffcf2356c7a5', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80245) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:32:00.597125 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7c17eda-973e-45d6-8273-8e065dc18beb admin admin] REQUEST_METHOD: `GET` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:32:00.597411 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7c17eda-973e-45d6-8273-8e065dc18beb admin admin] SCRIPT_NAME: `/identity` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:32:00.597689 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7c17eda-973e-45d6-8273-8e065dc18beb admin admin] PATH_INFO: `/v3/domains` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:32:00.598420 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7c17eda-973e-45d6-8273-8e065dc18beb admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80245) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 08:32:00.598846 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7c17eda-973e-45d6-8273-8e065dc18beb admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:32:00.600622 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7c17eda-973e-45d6-8273-8e065dc18beb admin admin] RBAC: Authorization granted {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:32:00.627885 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: [pid: 80245|app: 0|req: 1150/2293] 10.222.0.47 () {58 vars in 1152 bytes} [Thu Jul 21 08:32:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 08:32:00.628281 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28e24c98-7f79-49a1-b166-c8764213eefa None None] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:32:00.637953 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28e24c98-7f79-49a1-b166-c8764213eefa None None] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:32:00.642693 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b54c1c6-c26a-43c7-8442-6a739bbc115c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '42d15fc0515945acacbbffcf2356c7a5', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:32:00.647595 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b54c1c6-c26a-43c7-8442-6a739bbc115c admin admin] REQUEST_METHOD: `POST` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:32:00.647595 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b54c1c6-c26a-43c7-8442-6a739bbc115c admin admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:32:00.647595 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b54c1c6-c26a-43c7-8442-6a739bbc115c admin admin] PATH_INFO: `/v3/projects` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:32:00.647595 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b54c1c6-c26a-43c7-8442-6a739bbc115c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:32:00.647595 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b54c1c6-c26a-43c7-8442-6a739bbc115c admin admin] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:32:00.681299 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 1144/2294] 10.222.0.47 () {60 vars in 1152 bytes} [Thu Jul 21 08:32:00 2022] POST /identity/v3/projects => generated 379 bytes in 51 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 08:32:00.683140 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7c17eda-973e-45d6-8273-8e065dc18beb admin admin] Authenticating user token {{(pid=80245) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:32:00.693251 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7c17eda-973e-45d6-8273-8e065dc18beb admin admin] Validating token access rules against request {{(pid=80245) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:32:00.751127 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cf0aaec-f9ca-44bc-a4b7-5f81dea37f6a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '42d15fc0515945acacbbffcf2356c7a5', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80245) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:32:00.751852 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cf0aaec-f9ca-44bc-a4b7-5f81dea37f6a admin admin] REQUEST_METHOD: `POST` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:32:00.752137 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cf0aaec-f9ca-44bc-a4b7-5f81dea37f6a admin admin] SCRIPT_NAME: `/identity` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:32:00.752417 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cf0aaec-f9ca-44bc-a4b7-5f81dea37f6a admin admin] PATH_INFO: `/v3/users` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:32:00.753117 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cf0aaec-f9ca-44bc-a4b7-5f81dea37f6a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:32:00.755352 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cf0aaec-f9ca-44bc-a4b7-5f81dea37f6a admin admin] RBAC: Authorization granted {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:32:00.775643 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: [pid: 80245|app: 0|req: 1151/2295] 10.222.0.47 () {60 vars in 1143 bytes} [Thu Jul 21 08:32:00 2022] POST /identity/v3/users => generated 290 bytes in 94 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 08:32:00.780281 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b54c1c6-c26a-43c7-8442-6a739bbc115c admin admin] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:32:00.791430 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b54c1c6-c26a-43c7-8442-6a739bbc115c admin admin] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:32:00.796801 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d2514ec-5201-4540-9805-e781cad648a6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '42d15fc0515945acacbbffcf2356c7a5', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:32:00.797478 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d2514ec-5201-4540-9805-e781cad648a6 admin admin] REQUEST_METHOD: `GET` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:32:00.797762 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d2514ec-5201-4540-9805-e781cad648a6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:32:00.798037 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d2514ec-5201-4540-9805-e781cad648a6 admin admin] PATH_INFO: `/v3/roles` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:32:00.798777 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d2514ec-5201-4540-9805-e781cad648a6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:32:00.800699 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d2514ec-5201-4540-9805-e781cad648a6 admin admin] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:32:00.811895 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 1145/2296] 10.222.0.47 () {58 vars in 1121 bytes} [Thu Jul 21 08:32:00 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 08:32:00.816674 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cf0aaec-f9ca-44bc-a4b7-5f81dea37f6a admin admin] Authenticating user token {{(pid=80245) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:32:00.827405 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cf0aaec-f9ca-44bc-a4b7-5f81dea37f6a admin admin] Validating token access rules against request {{(pid=80245) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:32:00.832367 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19fb7561-8ebf-4322-9937-7a0a5a0b6971 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '42d15fc0515945acacbbffcf2356c7a5', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80245) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:32:00.833046 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19fb7561-8ebf-4322-9937-7a0a5a0b6971 admin admin] REQUEST_METHOD: `GET` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:32:00.833334 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19fb7561-8ebf-4322-9937-7a0a5a0b6971 admin admin] SCRIPT_NAME: `/identity` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:32:00.833615 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19fb7561-8ebf-4322-9937-7a0a5a0b6971 admin admin] PATH_INFO: `/v3/roles` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:32:00.834353 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19fb7561-8ebf-4322-9937-7a0a5a0b6971 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:32:00.836010 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19fb7561-8ebf-4322-9937-7a0a5a0b6971 admin admin] RBAC: Authorization granted {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:32:00.846960 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: [pid: 80245|app: 0|req: 1152/2297] 10.222.0.47 () {58 vars in 1121 bytes} [Thu Jul 21 08:32:00 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 08:32:00.851296 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d2514ec-5201-4540-9805-e781cad648a6 admin admin] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:32:00.862462 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d2514ec-5201-4540-9805-e781cad648a6 admin admin] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:32:00.868099 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25b393d1-2799-48e9-85f5-bb0f68c65830 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '42d15fc0515945acacbbffcf2356c7a5', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:32:00.869216 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25b393d1-2799-48e9-85f5-bb0f68c65830 admin admin] REQUEST_METHOD: `PUT` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:32:00.869500 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25b393d1-2799-48e9-85f5-bb0f68c65830 admin admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:32:00.869784 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25b393d1-2799-48e9-85f5-bb0f68c65830 admin admin] PATH_INFO: `/v3/projects/f9a023b2c4674bb0b7924ca2c5abdba7/users/9218205cb6d54504b12a58cf3d78fd62/roles/3a074738221e4601b5205715ba87d240` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:32:00.886526 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25b393d1-2799-48e9-85f5-bb0f68c65830 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f9a023b2c4674bb0b7924ca2c5abdba7, user_id=9218205cb6d54504b12a58cf3d78fd62, role_id=3a074738221e4601b5205715ba87d240)` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:32:00.889174 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25b393d1-2799-48e9-85f5-bb0f68c65830 admin admin] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:32:00.902868 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 1146/2298] 10.222.0.47 () {60 vars in 1482 bytes} [Thu Jul 21 08:32:00 2022] PUT /identity/v3/projects/f9a023b2c4674bb0b7924ca2c5abdba7/users/9218205cb6d54504b12a58cf3d78fd62/roles/3a074738221e4601b5205715ba87d240 => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 08:32:00.913584 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3437412d-2a46-4ded-89c3-0990e82e6f64 None None] REQUEST_METHOD: `POST` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:32:00.913898 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3437412d-2a46-4ded-89c3-0990e82e6f64 None None] SCRIPT_NAME: `/identity` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:32:00.914182 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3437412d-2a46-4ded-89c3-0990e82e6f64 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:32:00.942782 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.auth.core [None req-3437412d-2a46-4ded-89c3-0990e82e6f64 None None] MFA Rules not processed for user `9218205cb6d54504b12a58cf3d78fd62`. Rule list: `[]` (Enabled: `True`). {{(pid=80245) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 08:32:00.944146 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.fernet_utils [None req-3437412d-2a46-4ded-89c3-0990e82e6f64 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80245) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 08:32:00.990332 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG dbcounter [-] [80245] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80245) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:32:01.016845 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: [pid: 80245|app: 0|req: 1153/2299] 10.222.0.47 () {58 vars in 957 bytes} [Thu Jul 21 08:32:00 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 08:32:01.021285 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25b393d1-2799-48e9-85f5-bb0f68c65830 admin admin] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:32:01.095902 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25b393d1-2799-48e9-85f5-bb0f68c65830 admin admin] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:32:01.102121 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b3adb11-ed8e-4b8e-a76b-3fcc69fc2f41 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '42d15fc0515945acacbbffcf2356c7a5', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:32:01.104257 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b3adb11-ed8e-4b8e-a76b-3fcc69fc2f41 admin admin] REQUEST_METHOD: `POST` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:32:01.105437 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b3adb11-ed8e-4b8e-a76b-3fcc69fc2f41 admin admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:32:01.105784 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b3adb11-ed8e-4b8e-a76b-3fcc69fc2f41 admin admin] PATH_INFO: `/v3/projects` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:32:01.106860 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b3adb11-ed8e-4b8e-a76b-3fcc69fc2f41 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:32:01.109051 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b3adb11-ed8e-4b8e-a76b-3fcc69fc2f41 admin admin] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:32:01.144041 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 1147/2300] 10.222.0.47 () {60 vars in 1152 bytes} [Thu Jul 21 08:32:01 2022] POST /identity/v3/projects => generated 381 bytes in 123 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 08:32:01.148162 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3437412d-2a46-4ded-89c3-0990e82e6f64 None None] Authenticating user token {{(pid=80245) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:32:01.161198 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3437412d-2a46-4ded-89c3-0990e82e6f64 None None] Validating token access rules against request {{(pid=80245) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:32:01.237592 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91a86421-be15-47bb-bd59-c26a8fa4f280 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '42d15fc0515945acacbbffcf2356c7a5', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80245) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:32:01.238277 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91a86421-be15-47bb-bd59-c26a8fa4f280 admin admin] REQUEST_METHOD: `POST` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:32:01.238562 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91a86421-be15-47bb-bd59-c26a8fa4f280 admin admin] SCRIPT_NAME: `/identity` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:32:01.238843 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91a86421-be15-47bb-bd59-c26a8fa4f280 admin admin] PATH_INFO: `/v3/users` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:32:01.239587 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91a86421-be15-47bb-bd59-c26a8fa4f280 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:32:01.241523 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91a86421-be15-47bb-bd59-c26a8fa4f280 admin admin] RBAC: Authorization granted {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:32:01.267353 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: [pid: 80245|app: 0|req: 1154/2301] 10.222.0.47 () {60 vars in 1143 bytes} [Thu Jul 21 08:32:01 2022] POST /identity/v3/users => generated 297 bytes in 118 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 08:32:01.269215 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b3adb11-ed8e-4b8e-a76b-3fcc69fc2f41 admin admin] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:32:01.280301 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b3adb11-ed8e-4b8e-a76b-3fcc69fc2f41 admin admin] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:32:01.285055 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cec9b9ac-4ab0-43e6-bd57-edcf2b081223 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '42d15fc0515945acacbbffcf2356c7a5', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:32:01.285732 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cec9b9ac-4ab0-43e6-bd57-edcf2b081223 admin admin] REQUEST_METHOD: `GET` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:32:01.286015 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cec9b9ac-4ab0-43e6-bd57-edcf2b081223 admin admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:32:01.286290 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cec9b9ac-4ab0-43e6-bd57-edcf2b081223 admin admin] PATH_INFO: `/v3/roles` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:32:01.287026 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cec9b9ac-4ab0-43e6-bd57-edcf2b081223 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:32:01.288669 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cec9b9ac-4ab0-43e6-bd57-edcf2b081223 admin admin] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:32:01.300641 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 1148/2302] 10.222.0.47 () {58 vars in 1121 bytes} [Thu Jul 21 08:32:01 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 08:32:01.306256 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91a86421-be15-47bb-bd59-c26a8fa4f280 admin admin] Authenticating user token {{(pid=80245) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:32:01.317334 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91a86421-be15-47bb-bd59-c26a8fa4f280 admin admin] Validating token access rules against request {{(pid=80245) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:32:01.322445 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4051d8e5-15f7-422a-a2d4-257722ffcd3f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '42d15fc0515945acacbbffcf2356c7a5', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80245) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:32:01.324215 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4051d8e5-15f7-422a-a2d4-257722ffcd3f admin admin] REQUEST_METHOD: `PUT` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:32:01.324552 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4051d8e5-15f7-422a-a2d4-257722ffcd3f admin admin] SCRIPT_NAME: `/identity` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:32:01.324839 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4051d8e5-15f7-422a-a2d4-257722ffcd3f admin admin] PATH_INFO: `/v3/projects/f16ecc6fa9bd43f7ac679aeed9c7d99d/users/06059b3646c4426ab4e1321cd7ef44d4/roles/e29870f6e9e2411a9db0d5bf45f32ca5` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:32:01.341919 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4051d8e5-15f7-422a-a2d4-257722ffcd3f admin admin] RBAC: Authorizing `identity:create_grant(project_id=f16ecc6fa9bd43f7ac679aeed9c7d99d, user_id=06059b3646c4426ab4e1321cd7ef44d4, role_id=e29870f6e9e2411a9db0d5bf45f32ca5)` {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:32:01.344969 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4051d8e5-15f7-422a-a2d4-257722ffcd3f admin admin] RBAC: Authorization granted {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:32:01.362124 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: [pid: 80245|app: 0|req: 1155/2303] 10.222.0.47 () {60 vars in 1482 bytes} [Thu Jul 21 08:32:01 2022] PUT /identity/v3/projects/f16ecc6fa9bd43f7ac679aeed9c7d99d/users/06059b3646c4426ab4e1321cd7ef44d4/roles/e29870f6e9e2411a9db0d5bf45f32ca5 => generated 0 bytes in 56 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 08:32:01.366382 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cec9b9ac-4ab0-43e6-bd57-edcf2b081223 admin admin] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:32:01.377240 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cec9b9ac-4ab0-43e6-bd57-edcf2b081223 admin admin] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:32:01.397080 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG dbcounter [-] [80244] Writing DB stats keystone:DELETE=7,keystone:SELECT=89,keystone:INSERT=4 {{(pid=80244) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:32:01.449046 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20e35061-66da-481f-8b57-c94db20bf854 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '42d15fc0515945acacbbffcf2356c7a5', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:32:01.449751 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20e35061-66da-481f-8b57-c94db20bf854 admin admin] REQUEST_METHOD: `GET` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:32:01.450029 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20e35061-66da-481f-8b57-c94db20bf854 admin admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:32:01.450328 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20e35061-66da-481f-8b57-c94db20bf854 admin admin] PATH_INFO: `/v3/roles` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:32:01.451058 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20e35061-66da-481f-8b57-c94db20bf854 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:32:01.452924 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20e35061-66da-481f-8b57-c94db20bf854 admin admin] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:32:01.474826 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 1149/2304] 10.222.0.47 () {58 vars in 1121 bytes} [Thu Jul 21 08:32:01 2022] GET /identity/v3/roles => generated 1420 bytes in 106 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 08:32:01.476889 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4051d8e5-15f7-422a-a2d4-257722ffcd3f admin admin] Authenticating user token {{(pid=80245) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:32:01.493852 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4051d8e5-15f7-422a-a2d4-257722ffcd3f admin admin] Validating token access rules against request {{(pid=80245) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:32:01.499217 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3956ab24-aef8-4379-9904-71c469741fb0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '42d15fc0515945acacbbffcf2356c7a5', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80245) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:32:01.500162 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3956ab24-aef8-4379-9904-71c469741fb0 admin admin] REQUEST_METHOD: `PUT` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:32:01.500449 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3956ab24-aef8-4379-9904-71c469741fb0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:32:01.500727 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3956ab24-aef8-4379-9904-71c469741fb0 admin admin] PATH_INFO: `/v3/projects/f16ecc6fa9bd43f7ac679aeed9c7d99d/users/06059b3646c4426ab4e1321cd7ef44d4/roles/e29870f6e9e2411a9db0d5bf45f32ca5` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:32:01.502745 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3956ab24-aef8-4379-9904-71c469741fb0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f16ecc6fa9bd43f7ac679aeed9c7d99d, user_id=06059b3646c4426ab4e1321cd7ef44d4, role_id=e29870f6e9e2411a9db0d5bf45f32ca5)` {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:32:01.505694 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3956ab24-aef8-4379-9904-71c469741fb0 admin admin] RBAC: Authorization granted {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:32:01.518347 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: [pid: 80245|app: 0|req: 1156/2305] 10.222.0.47 () {60 vars in 1482 bytes} [Thu Jul 21 08:32:01 2022] PUT /identity/v3/projects/f16ecc6fa9bd43f7ac679aeed9c7d99d/users/06059b3646c4426ab4e1321cd7ef44d4/roles/e29870f6e9e2411a9db0d5bf45f32ca5 => generated 0 bytes in 43 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 08:32:01.529874 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a716a8f-5ce3-4473-a722-6f63520454b1 None None] REQUEST_METHOD: `POST` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:32:01.530191 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a716a8f-5ce3-4473-a722-6f63520454b1 None None] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:32:01.530458 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a716a8f-5ce3-4473-a722-6f63520454b1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:32:01.561508 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.auth.core [None req-8a716a8f-5ce3-4473-a722-6f63520454b1 None None] MFA Rules not processed for user `06059b3646c4426ab4e1321cd7ef44d4`. Rule list: `[]` (Enabled: `True`). {{(pid=80244) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 08:32:01.563028 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.fernet_utils [None req-8a716a8f-5ce3-4473-a722-6f63520454b1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80244) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 08:32:01.659190 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 1150/2306] 10.222.0.47 () {58 vars in 957 bytes} [Thu Jul 21 08:32:01 2022] POST /identity/v3/auth/tokens => generated 3315 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 08:32:01.708380 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3956ab24-aef8-4379-9904-71c469741fb0 admin admin] Authenticating user token {{(pid=80245) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:32:01.710221 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.fernet_utils [None req-3956ab24-aef8-4379-9904-71c469741fb0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80245) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 08:32:01.840063 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3956ab24-aef8-4379-9904-71c469741fb0 admin admin] Validating token access rules against request {{(pid=80245) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:32:01.845596 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b6a1152-d49a-4716-ae22-5b8c9d93cce0 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4f40410b7674687ab5362ac982248ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4faef13aa50e4e36b6c265fb1398fa14', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80245) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:32:01.846597 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b6a1152-d49a-4716-ae22-5b8c9d93cce0 service cinder] REQUEST_METHOD: `GET` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:32:01.847032 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b6a1152-d49a-4716-ae22-5b8c9d93cce0 service cinder] SCRIPT_NAME: `/identity` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:32:01.847496 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b6a1152-d49a-4716-ae22-5b8c9d93cce0 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:32:01.861710 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b6a1152-d49a-4716-ae22-5b8c9d93cce0 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:32:01.863909 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b6a1152-d49a-4716-ae22-5b8c9d93cce0 service cinder] RBAC: Authorization granted {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:32:01.878129 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: [pid: 80245|app: 0|req: 1157/2307] 10.222.0.47 () {60 vars in 1365 bytes} [Thu Jul 21 08:32:01 2022] GET /identity/v3/auth/tokens => generated 3315 bytes in 170 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 08:32:02.596759 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a716a8f-5ce3-4473-a722-6f63520454b1 None None] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:32:02.685126 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a716a8f-5ce3-4473-a722-6f63520454b1 None None] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:32:02.690614 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f33d7bb0-e64f-4025-b91a-2b6e9edf206d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '42d15fc0515945acacbbffcf2356c7a5', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:32:02.691956 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f33d7bb0-e64f-4025-b91a-2b6e9edf206d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:32:02.692400 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f33d7bb0-e64f-4025-b91a-2b6e9edf206d admin admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:32:02.692857 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f33d7bb0-e64f-4025-b91a-2b6e9edf206d admin admin] PATH_INFO: `/v3/users/9218205cb6d54504b12a58cf3d78fd62` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:32:02.694310 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f33d7bb0-e64f-4025-b91a-2b6e9edf206d admin admin] RBAC: Authorizing `identity:delete_user(user_id=9218205cb6d54504b12a58cf3d78fd62)` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:32:02.696838 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f33d7bb0-e64f-4025-b91a-2b6e9edf206d admin admin] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:32:02.765006 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG dbcounter [-] [80244] Writing DB stats keystone:SELECT=94,keystone:DELETE=6 {{(pid=80244) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:32:02.768784 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.notifications [None req-f33d7bb0-e64f-4025-b91a-2b6e9edf206d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9218205cb6d54504b12a58cf3d78fd62'} {{(pid=80244) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 08:32:02.796849 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.notifications [None req-f33d7bb0-e64f-4025-b91a-2b6e9edf206d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9218205cb6d54504b12a58cf3d78fd62'} {{(pid=80244) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 08:32:02.816397 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.notifications [None req-f33d7bb0-e64f-4025-b91a-2b6e9edf206d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9218205cb6d54504b12a58cf3d78fd62'} {{(pid=80244) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 08:32:02.817261 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.notifications [None req-f33d7bb0-e64f-4025-b91a-2b6e9edf206d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9218205cb6d54504b12a58cf3d78fd62'} {{(pid=80244) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 08:32:02.835195 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.notifications [None req-f33d7bb0-e64f-4025-b91a-2b6e9edf206d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9218205cb6d54504b12a58cf3d78fd62'} {{(pid=80244) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 08:32:02.845910 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 1151/2308] 10.222.0.47 () {58 vars in 1223 bytes} [Thu Jul 21 08:32:02 2022] DELETE /identity/v3/users/9218205cb6d54504b12a58cf3d78fd62 => generated 0 bytes in 248 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 08:32:02.854271 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5534e2c3-0319-462f-9259-d426f677671e None None] REQUEST_METHOD: `POST` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:32:02.854271 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5534e2c3-0319-462f-9259-d426f677671e None None] SCRIPT_NAME: `/identity` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:32:02.854271 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5534e2c3-0319-462f-9259-d426f677671e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:32:02.882310 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.auth.core [None req-5534e2c3-0319-462f-9259-d426f677671e None None] MFA Rules not processed for user `2dce40b8d74c41ec94cc36e6ff05d412`. Rule list: `[]` (Enabled: `True`). {{(pid=80245) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 08:32:02.882310 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.fernet_utils [None req-5534e2c3-0319-462f-9259-d426f677671e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80245) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 08:32:02.925788 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG dbcounter [-] [80245] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80245) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:32:02.963827 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: [pid: 80245|app: 0|req: 1158/2309] 10.222.0.47 () {58 vars in 957 bytes} [Thu Jul 21 08:32:02 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 08:32:02.973393 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f33d7bb0-e64f-4025-b91a-2b6e9edf206d admin admin] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:32:02.974845 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.fernet_utils [None req-f33d7bb0-e64f-4025-b91a-2b6e9edf206d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80244) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 08:32:03.042781 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f33d7bb0-e64f-4025-b91a-2b6e9edf206d admin admin] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:32:03.046569 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26818af3-c2f6-46f1-b994-75ccd9bb0d55 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20ca3c3bd3774691b3374cfbfec6e494', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4faef13aa50e4e36b6c265fb1398fa14', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:32:03.047237 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26818af3-c2f6-46f1-b994-75ccd9bb0d55 service neutron] REQUEST_METHOD: `GET` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:32:03.047559 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26818af3-c2f6-46f1-b994-75ccd9bb0d55 service neutron] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:32:03.047835 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26818af3-c2f6-46f1-b994-75ccd9bb0d55 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:32:03.058617 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26818af3-c2f6-46f1-b994-75ccd9bb0d55 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:32:03.060110 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26818af3-c2f6-46f1-b994-75ccd9bb0d55 service neutron] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:32:03.070699 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 1152/2310] 10.222.0.47 () {60 vars in 1365 bytes} [Thu Jul 21 08:32:02 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 98 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 08:32:03.338832 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5534e2c3-0319-462f-9259-d426f677671e None None] Authenticating user token {{(pid=80245) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:32:03.340327 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.fernet_utils [None req-5534e2c3-0319-462f-9259-d426f677671e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80245) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 08:32:03.354253 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5534e2c3-0319-462f-9259-d426f677671e None None] Validating token access rules against request {{(pid=80245) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:32:03.359633 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a8c5ecf-8f79-4328-b40c-4423ab7b72b2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '42d15fc0515945acacbbffcf2356c7a5', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80245) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:32:03.360656 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a8c5ecf-8f79-4328-b40c-4423ab7b72b2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:32:03.361028 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a8c5ecf-8f79-4328-b40c-4423ab7b72b2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:32:03.361172 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a8c5ecf-8f79-4328-b40c-4423ab7b72b2 admin admin] PATH_INFO: `/v3/projects/f9a023b2c4674bb0b7924ca2c5abdba7` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:32:03.362807 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a8c5ecf-8f79-4328-b40c-4423ab7b72b2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f9a023b2c4674bb0b7924ca2c5abdba7)` {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:32:03.364783 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a8c5ecf-8f79-4328-b40c-4423ab7b72b2 admin admin] RBAC: Authorization granted {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:32:03.414006 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.notifications [None req-6a8c5ecf-8f79-4328-b40c-4423ab7b72b2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f9a023b2c4674bb0b7924ca2c5abdba7', 'request_id': 'req-6a8c5ecf-8f79-4328-b40c-4423ab7b72b2'} {{(pid=80245) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 08:32:03.420925 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.notifications [None req-6a8c5ecf-8f79-4328-b40c-4423ab7b72b2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f9a023b2c4674bb0b7924ca2c5abdba7', 'request_id': 'req-6a8c5ecf-8f79-4328-b40c-4423ab7b72b2'} {{(pid=80245) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 08:32:03.426674 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.notifications [None req-6a8c5ecf-8f79-4328-b40c-4423ab7b72b2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f9a023b2c4674bb0b7924ca2c5abdba7', 'request_id': 'req-6a8c5ecf-8f79-4328-b40c-4423ab7b72b2'} {{(pid=80245) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 08:32:03.439594 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.notifications [None req-6a8c5ecf-8f79-4328-b40c-4423ab7b72b2 admin admin] The token cache is being invalidate because project f9a023b2c4674bb0b7924ca2c5abdba7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80245) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 21 08:32:03.439884 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.notifications [None req-6a8c5ecf-8f79-4328-b40c-4423ab7b72b2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80245) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 08:32:03.440506 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.notifications [None req-6a8c5ecf-8f79-4328-b40c-4423ab7b72b2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80245) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 08:32:03.442066 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: [pid: 80245|app: 0|req: 1159/2311] 10.222.0.47 () {58 vars in 1232 bytes} [Thu Jul 21 08:32:03 2022] DELETE /identity/v3/projects/f9a023b2c4674bb0b7924ca2c5abdba7 => generated 0 bytes in 104 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 08:32:03.446058 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26818af3-c2f6-46f1-b994-75ccd9bb0d55 service neutron] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:32:03.455820 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26818af3-c2f6-46f1-b994-75ccd9bb0d55 service neutron] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:32:03.515144 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae3ae3e6-bdc1-49e0-bdc2-ec48e7800006 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '42d15fc0515945acacbbffcf2356c7a5', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:32:03.516041 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae3ae3e6-bdc1-49e0-bdc2-ec48e7800006 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:32:03.516326 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae3ae3e6-bdc1-49e0-bdc2-ec48e7800006 admin admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:32:03.516600 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae3ae3e6-bdc1-49e0-bdc2-ec48e7800006 admin admin] PATH_INFO: `/v3/users/06059b3646c4426ab4e1321cd7ef44d4` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:32:03.517784 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae3ae3e6-bdc1-49e0-bdc2-ec48e7800006 admin admin] RBAC: Authorizing `identity:delete_user(user_id=06059b3646c4426ab4e1321cd7ef44d4)` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:32:03.519747 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae3ae3e6-bdc1-49e0-bdc2-ec48e7800006 admin admin] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:32:03.566074 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.notifications [None req-ae3ae3e6-bdc1-49e0-bdc2-ec48e7800006 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '06059b3646c4426ab4e1321cd7ef44d4'} {{(pid=80244) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 08:32:03.573701 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.notifications [None req-ae3ae3e6-bdc1-49e0-bdc2-ec48e7800006 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '06059b3646c4426ab4e1321cd7ef44d4'} {{(pid=80244) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 08:32:03.582904 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG dbcounter [-] [80244] Writing DB stats keystone:DELETE=13,keystone:SELECT=85,keystone:INSERT=2 {{(pid=80244) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:32:03.590006 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.notifications [None req-ae3ae3e6-bdc1-49e0-bdc2-ec48e7800006 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '06059b3646c4426ab4e1321cd7ef44d4'} {{(pid=80244) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 08:32:03.591084 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.notifications [None req-ae3ae3e6-bdc1-49e0-bdc2-ec48e7800006 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '06059b3646c4426ab4e1321cd7ef44d4'} {{(pid=80244) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 08:32:03.598738 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.notifications [None req-ae3ae3e6-bdc1-49e0-bdc2-ec48e7800006 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '06059b3646c4426ab4e1321cd7ef44d4'} {{(pid=80244) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 08:32:03.605523 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 1153/2312] 10.222.0.47 () {58 vars in 1223 bytes} [Thu Jul 21 08:32:03 2022] DELETE /identity/v3/users/06059b3646c4426ab4e1321cd7ef44d4 => generated 0 bytes in 160 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 08:32:03.974913 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a8c5ecf-8f79-4328-b40c-4423ab7b72b2 admin admin] Authenticating user token {{(pid=80245) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:32:03.976302 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.fernet_utils [None req-6a8c5ecf-8f79-4328-b40c-4423ab7b72b2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80245) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 08:32:03.990054 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a8c5ecf-8f79-4328-b40c-4423ab7b72b2 admin admin] Validating token access rules against request {{(pid=80245) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:32:04.049231 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eec11412-3e42-41da-9f48-f64ab9749fc5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '42d15fc0515945acacbbffcf2356c7a5', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80245) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:32:04.050105 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eec11412-3e42-41da-9f48-f64ab9749fc5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:32:04.050386 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eec11412-3e42-41da-9f48-f64ab9749fc5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:32:04.050661 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eec11412-3e42-41da-9f48-f64ab9749fc5 admin admin] PATH_INFO: `/v3/projects/f16ecc6fa9bd43f7ac679aeed9c7d99d` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:32:04.051897 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eec11412-3e42-41da-9f48-f64ab9749fc5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f16ecc6fa9bd43f7ac679aeed9c7d99d)` {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:32:04.053817 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eec11412-3e42-41da-9f48-f64ab9749fc5 admin admin] RBAC: Authorization granted {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:32:04.101591 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.notifications [None req-eec11412-3e42-41da-9f48-f64ab9749fc5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f16ecc6fa9bd43f7ac679aeed9c7d99d', 'request_id': 'req-eec11412-3e42-41da-9f48-f64ab9749fc5'} {{(pid=80245) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 08:32:04.109224 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.notifications [None req-eec11412-3e42-41da-9f48-f64ab9749fc5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f16ecc6fa9bd43f7ac679aeed9c7d99d', 'request_id': 'req-eec11412-3e42-41da-9f48-f64ab9749fc5'} {{(pid=80245) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 08:32:04.115382 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.notifications [None req-eec11412-3e42-41da-9f48-f64ab9749fc5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f16ecc6fa9bd43f7ac679aeed9c7d99d', 'request_id': 'req-eec11412-3e42-41da-9f48-f64ab9749fc5'} {{(pid=80245) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 08:32:04.117109 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG dbcounter [-] [80245] Writing DB stats keystone:SELECT=86,keystone:DELETE=13,keystone:INSERT=1 {{(pid=80245) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:32:04.132284 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.notifications [None req-eec11412-3e42-41da-9f48-f64ab9749fc5 admin admin] The token cache is being invalidate because project f16ecc6fa9bd43f7ac679aeed9c7d99d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80245) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 21 08:32:04.132571 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.notifications [None req-eec11412-3e42-41da-9f48-f64ab9749fc5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80245) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 08:32:04.133200 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.notifications [None req-eec11412-3e42-41da-9f48-f64ab9749fc5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80245) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 08:32:04.135576 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: [pid: 80245|app: 0|req: 1160/2313] 10.222.0.47 () {58 vars in 1232 bytes} [Thu Jul 21 08:32:03 2022] DELETE /identity/v3/projects/f16ecc6fa9bd43f7ac679aeed9c7d99d => generated 0 bytes in 161 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 08:32:04.140327 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bf5f184-3f00-4d09-af6c-ed00c26904de None None] REQUEST_METHOD: `POST` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:32:04.140621 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bf5f184-3f00-4d09-af6c-ed00c26904de None None] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:32:04.140896 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bf5f184-3f00-4d09-af6c-ed00c26904de None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:32:04.159628 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.auth.core [None req-1bf5f184-3f00-4d09-af6c-ed00c26904de None None] MFA Rules not processed for user `2dce40b8d74c41ec94cc36e6ff05d412`. Rule list: `[]` (Enabled: `True`). {{(pid=80244) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 08:32:04.160889 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.fernet_utils [None req-1bf5f184-3f00-4d09-af6c-ed00c26904de None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80244) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 08:32:04.222101 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 1154/2314] 10.222.0.47 () {58 vars in 957 bytes} [Thu Jul 21 08:32:04 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 08:32:04.231659 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e6f02a9-4a4f-4c26-a436-f91b18cecc59 None None] REQUEST_METHOD: `POST` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:32:04.231997 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e6f02a9-4a4f-4c26-a436-f91b18cecc59 None None] SCRIPT_NAME: `/identity` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:32:04.232311 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e6f02a9-4a4f-4c26-a436-f91b18cecc59 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:32:04.251614 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.auth.core [None req-8e6f02a9-4a4f-4c26-a436-f91b18cecc59 None None] MFA Rules not processed for user `2dce40b8d74c41ec94cc36e6ff05d412`. Rule list: `[]` (Enabled: `True`). {{(pid=80245) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 08:32:04.252892 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.fernet_utils [None req-8e6f02a9-4a4f-4c26-a436-f91b18cecc59 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80245) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 08:32:04.262322 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: [pid: 80245|app: 0|req: 1161/2315] 10.222.0.47 () {58 vars in 957 bytes} [Thu Jul 21 08:32:04 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 08:32:04.267248 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bf5f184-3f00-4d09-af6c-ed00c26904de None None] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:32:04.278572 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bf5f184-3f00-4d09-af6c-ed00c26904de None None] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:32:04.283295 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd90e5b1-ffd3-47d8-9452-89fa4b3d71e1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '42d15fc0515945acacbbffcf2356c7a5', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:32:04.283927 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd90e5b1-ffd3-47d8-9452-89fa4b3d71e1 admin admin] REQUEST_METHOD: `GET` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:32:04.284212 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd90e5b1-ffd3-47d8-9452-89fa4b3d71e1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:32:04.284489 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd90e5b1-ffd3-47d8-9452-89fa4b3d71e1 admin admin] PATH_INFO: `/v3/domains` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:32:04.285222 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd90e5b1-ffd3-47d8-9452-89fa4b3d71e1 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80244) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 08:32:04.285530 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd90e5b1-ffd3-47d8-9452-89fa4b3d71e1 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:32:04.287156 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd90e5b1-ffd3-47d8-9452-89fa4b3d71e1 admin admin] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:32:04.298954 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 1155/2316] 10.222.0.47 () {58 vars in 1152 bytes} [Thu Jul 21 08:32:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 08:32:04.303476 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e6f02a9-4a4f-4c26-a436-f91b18cecc59 None None] Authenticating user token {{(pid=80245) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:32:04.312849 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e6f02a9-4a4f-4c26-a436-f91b18cecc59 None None] Validating token access rules against request {{(pid=80245) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:32:04.317625 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ad3f48c-00d7-46d9-8841-f14706fb331d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '42d15fc0515945acacbbffcf2356c7a5', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80245) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:32:04.318279 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ad3f48c-00d7-46d9-8841-f14706fb331d admin admin] REQUEST_METHOD: `POST` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:32:04.318554 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ad3f48c-00d7-46d9-8841-f14706fb331d admin admin] SCRIPT_NAME: `/identity` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:32:04.318828 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ad3f48c-00d7-46d9-8841-f14706fb331d admin admin] PATH_INFO: `/v3/projects` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:32:04.319559 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ad3f48c-00d7-46d9-8841-f14706fb331d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:32:04.321436 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ad3f48c-00d7-46d9-8841-f14706fb331d admin admin] RBAC: Authorization granted {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:32:04.353218 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: [pid: 80245|app: 0|req: 1162/2317] 10.222.0.47 () {60 vars in 1152 bytes} [Thu Jul 21 08:32:04 2022] POST /identity/v3/projects => generated 373 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 08:32:04.354754 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd90e5b1-ffd3-47d8-9452-89fa4b3d71e1 admin admin] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:32:04.372831 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd90e5b1-ffd3-47d8-9452-89fa4b3d71e1 admin admin] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:32:04.435434 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1954b218-a136-4c17-89f1-9b95bd040720 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '42d15fc0515945acacbbffcf2356c7a5', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:32:04.436325 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1954b218-a136-4c17-89f1-9b95bd040720 admin admin] REQUEST_METHOD: `POST` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:32:04.436640 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1954b218-a136-4c17-89f1-9b95bd040720 admin admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:32:04.436913 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1954b218-a136-4c17-89f1-9b95bd040720 admin admin] PATH_INFO: `/v3/users` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:32:04.437610 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1954b218-a136-4c17-89f1-9b95bd040720 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:32:04.439749 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1954b218-a136-4c17-89f1-9b95bd040720 admin admin] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:32:04.465307 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 1156/2318] 10.222.0.47 () {60 vars in 1143 bytes} [Thu Jul 21 08:32:04 2022] POST /identity/v3/users => generated 287 bytes in 108 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 08:32:04.466532 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ad3f48c-00d7-46d9-8841-f14706fb331d admin admin] Authenticating user token {{(pid=80245) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:32:04.476828 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ad3f48c-00d7-46d9-8841-f14706fb331d admin admin] Validating token access rules against request {{(pid=80245) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:32:04.481577 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a0cec87-91f8-4c1a-846e-698650cee84e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '42d15fc0515945acacbbffcf2356c7a5', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80245) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:32:04.482248 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a0cec87-91f8-4c1a-846e-698650cee84e admin admin] REQUEST_METHOD: `GET` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:32:04.482535 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a0cec87-91f8-4c1a-846e-698650cee84e admin admin] SCRIPT_NAME: `/identity` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:32:04.482817 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a0cec87-91f8-4c1a-846e-698650cee84e admin admin] PATH_INFO: `/v3/roles` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:32:04.483606 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a0cec87-91f8-4c1a-846e-698650cee84e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:32:04.485213 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a0cec87-91f8-4c1a-846e-698650cee84e admin admin] RBAC: Authorization granted {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:32:04.496423 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: [pid: 80245|app: 0|req: 1163/2319] 10.222.0.47 () {58 vars in 1121 bytes} [Thu Jul 21 08:32:04 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 08:32:04.500494 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1954b218-a136-4c17-89f1-9b95bd040720 admin admin] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:32:04.511313 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1954b218-a136-4c17-89f1-9b95bd040720 admin admin] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:32:04.516134 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7df9120b-42fd-48fa-b69c-11546edd8d74 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '42d15fc0515945acacbbffcf2356c7a5', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:32:04.516804 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7df9120b-42fd-48fa-b69c-11546edd8d74 admin admin] REQUEST_METHOD: `GET` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:32:04.517113 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7df9120b-42fd-48fa-b69c-11546edd8d74 admin admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:32:04.517394 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7df9120b-42fd-48fa-b69c-11546edd8d74 admin admin] PATH_INFO: `/v3/roles` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:32:04.518149 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7df9120b-42fd-48fa-b69c-11546edd8d74 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:32:04.519813 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7df9120b-42fd-48fa-b69c-11546edd8d74 admin admin] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:32:04.530313 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 1157/2320] 10.222.0.47 () {58 vars in 1121 bytes} [Thu Jul 21 08:32:04 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 08:32:04.534501 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a0cec87-91f8-4c1a-846e-698650cee84e admin admin] Authenticating user token {{(pid=80245) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:32:04.544761 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a0cec87-91f8-4c1a-846e-698650cee84e admin admin] Validating token access rules against request {{(pid=80245) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:32:04.549363 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2f328f2-2ef4-432d-b721-19e0a411b41a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '42d15fc0515945acacbbffcf2356c7a5', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80245) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:32:04.550247 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2f328f2-2ef4-432d-b721-19e0a411b41a admin admin] REQUEST_METHOD: `PUT` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:32:04.550527 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2f328f2-2ef4-432d-b721-19e0a411b41a admin admin] SCRIPT_NAME: `/identity` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:32:04.550801 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2f328f2-2ef4-432d-b721-19e0a411b41a admin admin] PATH_INFO: `/v3/projects/06549f7a0a214370b4fe64ae3a3c1b96/users/9e83bb1f79f640d49cab6ca3249f37ab/roles/3a074738221e4601b5205715ba87d240` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:32:04.568627 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2f328f2-2ef4-432d-b721-19e0a411b41a admin admin] RBAC: Authorizing `identity:create_grant(project_id=06549f7a0a214370b4fe64ae3a3c1b96, user_id=9e83bb1f79f640d49cab6ca3249f37ab, role_id=3a074738221e4601b5205715ba87d240)` {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:32:04.571277 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2f328f2-2ef4-432d-b721-19e0a411b41a admin admin] RBAC: Authorization granted {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:32:04.584561 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: [pid: 80245|app: 0|req: 1164/2321] 10.222.0.47 () {60 vars in 1482 bytes} [Thu Jul 21 08:32:04 2022] PUT /identity/v3/projects/06549f7a0a214370b4fe64ae3a3c1b96/users/9e83bb1f79f640d49cab6ca3249f37ab/roles/3a074738221e4601b5205715ba87d240 => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 08:32:04.594625 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bcb9920-3632-4948-86ab-c8e72d5909e9 None None] REQUEST_METHOD: `POST` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:32:04.594922 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bcb9920-3632-4948-86ab-c8e72d5909e9 None None] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:32:04.595205 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bcb9920-3632-4948-86ab-c8e72d5909e9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:32:04.624395 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.auth.core [None req-6bcb9920-3632-4948-86ab-c8e72d5909e9 None None] MFA Rules not processed for user `9e83bb1f79f640d49cab6ca3249f37ab`. Rule list: `[]` (Enabled: `True`). {{(pid=80244) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 08:32:04.625634 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.fernet_utils [None req-6bcb9920-3632-4948-86ab-c8e72d5909e9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80244) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 08:32:04.666671 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG dbcounter [-] [80244] Writing DB stats keystone:SELECT=96,keystone:DELETE=1,keystone:INSERT=3 {{(pid=80244) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:32:04.701277 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 1158/2322] 10.222.0.47 () {58 vars in 957 bytes} [Thu Jul 21 08:32:04 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 08:32:04.705882 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2f328f2-2ef4-432d-b721-19e0a411b41a admin admin] Authenticating user token {{(pid=80245) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:32:04.773944 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2f328f2-2ef4-432d-b721-19e0a411b41a admin admin] Validating token access rules against request {{(pid=80245) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:32:04.778736 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0af03a3a-ac65-4a2c-b72a-bec5ac435141 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '42d15fc0515945acacbbffcf2356c7a5', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80245) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:32:04.779433 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0af03a3a-ac65-4a2c-b72a-bec5ac435141 admin admin] REQUEST_METHOD: `POST` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:32:04.779720 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0af03a3a-ac65-4a2c-b72a-bec5ac435141 admin admin] SCRIPT_NAME: `/identity` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:32:04.780001 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0af03a3a-ac65-4a2c-b72a-bec5ac435141 admin admin] PATH_INFO: `/v3/projects` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:32:04.780703 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0af03a3a-ac65-4a2c-b72a-bec5ac435141 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:32:04.782640 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0af03a3a-ac65-4a2c-b72a-bec5ac435141 admin admin] RBAC: Authorization granted {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:32:04.808380 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: [pid: 80245|app: 0|req: 1165/2323] 10.222.0.47 () {60 vars in 1152 bytes} [Thu Jul 21 08:32:04 2022] POST /identity/v3/projects => generated 375 bytes in 103 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 08:32:04.812715 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bcb9920-3632-4948-86ab-c8e72d5909e9 None None] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:32:04.826150 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bcb9920-3632-4948-86ab-c8e72d5909e9 None None] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:32:04.892142 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24261cac-92bc-4bb6-9e8a-04b5150558a7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '42d15fc0515945acacbbffcf2356c7a5', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:32:04.892142 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24261cac-92bc-4bb6-9e8a-04b5150558a7 admin admin] REQUEST_METHOD: `POST` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:32:04.892142 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24261cac-92bc-4bb6-9e8a-04b5150558a7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:32:04.892142 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24261cac-92bc-4bb6-9e8a-04b5150558a7 admin admin] PATH_INFO: `/v3/users` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:32:04.892432 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24261cac-92bc-4bb6-9e8a-04b5150558a7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:32:04.893752 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24261cac-92bc-4bb6-9e8a-04b5150558a7 admin admin] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:32:04.919640 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 1159/2324] 10.222.0.47 () {60 vars in 1143 bytes} [Thu Jul 21 08:32:04 2022] POST /identity/v3/users => generated 294 bytes in 104 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 08:32:04.920960 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0af03a3a-ac65-4a2c-b72a-bec5ac435141 admin admin] Authenticating user token {{(pid=80245) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:32:04.935033 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0af03a3a-ac65-4a2c-b72a-bec5ac435141 admin admin] Validating token access rules against request {{(pid=80245) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:32:04.944681 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-965ace24-1715-4397-8201-71376b9e96d6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '42d15fc0515945acacbbffcf2356c7a5', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80245) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:32:04.945383 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-965ace24-1715-4397-8201-71376b9e96d6 admin admin] REQUEST_METHOD: `GET` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:32:04.945670 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-965ace24-1715-4397-8201-71376b9e96d6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:32:04.945953 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-965ace24-1715-4397-8201-71376b9e96d6 admin admin] PATH_INFO: `/v3/roles` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:32:04.946730 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-965ace24-1715-4397-8201-71376b9e96d6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:32:04.948485 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-965ace24-1715-4397-8201-71376b9e96d6 admin admin] RBAC: Authorization granted {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:32:04.960049 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: [pid: 80245|app: 0|req: 1166/2325] 10.222.0.47 () {58 vars in 1121 bytes} [Thu Jul 21 08:32:04 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 21 08:32:04.964807 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24261cac-92bc-4bb6-9e8a-04b5150558a7 admin admin] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:32:04.981398 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24261cac-92bc-4bb6-9e8a-04b5150558a7 admin admin] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:32:04.986290 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24fad28a-794f-4c1f-92eb-25311c824101 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '42d15fc0515945acacbbffcf2356c7a5', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:32:04.987287 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24fad28a-794f-4c1f-92eb-25311c824101 admin admin] REQUEST_METHOD: `PUT` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:32:04.987633 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24fad28a-794f-4c1f-92eb-25311c824101 admin admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:32:04.987924 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24fad28a-794f-4c1f-92eb-25311c824101 admin admin] PATH_INFO: `/v3/projects/1154d87830ea4f85b7236414b05c8e46/users/a1c1076a63724f84b6bf3eb9eb51ece7/roles/e29870f6e9e2411a9db0d5bf45f32ca5` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:32:05.003277 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24fad28a-794f-4c1f-92eb-25311c824101 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1154d87830ea4f85b7236414b05c8e46, user_id=a1c1076a63724f84b6bf3eb9eb51ece7, role_id=e29870f6e9e2411a9db0d5bf45f32ca5)` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:32:05.006197 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24fad28a-794f-4c1f-92eb-25311c824101 admin admin] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:32:05.021435 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 1160/2326] 10.222.0.47 () {60 vars in 1482 bytes} [Thu Jul 21 08:32:04 2022] PUT /identity/v3/projects/1154d87830ea4f85b7236414b05c8e46/users/a1c1076a63724f84b6bf3eb9eb51ece7/roles/e29870f6e9e2411a9db0d5bf45f32ca5 => generated 0 bytes in 57 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 08:32:05.025543 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-965ace24-1715-4397-8201-71376b9e96d6 admin admin] Authenticating user token {{(pid=80245) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:32:05.035689 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-965ace24-1715-4397-8201-71376b9e96d6 admin admin] Validating token access rules against request {{(pid=80245) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:32:05.097105 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28df0f3a-51a0-47eb-bbb7-0b90b907dc81 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '42d15fc0515945acacbbffcf2356c7a5', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80245) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:32:05.097697 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28df0f3a-51a0-47eb-bbb7-0b90b907dc81 admin admin] REQUEST_METHOD: `GET` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:32:05.098018 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28df0f3a-51a0-47eb-bbb7-0b90b907dc81 admin admin] SCRIPT_NAME: `/identity` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:32:05.098267 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28df0f3a-51a0-47eb-bbb7-0b90b907dc81 admin admin] PATH_INFO: `/v3/roles` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:32:05.099069 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28df0f3a-51a0-47eb-bbb7-0b90b907dc81 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:32:05.100975 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28df0f3a-51a0-47eb-bbb7-0b90b907dc81 admin admin] RBAC: Authorization granted {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:32:05.104252 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG dbcounter [-] [80245] Writing DB stats keystone:INSERT=4,keystone:DELETE=1,keystone:SELECT=95 {{(pid=80245) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:32:05.114654 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: [pid: 80245|app: 0|req: 1167/2327] 10.222.0.47 () {58 vars in 1121 bytes} [Thu Jul 21 08:32:05 2022] GET /identity/v3/roles => generated 1420 bytes in 89 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 08:32:05.118918 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24fad28a-794f-4c1f-92eb-25311c824101 admin admin] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:32:05.130129 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24fad28a-794f-4c1f-92eb-25311c824101 admin admin] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:32:05.134831 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9f99aba-92dc-4ae3-868f-d5cd8de6814c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '42d15fc0515945acacbbffcf2356c7a5', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:32:05.136089 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9f99aba-92dc-4ae3-868f-d5cd8de6814c admin admin] REQUEST_METHOD: `PUT` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:32:05.136089 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9f99aba-92dc-4ae3-868f-d5cd8de6814c admin admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:32:05.136334 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9f99aba-92dc-4ae3-868f-d5cd8de6814c admin admin] PATH_INFO: `/v3/projects/1154d87830ea4f85b7236414b05c8e46/users/a1c1076a63724f84b6bf3eb9eb51ece7/roles/e29870f6e9e2411a9db0d5bf45f32ca5` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:32:05.138297 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9f99aba-92dc-4ae3-868f-d5cd8de6814c admin admin] RBAC: Authorizing `identity:create_grant(project_id=1154d87830ea4f85b7236414b05c8e46, user_id=a1c1076a63724f84b6bf3eb9eb51ece7, role_id=e29870f6e9e2411a9db0d5bf45f32ca5)` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:32:05.140995 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9f99aba-92dc-4ae3-868f-d5cd8de6814c admin admin] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:32:05.151160 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 1161/2328] 10.222.0.47 () {60 vars in 1482 bytes} [Thu Jul 21 08:32:05 2022] PUT /identity/v3/projects/1154d87830ea4f85b7236414b05c8e46/users/a1c1076a63724f84b6bf3eb9eb51ece7/roles/e29870f6e9e2411a9db0d5bf45f32ca5 => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 08:32:05.220925 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2cfefe2-32f6-4b8c-ab9a-25dd802db0c8 None None] REQUEST_METHOD: `POST` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:32:05.221218 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2cfefe2-32f6-4b8c-ab9a-25dd802db0c8 None None] SCRIPT_NAME: `/identity` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:32:05.221511 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2cfefe2-32f6-4b8c-ab9a-25dd802db0c8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:32:05.251891 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.auth.core [None req-e2cfefe2-32f6-4b8c-ab9a-25dd802db0c8 None None] MFA Rules not processed for user `a1c1076a63724f84b6bf3eb9eb51ece7`. Rule list: `[]` (Enabled: `True`). {{(pid=80245) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 08:32:05.254500 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.fernet_utils [None req-e2cfefe2-32f6-4b8c-ab9a-25dd802db0c8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80245) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 08:32:05.338014 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: [pid: 80245|app: 0|req: 1168/2329] 10.222.0.47 () {58 vars in 957 bytes} [Thu Jul 21 08:32:05 2022] POST /identity/v3/auth/tokens => generated 3309 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 08:32:05.349205 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9f99aba-92dc-4ae3-868f-d5cd8de6814c admin admin] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:32:05.350842 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.fernet_utils [None req-d9f99aba-92dc-4ae3-868f-d5cd8de6814c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80244) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 08:32:05.432030 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9f99aba-92dc-4ae3-868f-d5cd8de6814c admin admin] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:32:05.436077 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a6a1bef-3778-4c24-9c63-1a11d3455f93 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4f40410b7674687ab5362ac982248ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4faef13aa50e4e36b6c265fb1398fa14', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:32:05.436800 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a6a1bef-3778-4c24-9c63-1a11d3455f93 service cinder] REQUEST_METHOD: `GET` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:32:05.437149 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a6a1bef-3778-4c24-9c63-1a11d3455f93 service cinder] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:32:05.437488 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a6a1bef-3778-4c24-9c63-1a11d3455f93 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:32:05.490578 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG dbcounter [-] [80244] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80244) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:32:05.510655 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a6a1bef-3778-4c24-9c63-1a11d3455f93 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:32:05.513046 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a6a1bef-3778-4c24-9c63-1a11d3455f93 service cinder] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:32:05.527341 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 1162/2330] 10.222.0.47 () {60 vars in 1365 bytes} [Thu Jul 21 08:32:05 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 178 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 08:32:07.124523 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2cfefe2-32f6-4b8c-ab9a-25dd802db0c8 None None] Authenticating user token {{(pid=80245) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:32:07.128752 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.fernet_utils [None req-e2cfefe2-32f6-4b8c-ab9a-25dd802db0c8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80245) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 08:32:07.200976 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2cfefe2-32f6-4b8c-ab9a-25dd802db0c8 None None] Validating token access rules against request {{(pid=80245) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:32:07.205580 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a74f58b-7ec3-467b-8efc-139fcded3a2b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '42d15fc0515945acacbbffcf2356c7a5', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80245) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:32:07.206514 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a74f58b-7ec3-467b-8efc-139fcded3a2b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:32:07.206812 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a74f58b-7ec3-467b-8efc-139fcded3a2b admin admin] SCRIPT_NAME: `/identity` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:32:07.207089 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a74f58b-7ec3-467b-8efc-139fcded3a2b admin admin] PATH_INFO: `/v3/users/c7e96b1679ae4ce9aec31209202a894a` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:32:07.208313 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a74f58b-7ec3-467b-8efc-139fcded3a2b admin admin] RBAC: Authorizing `identity:delete_user(user_id=c7e96b1679ae4ce9aec31209202a894a)` {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:32:07.210240 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a74f58b-7ec3-467b-8efc-139fcded3a2b admin admin] RBAC: Authorization granted {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:32:07.258198 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.notifications [None req-2a74f58b-7ec3-467b-8efc-139fcded3a2b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c7e96b1679ae4ce9aec31209202a894a'} {{(pid=80245) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 08:32:07.266267 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.notifications [None req-2a74f58b-7ec3-467b-8efc-139fcded3a2b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c7e96b1679ae4ce9aec31209202a894a'} {{(pid=80245) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 08:32:07.279711 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.notifications [None req-2a74f58b-7ec3-467b-8efc-139fcded3a2b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c7e96b1679ae4ce9aec31209202a894a'} {{(pid=80245) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 08:32:07.286830 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.notifications [None req-2a74f58b-7ec3-467b-8efc-139fcded3a2b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c7e96b1679ae4ce9aec31209202a894a'} {{(pid=80245) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 08:32:07.287566 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.notifications [None req-2a74f58b-7ec3-467b-8efc-139fcded3a2b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c7e96b1679ae4ce9aec31209202a894a'} {{(pid=80245) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 08:32:07.294936 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: [pid: 80245|app: 0|req: 1169/2331] 10.222.0.47 () {58 vars in 1223 bytes} [Thu Jul 21 08:32:07 2022] DELETE /identity/v3/users/c7e96b1679ae4ce9aec31209202a894a => generated 0 bytes in 171 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 21 08:32:07.299753 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-469bd6c2-5a60-41f0-a93e-aa9efb43a10e None None] REQUEST_METHOD: `POST` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:32:07.300652 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-469bd6c2-5a60-41f0-a93e-aa9efb43a10e None None] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:32:07.300987 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-469bd6c2-5a60-41f0-a93e-aa9efb43a10e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:32:07.324422 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.auth.core [None req-469bd6c2-5a60-41f0-a93e-aa9efb43a10e None None] MFA Rules not processed for user `2dce40b8d74c41ec94cc36e6ff05d412`. Rule list: `[]` (Enabled: `True`). {{(pid=80244) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 08:32:07.325678 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.fernet_utils [None req-469bd6c2-5a60-41f0-a93e-aa9efb43a10e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80244) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 08:32:07.391527 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 1163/2332] 10.222.0.47 () {58 vars in 957 bytes} [Thu Jul 21 08:32:07 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 08:32:07.405376 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a74f58b-7ec3-467b-8efc-139fcded3a2b admin admin] Authenticating user token {{(pid=80245) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:32:07.406839 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.fernet_utils [None req-2a74f58b-7ec3-467b-8efc-139fcded3a2b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80245) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 08:32:07.413172 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG dbcounter [-] [80245] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80245) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:32:07.476431 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a74f58b-7ec3-467b-8efc-139fcded3a2b admin admin] Validating token access rules against request {{(pid=80245) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:32:07.480327 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28cd14b8-6d39-44cc-a4a1-295c6987b665 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20ca3c3bd3774691b3374cfbfec6e494', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4faef13aa50e4e36b6c265fb1398fa14', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80245) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:32:07.480989 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28cd14b8-6d39-44cc-a4a1-295c6987b665 service neutron] REQUEST_METHOD: `GET` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:32:07.481278 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28cd14b8-6d39-44cc-a4a1-295c6987b665 service neutron] SCRIPT_NAME: `/identity` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:32:07.481559 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28cd14b8-6d39-44cc-a4a1-295c6987b665 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:32:07.491472 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28cd14b8-6d39-44cc-a4a1-295c6987b665 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:32:07.492969 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28cd14b8-6d39-44cc-a4a1-295c6987b665 service neutron] RBAC: Authorization granted {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:32:07.503225 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: [pid: 80245|app: 0|req: 1170/2333] 10.222.0.47 () {60 vars in 1365 bytes} [Thu Jul 21 08:32:07 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 98 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 08:32:07.899291 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-469bd6c2-5a60-41f0-a93e-aa9efb43a10e None None] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:32:07.900828 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.fernet_utils [None req-469bd6c2-5a60-41f0-a93e-aa9efb43a10e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80244) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 08:32:07.915004 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-469bd6c2-5a60-41f0-a93e-aa9efb43a10e None None] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:32:07.928611 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82bc5067-5822-4696-ae78-47df851ccc5d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '42d15fc0515945acacbbffcf2356c7a5', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:32:07.930375 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82bc5067-5822-4696-ae78-47df851ccc5d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:32:07.930375 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82bc5067-5822-4696-ae78-47df851ccc5d admin admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:32:07.930375 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82bc5067-5822-4696-ae78-47df851ccc5d admin admin] PATH_INFO: `/v3/projects/ddde32ff934748c487b54e47f33b4125` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:32:07.931610 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82bc5067-5822-4696-ae78-47df851ccc5d admin admin] RBAC: Authorizing `identity:delete_project(project_id=ddde32ff934748c487b54e47f33b4125)` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:32:07.934561 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82bc5067-5822-4696-ae78-47df851ccc5d admin admin] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:32:07.984329 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.notifications [None req-82bc5067-5822-4696-ae78-47df851ccc5d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ddde32ff934748c487b54e47f33b4125', 'request_id': 'req-82bc5067-5822-4696-ae78-47df851ccc5d'} {{(pid=80244) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 08:32:07.993725 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.notifications [None req-82bc5067-5822-4696-ae78-47df851ccc5d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ddde32ff934748c487b54e47f33b4125', 'request_id': 'req-82bc5067-5822-4696-ae78-47df851ccc5d'} {{(pid=80244) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 08:32:08.001027 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.notifications [None req-82bc5067-5822-4696-ae78-47df851ccc5d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ddde32ff934748c487b54e47f33b4125', 'request_id': 'req-82bc5067-5822-4696-ae78-47df851ccc5d'} {{(pid=80244) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 08:32:08.011961 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.notifications [None req-82bc5067-5822-4696-ae78-47df851ccc5d admin admin] The token cache is being invalidate because project ddde32ff934748c487b54e47f33b4125 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80244) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 21 08:32:08.012248 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.notifications [None req-82bc5067-5822-4696-ae78-47df851ccc5d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80244) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 08:32:08.012896 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.notifications [None req-82bc5067-5822-4696-ae78-47df851ccc5d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80244) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 08:32:08.014462 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 1164/2334] 10.222.0.47 () {58 vars in 1232 bytes} [Thu Jul 21 08:32:07 2022] DELETE /identity/v3/projects/ddde32ff934748c487b54e47f33b4125 => generated 0 bytes in 116 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 08:32:08.019535 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9018f531-bcd0-4ea0-a8a8-f2bac623f970 None None] REQUEST_METHOD: `POST` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:32:08.019906 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9018f531-bcd0-4ea0-a8a8-f2bac623f970 None None] SCRIPT_NAME: `/identity` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:32:08.020203 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9018f531-bcd0-4ea0-a8a8-f2bac623f970 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:32:08.039493 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.auth.core [None req-9018f531-bcd0-4ea0-a8a8-f2bac623f970 None None] MFA Rules not processed for user `2dce40b8d74c41ec94cc36e6ff05d412`. Rule list: `[]` (Enabled: `True`). {{(pid=80245) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 08:32:08.040758 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.fernet_utils [None req-9018f531-bcd0-4ea0-a8a8-f2bac623f970 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80245) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 08:32:08.112711 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: [pid: 80245|app: 0|req: 1171/2335] 10.222.0.47 () {58 vars in 957 bytes} [Thu Jul 21 08:32:08 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 21 08:32:08.177723 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-934d602c-71e5-4e17-89c3-cbe24e2d9997 None None] REQUEST_METHOD: `POST` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:32:08.178009 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-934d602c-71e5-4e17-89c3-cbe24e2d9997 None None] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:32:08.178285 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-934d602c-71e5-4e17-89c3-cbe24e2d9997 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:32:08.197274 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.auth.core [None req-934d602c-71e5-4e17-89c3-cbe24e2d9997 None None] MFA Rules not processed for user `2dce40b8d74c41ec94cc36e6ff05d412`. Rule list: `[]` (Enabled: `True`). {{(pid=80244) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 08:32:08.198497 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.fernet_utils [None req-934d602c-71e5-4e17-89c3-cbe24e2d9997 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80244) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 08:32:08.203454 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9018f531-bcd0-4ea0-a8a8-f2bac623f970 None None] Authenticating user token {{(pid=80245) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:32:08.204833 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.fernet_utils [None req-9018f531-bcd0-4ea0-a8a8-f2bac623f970 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80245) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 08:32:08.206594 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 1165/2336] 10.222.0.47 () {58 vars in 957 bytes} [Thu Jul 21 08:32:08 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 21 08:32:08.210579 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-934d602c-71e5-4e17-89c3-cbe24e2d9997 None None] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:32:08.217426 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9018f531-bcd0-4ea0-a8a8-f2bac623f970 None None] Validating token access rules against request {{(pid=80245) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:32:08.219487 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-934d602c-71e5-4e17-89c3-cbe24e2d9997 None None] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:32:08.222186 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be94bae3-2bc3-4d66-9f95-35f1a1bfffc0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '42d15fc0515945acacbbffcf2356c7a5', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80245) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:32:08.223085 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be94bae3-2bc3-4d66-9f95-35f1a1bfffc0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:32:08.223514 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be94bae3-2bc3-4d66-9f95-35f1a1bfffc0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:32:08.223798 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be94bae3-2bc3-4d66-9f95-35f1a1bfffc0 admin admin] PATH_INFO: `/v3/users/9e83bb1f79f640d49cab6ca3249f37ab` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:32:08.224666 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0c4333d-49d0-45cd-ba4e-21ab0c95d42e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '42d15fc0515945acacbbffcf2356c7a5', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:32:08.225075 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be94bae3-2bc3-4d66-9f95-35f1a1bfffc0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=9e83bb1f79f640d49cab6ca3249f37ab)` {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:32:08.225308 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0c4333d-49d0-45cd-ba4e-21ab0c95d42e admin admin] REQUEST_METHOD: `GET` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:32:08.225586 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0c4333d-49d0-45cd-ba4e-21ab0c95d42e admin admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:32:08.225940 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0c4333d-49d0-45cd-ba4e-21ab0c95d42e admin admin] PATH_INFO: `/v3/domains` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:32:08.226609 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0c4333d-49d0-45cd-ba4e-21ab0c95d42e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80244) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 08:32:08.226910 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0c4333d-49d0-45cd-ba4e-21ab0c95d42e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:32:08.227027 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be94bae3-2bc3-4d66-9f95-35f1a1bfffc0 admin admin] RBAC: Authorization granted {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:32:08.228520 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0c4333d-49d0-45cd-ba4e-21ab0c95d42e admin admin] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:32:08.242818 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 1166/2337] 10.222.0.47 () {58 vars in 1152 bytes} [Thu Jul 21 08:32:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 21 08:32:08.249029 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0c4333d-49d0-45cd-ba4e-21ab0c95d42e admin admin] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:32:08.262608 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0c4333d-49d0-45cd-ba4e-21ab0c95d42e admin admin] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:32:08.268012 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2fef9cc-8f2c-45d7-9936-3da9f0ff665d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '42d15fc0515945acacbbffcf2356c7a5', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:32:08.268775 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2fef9cc-8f2c-45d7-9936-3da9f0ff665d admin admin] REQUEST_METHOD: `POST` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:32:08.269129 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2fef9cc-8f2c-45d7-9936-3da9f0ff665d admin admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:32:08.269470 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2fef9cc-8f2c-45d7-9936-3da9f0ff665d admin admin] PATH_INFO: `/v3/projects` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:32:08.270221 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2fef9cc-8f2c-45d7-9936-3da9f0ff665d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:32:08.272492 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2fef9cc-8f2c-45d7-9936-3da9f0ff665d admin admin] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:32:08.279544 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.notifications [None req-be94bae3-2bc3-4d66-9f95-35f1a1bfffc0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9e83bb1f79f640d49cab6ca3249f37ab'} {{(pid=80245) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 08:32:08.288088 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.notifications [None req-be94bae3-2bc3-4d66-9f95-35f1a1bfffc0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9e83bb1f79f640d49cab6ca3249f37ab'} {{(pid=80245) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 08:32:08.304186 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 1167/2338] 10.222.0.47 () {60 vars in 1152 bytes} [Thu Jul 21 08:32:08 2022] POST /identity/v3/projects => generated 371 bytes in 56 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 21 08:32:08.307456 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.notifications [None req-be94bae3-2bc3-4d66-9f95-35f1a1bfffc0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9e83bb1f79f640d49cab6ca3249f37ab'} {{(pid=80245) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 08:32:08.308402 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2fef9cc-8f2c-45d7-9936-3da9f0ff665d admin admin] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:32:08.315625 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.notifications [None req-be94bae3-2bc3-4d66-9f95-35f1a1bfffc0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9e83bb1f79f640d49cab6ca3249f37ab'} {{(pid=80245) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 08:32:08.316349 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.notifications [None req-be94bae3-2bc3-4d66-9f95-35f1a1bfffc0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9e83bb1f79f640d49cab6ca3249f37ab'} {{(pid=80245) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 08:32:08.322686 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: [pid: 80245|app: 0|req: 1172/2339] 10.222.0.47 () {58 vars in 1223 bytes} [Thu Jul 21 08:32:08 2022] DELETE /identity/v3/users/9e83bb1f79f640d49cab6ca3249f37ab => generated 0 bytes in 120 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 21 08:32:08.326950 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4ed30f8-f0ee-4c36-9350-6678bfd45fb7 None None] REQUEST_METHOD: `POST` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:32:08.327244 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4ed30f8-f0ee-4c36-9350-6678bfd45fb7 None None] SCRIPT_NAME: `/identity` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:32:08.327568 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4ed30f8-f0ee-4c36-9350-6678bfd45fb7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:32:08.338882 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG dbcounter [-] [80244] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80244) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:32:08.345059 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.auth.core [None req-b4ed30f8-f0ee-4c36-9350-6678bfd45fb7 None None] MFA Rules not processed for user `2dce40b8d74c41ec94cc36e6ff05d412`. Rule list: `[]` (Enabled: `True`). {{(pid=80245) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 08:32:08.346287 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.fernet_utils [None req-b4ed30f8-f0ee-4c36-9350-6678bfd45fb7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80245) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 08:32:08.357996 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG dbcounter [-] [80245] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80245) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:32:08.377306 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2fef9cc-8f2c-45d7-9936-3da9f0ff665d admin admin] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:32:08.411607 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: [pid: 80245|app: 0|req: 1173/2340] 10.222.0.47 () {58 vars in 957 bytes} [Thu Jul 21 08:32:08 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 21 08:32:08.421468 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4ed30f8-f0ee-4c36-9350-6678bfd45fb7 None None] Authenticating user token {{(pid=80245) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:32:08.422848 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.fernet_utils [None req-b4ed30f8-f0ee-4c36-9350-6678bfd45fb7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80245) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 08:32:08.435579 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95cda522-8b8a-480d-89db-9298ac3c3ef0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '42d15fc0515945acacbbffcf2356c7a5', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:32:08.436953 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95cda522-8b8a-480d-89db-9298ac3c3ef0 admin admin] REQUEST_METHOD: `POST` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:32:08.437223 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95cda522-8b8a-480d-89db-9298ac3c3ef0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:32:08.437498 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95cda522-8b8a-480d-89db-9298ac3c3ef0 admin admin] PATH_INFO: `/v3/users` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:32:08.438190 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95cda522-8b8a-480d-89db-9298ac3c3ef0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:32:08.440281 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95cda522-8b8a-480d-89db-9298ac3c3ef0 admin admin] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:32:08.465748 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 1168/2341] 10.222.0.47 () {60 vars in 1143 bytes} [Thu Jul 21 08:32:08 2022] POST /identity/v3/users => generated 286 bytes in 158 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 08:32:08.469887 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95cda522-8b8a-480d-89db-9298ac3c3ef0 admin admin] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:32:08.472852 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.fernet_utils [None req-95cda522-8b8a-480d-89db-9298ac3c3ef0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80244) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 08:32:08.489889 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95cda522-8b8a-480d-89db-9298ac3c3ef0 admin admin] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:32:08.495475 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4ed30f8-f0ee-4c36-9350-6678bfd45fb7 None None] Validating token access rules against request {{(pid=80245) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:32:08.496045 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0f47b4c-d1fc-4447-9255-90f7068bf046 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '42d15fc0515945acacbbffcf2356c7a5', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:32:08.496742 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0f47b4c-d1fc-4447-9255-90f7068bf046 admin admin] REQUEST_METHOD: `GET` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:32:08.497029 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0f47b4c-d1fc-4447-9255-90f7068bf046 admin admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:32:08.497306 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0f47b4c-d1fc-4447-9255-90f7068bf046 admin admin] PATH_INFO: `/v3/roles` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:32:08.498207 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0f47b4c-d1fc-4447-9255-90f7068bf046 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:32:08.499901 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0f47b4c-d1fc-4447-9255-90f7068bf046 admin admin] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:32:08.503067 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd9b72bd-28bb-4c6c-a30f-524f8937b2d0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20ca3c3bd3774691b3374cfbfec6e494', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4faef13aa50e4e36b6c265fb1398fa14', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80245) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:32:08.504268 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd9b72bd-28bb-4c6c-a30f-524f8937b2d0 service neutron] REQUEST_METHOD: `GET` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:32:08.504558 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd9b72bd-28bb-4c6c-a30f-524f8937b2d0 service neutron] SCRIPT_NAME: `/identity` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:32:08.504841 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd9b72bd-28bb-4c6c-a30f-524f8937b2d0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:32:08.511727 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 1169/2342] 10.222.0.47 () {58 vars in 1121 bytes} [Thu Jul 21 08:32:08 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 21 08:32:08.515740 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0f47b4c-d1fc-4447-9255-90f7068bf046 admin admin] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:32:08.527684 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd9b72bd-28bb-4c6c-a30f-524f8937b2d0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:32:08.529070 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0f47b4c-d1fc-4447-9255-90f7068bf046 admin admin] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:32:08.529451 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd9b72bd-28bb-4c6c-a30f-524f8937b2d0 service neutron] RBAC: Authorization granted {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:32:08.534718 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39ca12bd-62cb-4a81-8ee0-620c21f85c85 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '42d15fc0515945acacbbffcf2356c7a5', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:32:08.536137 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39ca12bd-62cb-4a81-8ee0-620c21f85c85 admin admin] REQUEST_METHOD: `GET` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:32:08.536499 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39ca12bd-62cb-4a81-8ee0-620c21f85c85 admin admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:32:08.536849 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39ca12bd-62cb-4a81-8ee0-620c21f85c85 admin admin] PATH_INFO: `/v3/roles` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:32:08.537790 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39ca12bd-62cb-4a81-8ee0-620c21f85c85 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:32:08.539513 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39ca12bd-62cb-4a81-8ee0-620c21f85c85 admin admin] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:32:08.540833 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: [pid: 80245|app: 0|req: 1174/2343] 10.222.0.47 () {60 vars in 1365 bytes} [Thu Jul 21 08:32:08 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 120 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 21 08:32:08.550831 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 1170/2344] 10.222.0.47 () {58 vars in 1121 bytes} [Thu Jul 21 08:32:08 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 08:32:08.554700 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd9b72bd-28bb-4c6c-a30f-524f8937b2d0 service neutron] Authenticating user token {{(pid=80245) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:32:08.564772 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd9b72bd-28bb-4c6c-a30f-524f8937b2d0 service neutron] Validating token access rules against request {{(pid=80245) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:32:08.569548 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81a69b07-4bb7-4e6d-94dc-1b54939db578 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '42d15fc0515945acacbbffcf2356c7a5', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80245) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:32:08.570441 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81a69b07-4bb7-4e6d-94dc-1b54939db578 admin admin] REQUEST_METHOD: `PUT` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:32:08.570726 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81a69b07-4bb7-4e6d-94dc-1b54939db578 admin admin] SCRIPT_NAME: `/identity` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:32:08.571004 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81a69b07-4bb7-4e6d-94dc-1b54939db578 admin admin] PATH_INFO: `/v3/projects/16dc806bd5c842258c652409fa4bc353/users/1b3e7b164a7a49a9a7318072f5ba523c/roles/3a074738221e4601b5205715ba87d240` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:32:08.586874 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81a69b07-4bb7-4e6d-94dc-1b54939db578 admin admin] RBAC: Authorizing `identity:create_grant(project_id=16dc806bd5c842258c652409fa4bc353, user_id=1b3e7b164a7a49a9a7318072f5ba523c, role_id=3a074738221e4601b5205715ba87d240)` {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:32:08.590292 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81a69b07-4bb7-4e6d-94dc-1b54939db578 admin admin] RBAC: Authorization granted {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:32:08.606846 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: [pid: 80245|app: 0|req: 1175/2345] 10.222.0.47 () {60 vars in 1482 bytes} [Thu Jul 21 08:32:08 2022] PUT /identity/v3/projects/16dc806bd5c842258c652409fa4bc353/users/1b3e7b164a7a49a9a7318072f5ba523c/roles/3a074738221e4601b5205715ba87d240 => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 21 08:32:08.617224 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-528c4941-daf1-4726-9d42-31af4ed1abce None None] REQUEST_METHOD: `POST` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:32:08.617509 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-528c4941-daf1-4726-9d42-31af4ed1abce None None] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:32:08.617788 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-528c4941-daf1-4726-9d42-31af4ed1abce None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:32:08.645812 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.auth.core [None req-528c4941-daf1-4726-9d42-31af4ed1abce None None] MFA Rules not processed for user `1b3e7b164a7a49a9a7318072f5ba523c`. Rule list: `[]` (Enabled: `True`). {{(pid=80244) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 08:32:08.647176 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.fernet_utils [None req-528c4941-daf1-4726-9d42-31af4ed1abce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80244) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 08:32:08.737337 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 1171/2346] 10.222.0.47 () {58 vars in 957 bytes} [Thu Jul 21 08:32:08 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 08:32:08.748528 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81a69b07-4bb7-4e6d-94dc-1b54939db578 admin admin] Authenticating user token {{(pid=80245) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:32:08.750730 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.fernet_utils [None req-81a69b07-4bb7-4e6d-94dc-1b54939db578 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80245) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 08:32:08.818440 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81a69b07-4bb7-4e6d-94dc-1b54939db578 admin admin] Validating token access rules against request {{(pid=80245) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:32:08.822200 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2b5d9ce-d952-4312-a481-9b9cfdc2515b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4f40410b7674687ab5362ac982248ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4faef13aa50e4e36b6c265fb1398fa14', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80245) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:32:08.823408 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2b5d9ce-d952-4312-a481-9b9cfdc2515b service cinder] REQUEST_METHOD: `GET` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:32:08.823698 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2b5d9ce-d952-4312-a481-9b9cfdc2515b service cinder] SCRIPT_NAME: `/identity` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:32:08.823978 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2b5d9ce-d952-4312-a481-9b9cfdc2515b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:32:08.832651 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2b5d9ce-d952-4312-a481-9b9cfdc2515b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:32:08.834084 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2b5d9ce-d952-4312-a481-9b9cfdc2515b service cinder] RBAC: Authorization granted {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:32:08.843675 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: [pid: 80245|app: 0|req: 1176/2347] 10.222.0.47 () {60 vars in 1365 bytes} [Thu Jul 21 08:32:08 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 08:32:08.939956 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-528c4941-daf1-4726-9d42-31af4ed1abce None None] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:32:08.941513 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.fernet_utils [None req-528c4941-daf1-4726-9d42-31af4ed1abce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80244) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 08:32:09.041275 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG dbcounter [-] [80244] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80244) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:32:09.061064 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2b5d9ce-d952-4312-a481-9b9cfdc2515b service cinder] Authenticating user token {{(pid=80245) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:32:09.062903 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.fernet_utils [None req-e2b5d9ce-d952-4312-a481-9b9cfdc2515b service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80245) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 08:32:09.141823 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-528c4941-daf1-4726-9d42-31af4ed1abce None None] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:32:09.154223 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d560039f-44fc-4380-86d7-9570e8055e0a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '42d15fc0515945acacbbffcf2356c7a5', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:32:09.155548 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d560039f-44fc-4380-86d7-9570e8055e0a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:32:09.156164 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d560039f-44fc-4380-86d7-9570e8055e0a admin admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:32:09.156582 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG dbcounter [-] [80245] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80245) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:32:09.156841 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d560039f-44fc-4380-86d7-9570e8055e0a admin admin] PATH_INFO: `/v3/projects/06549f7a0a214370b4fe64ae3a3c1b96` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:32:09.158409 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d560039f-44fc-4380-86d7-9570e8055e0a admin admin] RBAC: Authorizing `identity:delete_project(project_id=06549f7a0a214370b4fe64ae3a3c1b96)` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:32:09.160821 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d560039f-44fc-4380-86d7-9570e8055e0a admin admin] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:32:09.185812 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2b5d9ce-d952-4312-a481-9b9cfdc2515b service cinder] Validating token access rules against request {{(pid=80245) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:32:09.191917 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f81aca4f-e1ac-4196-a764-c5078202df12 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '42d15fc0515945acacbbffcf2356c7a5', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80245) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:32:09.192994 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f81aca4f-e1ac-4196-a764-c5078202df12 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:32:09.193283 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f81aca4f-e1ac-4196-a764-c5078202df12 admin admin] SCRIPT_NAME: `/identity` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:32:09.193562 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f81aca4f-e1ac-4196-a764-c5078202df12 admin admin] PATH_INFO: `/v3/users/253aa2f7704e48b59ce4cf39d6ab50ca` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:32:09.195014 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f81aca4f-e1ac-4196-a764-c5078202df12 admin admin] RBAC: Authorizing `identity:delete_user(user_id=253aa2f7704e48b59ce4cf39d6ab50ca)` {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:32:09.197430 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f81aca4f-e1ac-4196-a764-c5078202df12 admin admin] RBAC: Authorization granted {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:32:09.222624 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.notifications [None req-d560039f-44fc-4380-86d7-9570e8055e0a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '06549f7a0a214370b4fe64ae3a3c1b96', 'request_id': 'req-d560039f-44fc-4380-86d7-9570e8055e0a'} {{(pid=80244) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 08:32:09.235428 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.notifications [None req-d560039f-44fc-4380-86d7-9570e8055e0a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '06549f7a0a214370b4fe64ae3a3c1b96', 'request_id': 'req-d560039f-44fc-4380-86d7-9570e8055e0a'} {{(pid=80244) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 08:32:09.245787 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.notifications [None req-d560039f-44fc-4380-86d7-9570e8055e0a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '06549f7a0a214370b4fe64ae3a3c1b96', 'request_id': 'req-d560039f-44fc-4380-86d7-9570e8055e0a'} {{(pid=80244) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 08:32:09.261971 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.notifications [None req-d560039f-44fc-4380-86d7-9570e8055e0a admin admin] The token cache is being invalidate because project 06549f7a0a214370b4fe64ae3a3c1b96 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80244) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 21 08:32:09.262281 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.notifications [None req-d560039f-44fc-4380-86d7-9570e8055e0a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80244) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 08:32:09.262928 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.notifications [None req-d560039f-44fc-4380-86d7-9570e8055e0a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80244) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 08:32:09.264988 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 1172/2348] 10.222.0.47 () {58 vars in 1232 bytes} [Thu Jul 21 08:32:08 2022] DELETE /identity/v3/projects/06549f7a0a214370b4fe64ae3a3c1b96 => generated 0 bytes in 326 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 08:32:09.265127 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.notifications [None req-f81aca4f-e1ac-4196-a764-c5078202df12 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '253aa2f7704e48b59ce4cf39d6ab50ca'} {{(pid=80245) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 08:32:09.268965 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d560039f-44fc-4380-86d7-9570e8055e0a admin admin] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:32:09.270386 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.fernet_utils [None req-d560039f-44fc-4380-86d7-9570e8055e0a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80244) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 08:32:09.274141 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.notifications [None req-f81aca4f-e1ac-4196-a764-c5078202df12 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '253aa2f7704e48b59ce4cf39d6ab50ca'} {{(pid=80245) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 08:32:09.288283 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.notifications [None req-f81aca4f-e1ac-4196-a764-c5078202df12 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '253aa2f7704e48b59ce4cf39d6ab50ca'} {{(pid=80245) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 08:32:09.297234 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.notifications [None req-f81aca4f-e1ac-4196-a764-c5078202df12 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '253aa2f7704e48b59ce4cf39d6ab50ca'} {{(pid=80245) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 08:32:09.297890 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.notifications [None req-f81aca4f-e1ac-4196-a764-c5078202df12 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '253aa2f7704e48b59ce4cf39d6ab50ca'} {{(pid=80245) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 08:32:09.304710 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: [pid: 80245|app: 0|req: 1177/2349] 10.222.0.47 () {58 vars in 1223 bytes} [Thu Jul 21 08:32:09 2022] DELETE /identity/v3/users/253aa2f7704e48b59ce4cf39d6ab50ca => generated 0 bytes in 245 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 08:32:09.310547 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5489685-75c1-47b2-95cb-bc8b50cbcb6f None None] REQUEST_METHOD: `POST` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:32:09.310837 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5489685-75c1-47b2-95cb-bc8b50cbcb6f None None] SCRIPT_NAME: `/identity` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:32:09.311118 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5489685-75c1-47b2-95cb-bc8b50cbcb6f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:32:09.343125 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.auth.core [None req-e5489685-75c1-47b2-95cb-bc8b50cbcb6f None None] MFA Rules not processed for user `2dce40b8d74c41ec94cc36e6ff05d412`. Rule list: `[]` (Enabled: `True`). {{(pid=80245) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 08:32:09.344663 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.fernet_utils [None req-e5489685-75c1-47b2-95cb-bc8b50cbcb6f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80245) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 08:32:09.362362 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d560039f-44fc-4380-86d7-9570e8055e0a admin admin] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:32:09.427531 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: [pid: 80245|app: 0|req: 1178/2350] 10.222.0.47 () {58 vars in 957 bytes} [Thu Jul 21 08:32:09 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 08:32:09.432536 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd827ab4-61b3-4209-a622-0aa1d75f2dc8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '42d15fc0515945acacbbffcf2356c7a5', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:32:09.433418 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd827ab4-61b3-4209-a622-0aa1d75f2dc8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:32:09.433699 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd827ab4-61b3-4209-a622-0aa1d75f2dc8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:32:09.433972 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd827ab4-61b3-4209-a622-0aa1d75f2dc8 admin admin] PATH_INFO: `/v3/users/a1c1076a63724f84b6bf3eb9eb51ece7` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:32:09.435183 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd827ab4-61b3-4209-a622-0aa1d75f2dc8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a1c1076a63724f84b6bf3eb9eb51ece7)` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:32:09.436902 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5489685-75c1-47b2-95cb-bc8b50cbcb6f None None] Authenticating user token {{(pid=80245) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:32:09.437262 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd827ab4-61b3-4209-a622-0aa1d75f2dc8 admin admin] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:32:09.438336 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.fernet_utils [None req-e5489685-75c1-47b2-95cb-bc8b50cbcb6f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80245) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 08:32:09.448651 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG dbcounter [-] [80244] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80244) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:32:09.488049 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.notifications [None req-fd827ab4-61b3-4209-a622-0aa1d75f2dc8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a1c1076a63724f84b6bf3eb9eb51ece7'} {{(pid=80244) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 08:32:09.496100 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.notifications [None req-fd827ab4-61b3-4209-a622-0aa1d75f2dc8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a1c1076a63724f84b6bf3eb9eb51ece7'} {{(pid=80244) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 08:32:09.502240 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5489685-75c1-47b2-95cb-bc8b50cbcb6f None None] Validating token access rules against request {{(pid=80245) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:32:09.506319 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-216ab6e9-1c35-42a9-9425-afe6b4e71bac service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20ca3c3bd3774691b3374cfbfec6e494', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4faef13aa50e4e36b6c265fb1398fa14', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80245) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:32:09.507116 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-216ab6e9-1c35-42a9-9425-afe6b4e71bac service neutron] REQUEST_METHOD: `GET` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:32:09.507508 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-216ab6e9-1c35-42a9-9425-afe6b4e71bac service neutron] SCRIPT_NAME: `/identity` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:32:09.507856 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-216ab6e9-1c35-42a9-9425-afe6b4e71bac service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:32:09.510762 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.notifications [None req-fd827ab4-61b3-4209-a622-0aa1d75f2dc8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a1c1076a63724f84b6bf3eb9eb51ece7'} {{(pid=80244) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 08:32:09.511839 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.notifications [None req-fd827ab4-61b3-4209-a622-0aa1d75f2dc8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a1c1076a63724f84b6bf3eb9eb51ece7'} {{(pid=80244) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 08:32:09.519415 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-216ab6e9-1c35-42a9-9425-afe6b4e71bac service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:32:09.520340 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.notifications [None req-fd827ab4-61b3-4209-a622-0aa1d75f2dc8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a1c1076a63724f84b6bf3eb9eb51ece7'} {{(pid=80244) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 08:32:09.521078 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-216ab6e9-1c35-42a9-9425-afe6b4e71bac service neutron] RBAC: Authorization granted {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:32:09.526601 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 1173/2351] 10.222.0.47 () {58 vars in 1223 bytes} [Thu Jul 21 08:32:09 2022] DELETE /identity/v3/users/a1c1076a63724f84b6bf3eb9eb51ece7 => generated 0 bytes in 258 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 21 08:32:09.531404 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: [pid: 80245|app: 0|req: 1179/2352] 10.222.0.47 () {60 vars in 1365 bytes} [Thu Jul 21 08:32:09 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 21 08:32:09.941267 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd827ab4-61b3-4209-a622-0aa1d75f2dc8 admin admin] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:32:09.942761 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.fernet_utils [None req-fd827ab4-61b3-4209-a622-0aa1d75f2dc8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80244) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 08:32:10.017670 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-216ab6e9-1c35-42a9-9425-afe6b4e71bac service neutron] Authenticating user token {{(pid=80245) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:32:10.019250 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.fernet_utils [None req-216ab6e9-1c35-42a9-9425-afe6b4e71bac service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80245) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 08:32:10.028950 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG dbcounter [-] [80245] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80245) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:32:10.035098 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-216ab6e9-1c35-42a9-9425-afe6b4e71bac service neutron] Validating token access rules against request {{(pid=80245) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:32:10.042609 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc15ccef-0028-4bf0-a821-71342ad44f7d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '42d15fc0515945acacbbffcf2356c7a5', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80245) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:32:10.043934 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc15ccef-0028-4bf0-a821-71342ad44f7d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:32:10.044344 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc15ccef-0028-4bf0-a821-71342ad44f7d admin admin] SCRIPT_NAME: `/identity` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:32:10.044638 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc15ccef-0028-4bf0-a821-71342ad44f7d admin admin] PATH_INFO: `/v3/projects/1154d87830ea4f85b7236414b05c8e46` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:32:10.045600 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd827ab4-61b3-4209-a622-0aa1d75f2dc8 admin admin] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:32:10.046466 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc15ccef-0028-4bf0-a821-71342ad44f7d admin admin] RBAC: Authorizing `identity:delete_project(project_id=1154d87830ea4f85b7236414b05c8e46)` {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:32:10.048823 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc15ccef-0028-4bf0-a821-71342ad44f7d admin admin] RBAC: Authorization granted {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:32:10.051548 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92118853-9bbf-4b03-be25-7ef3b14c4ddb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '42d15fc0515945acacbbffcf2356c7a5', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:32:10.052808 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92118853-9bbf-4b03-be25-7ef3b14c4ddb admin admin] REQUEST_METHOD: `DELETE` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:32:10.053095 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92118853-9bbf-4b03-be25-7ef3b14c4ddb admin admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:32:10.053363 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92118853-9bbf-4b03-be25-7ef3b14c4ddb admin admin] PATH_INFO: `/v3/projects/93539ede044a41b696986aaf5b86da65` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:32:10.054793 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92118853-9bbf-4b03-be25-7ef3b14c4ddb admin admin] RBAC: Authorizing `identity:delete_project(project_id=93539ede044a41b696986aaf5b86da65)` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:32:10.056717 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92118853-9bbf-4b03-be25-7ef3b14c4ddb admin admin] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:32:10.099984 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.notifications [None req-dc15ccef-0028-4bf0-a821-71342ad44f7d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1154d87830ea4f85b7236414b05c8e46', 'request_id': 'req-dc15ccef-0028-4bf0-a821-71342ad44f7d'} {{(pid=80245) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 08:32:10.107244 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.notifications [None req-dc15ccef-0028-4bf0-a821-71342ad44f7d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1154d87830ea4f85b7236414b05c8e46', 'request_id': 'req-dc15ccef-0028-4bf0-a821-71342ad44f7d'} {{(pid=80245) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 08:32:10.109671 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.notifications [None req-92118853-9bbf-4b03-be25-7ef3b14c4ddb admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '93539ede044a41b696986aaf5b86da65', 'request_id': 'req-92118853-9bbf-4b03-be25-7ef3b14c4ddb'} {{(pid=80244) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 08:32:10.113088 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.notifications [None req-dc15ccef-0028-4bf0-a821-71342ad44f7d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1154d87830ea4f85b7236414b05c8e46', 'request_id': 'req-dc15ccef-0028-4bf0-a821-71342ad44f7d'} {{(pid=80245) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 08:32:10.118182 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.notifications [None req-92118853-9bbf-4b03-be25-7ef3b14c4ddb admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '93539ede044a41b696986aaf5b86da65', 'request_id': 'req-92118853-9bbf-4b03-be25-7ef3b14c4ddb'} {{(pid=80244) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 08:32:10.124725 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.notifications [None req-92118853-9bbf-4b03-be25-7ef3b14c4ddb admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '93539ede044a41b696986aaf5b86da65', 'request_id': 'req-92118853-9bbf-4b03-be25-7ef3b14c4ddb'} {{(pid=80244) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 08:32:10.126217 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.notifications [None req-dc15ccef-0028-4bf0-a821-71342ad44f7d admin admin] The token cache is being invalidate because project 1154d87830ea4f85b7236414b05c8e46 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80245) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 21 08:32:10.126721 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.notifications [None req-dc15ccef-0028-4bf0-a821-71342ad44f7d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80245) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 08:32:10.127570 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.notifications [None req-dc15ccef-0028-4bf0-a821-71342ad44f7d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80245) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 08:32:10.129615 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: [pid: 80245|app: 0|req: 1180/2353] 10.222.0.47 () {58 vars in 1232 bytes} [Thu Jul 21 08:32:10 2022] DELETE /identity/v3/projects/1154d87830ea4f85b7236414b05c8e46 => generated 0 bytes in 113 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 21 08:32:10.134508 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fec61f6-5185-461f-857c-47abcb5fe6e4 None None] REQUEST_METHOD: `POST` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:32:10.135586 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fec61f6-5185-461f-857c-47abcb5fe6e4 None None] SCRIPT_NAME: `/identity` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:32:10.136090 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fec61f6-5185-461f-857c-47abcb5fe6e4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:32:10.138833 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.notifications [None req-92118853-9bbf-4b03-be25-7ef3b14c4ddb admin admin] The token cache is being invalidate because project 93539ede044a41b696986aaf5b86da65 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80244) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 21 08:32:10.139128 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.notifications [None req-92118853-9bbf-4b03-be25-7ef3b14c4ddb admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80244) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 08:32:10.140865 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.notifications [None req-92118853-9bbf-4b03-be25-7ef3b14c4ddb admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80244) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 08:32:10.146611 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 1174/2354] 10.222.0.47 () {58 vars in 1232 bytes} [Thu Jul 21 08:32:09 2022] DELETE /identity/v3/projects/93539ede044a41b696986aaf5b86da65 => generated 0 bytes in 202 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 08:32:10.151054 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-376418d4-a333-4e0d-99e9-dbb952b334dc None None] REQUEST_METHOD: `POST` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:32:10.151863 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-376418d4-a333-4e0d-99e9-dbb952b334dc None None] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:32:10.152177 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-376418d4-a333-4e0d-99e9-dbb952b334dc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:32:10.172595 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.auth.core [None req-7fec61f6-5185-461f-857c-47abcb5fe6e4 None None] MFA Rules not processed for user `2dce40b8d74c41ec94cc36e6ff05d412`. Rule list: `[]` (Enabled: `True`). {{(pid=80245) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 08:32:10.174400 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.fernet_utils [None req-7fec61f6-5185-461f-857c-47abcb5fe6e4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80245) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 08:32:10.174969 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.auth.core [None req-376418d4-a333-4e0d-99e9-dbb952b334dc None None] MFA Rules not processed for user `2dce40b8d74c41ec94cc36e6ff05d412`. Rule list: `[]` (Enabled: `True`). {{(pid=80244) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 08:32:10.176250 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.fernet_utils [None req-376418d4-a333-4e0d-99e9-dbb952b334dc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80244) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 08:32:10.190088 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG dbcounter [-] [80244] Writing DB stats keystone:DELETE=17,keystone:SELECT=81,keystone:INSERT=2 {{(pid=80244) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:32:10.259007 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 1175/2355] 10.222.0.47 () {58 vars in 957 bytes} [Thu Jul 21 08:32:10 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 08:32:10.268546 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a5a1d1e-e695-4184-ba62-ee29230b4eca None None] REQUEST_METHOD: `POST` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:32:10.268879 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a5a1d1e-e695-4184-ba62-ee29230b4eca None None] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:32:10.269249 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a5a1d1e-e695-4184-ba62-ee29230b4eca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:32:10.280653 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: [pid: 80245|app: 0|req: 1181/2356] 10.222.0.47 () {58 vars in 957 bytes} [Thu Jul 21 08:32:10 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 147 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 21 08:32:10.288984 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1740d5e6-f5d7-47ef-9c77-e3ece624ffa5 None None] REQUEST_METHOD: `POST` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:32:10.289370 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1740d5e6-f5d7-47ef-9c77-e3ece624ffa5 None None] SCRIPT_NAME: `/identity` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:32:10.289717 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1740d5e6-f5d7-47ef-9c77-e3ece624ffa5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:32:10.300302 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.auth.core [None req-4a5a1d1e-e695-4184-ba62-ee29230b4eca None None] MFA Rules not processed for user `2dce40b8d74c41ec94cc36e6ff05d412`. Rule list: `[]` (Enabled: `True`). {{(pid=80244) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 08:32:10.301581 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.fernet_utils [None req-4a5a1d1e-e695-4184-ba62-ee29230b4eca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80244) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 08:32:10.318808 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 1176/2357] 10.222.0.47 () {58 vars in 957 bytes} [Thu Jul 21 08:32:10 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 52 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 21 08:32:10.320411 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.auth.core [None req-1740d5e6-f5d7-47ef-9c77-e3ece624ffa5 None None] MFA Rules not processed for user `2dce40b8d74c41ec94cc36e6ff05d412`. Rule list: `[]` (Enabled: `True`). {{(pid=80245) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 08:32:10.321793 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.fernet_utils [None req-1740d5e6-f5d7-47ef-9c77-e3ece624ffa5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80245) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 08:32:10.323996 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a5a1d1e-e695-4184-ba62-ee29230b4eca None None] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:32:10.337599 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: [pid: 80245|app: 0|req: 1182/2358] 10.222.0.47 () {58 vars in 957 bytes} [Thu Jul 21 08:32:10 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 49 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 21 08:32:10.337955 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1740d5e6-f5d7-47ef-9c77-e3ece624ffa5 None None] Authenticating user token {{(pid=80245) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:32:10.345777 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a5a1d1e-e695-4184-ba62-ee29230b4eca None None] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:32:10.347865 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1740d5e6-f5d7-47ef-9c77-e3ece624ffa5 None None] Validating token access rules against request {{(pid=80245) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:32:10.351287 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3026dc5e-6432-49c9-a402-705bcc609c3a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '42d15fc0515945acacbbffcf2356c7a5', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:32:10.351935 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3026dc5e-6432-49c9-a402-705bcc609c3a admin admin] REQUEST_METHOD: `GET` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:32:10.352223 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3026dc5e-6432-49c9-a402-705bcc609c3a admin admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:32:10.352501 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3026dc5e-6432-49c9-a402-705bcc609c3a admin admin] PATH_INFO: `/v3/domains` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:32:10.353240 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3026dc5e-6432-49c9-a402-705bcc609c3a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80244) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 08:32:10.353728 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3026dc5e-6432-49c9-a402-705bcc609c3a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:32:10.353874 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c9bf190-23c4-4b73-a975-a06eb571983b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '42d15fc0515945acacbbffcf2356c7a5', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80245) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:32:10.354672 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c9bf190-23c4-4b73-a975-a06eb571983b admin admin] REQUEST_METHOD: `GET` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:32:10.355054 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c9bf190-23c4-4b73-a975-a06eb571983b admin admin] SCRIPT_NAME: `/identity` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:32:10.355414 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c9bf190-23c4-4b73-a975-a06eb571983b admin admin] PATH_INFO: `/v3/domains` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:32:10.355631 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3026dc5e-6432-49c9-a402-705bcc609c3a admin admin] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:32:10.356289 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c9bf190-23c4-4b73-a975-a06eb571983b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80245) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 08:32:10.356664 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c9bf190-23c4-4b73-a975-a06eb571983b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:32:10.358581 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c9bf190-23c4-4b73-a975-a06eb571983b admin admin] RBAC: Authorization granted {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:32:10.373289 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 1177/2359] 10.222.0.47 () {58 vars in 1152 bytes} [Thu Jul 21 08:32:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 51 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 08:32:10.377755 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3026dc5e-6432-49c9-a402-705bcc609c3a admin admin] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:32:10.378869 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: [pid: 80245|app: 0|req: 1183/2360] 10.222.0.47 () {58 vars in 1152 bytes} [Thu Jul 21 08:32:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 08:32:10.393138 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c9bf190-23c4-4b73-a975-a06eb571983b admin admin] Authenticating user token {{(pid=80245) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:32:10.402078 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3026dc5e-6432-49c9-a402-705bcc609c3a admin admin] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:32:10.407278 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c9bf190-23c4-4b73-a975-a06eb571983b admin admin] Validating token access rules against request {{(pid=80245) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:32:10.413705 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea202baa-c8f6-4969-9004-084e6518fe99 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '42d15fc0515945acacbbffcf2356c7a5', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:32:10.414417 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea202baa-c8f6-4969-9004-084e6518fe99 admin admin] REQUEST_METHOD: `POST` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:32:10.414568 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b97bbb81-dfda-48e6-b92c-71e0fdbf7131 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '42d15fc0515945acacbbffcf2356c7a5', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80245) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:32:10.414714 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea202baa-c8f6-4969-9004-084e6518fe99 admin admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:32:10.414922 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea202baa-c8f6-4969-9004-084e6518fe99 admin admin] PATH_INFO: `/v3/projects` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:32:10.415536 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b97bbb81-dfda-48e6-b92c-71e0fdbf7131 admin admin] REQUEST_METHOD: `POST` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:32:10.415663 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea202baa-c8f6-4969-9004-084e6518fe99 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:32:10.415839 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b97bbb81-dfda-48e6-b92c-71e0fdbf7131 admin admin] SCRIPT_NAME: `/identity` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:32:10.416136 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b97bbb81-dfda-48e6-b92c-71e0fdbf7131 admin admin] PATH_INFO: `/v3/projects` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:32:10.417094 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b97bbb81-dfda-48e6-b92c-71e0fdbf7131 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:32:10.417605 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea202baa-c8f6-4969-9004-084e6518fe99 admin admin] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:32:10.418997 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b97bbb81-dfda-48e6-b92c-71e0fdbf7131 admin admin] RBAC: Authorization granted {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:32:10.451204 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 1178/2361] 10.222.0.47 () {60 vars in 1152 bytes} [Thu Jul 21 08:32:10 2022] POST /identity/v3/projects => generated 375 bytes in 74 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 08:32:10.454187 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: [pid: 80245|app: 0|req: 1184/2362] 10.222.0.47 () {60 vars in 1152 bytes} [Thu Jul 21 08:32:10 2022] POST /identity/v3/projects => generated 375 bytes in 74 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 08:32:10.458500 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b97bbb81-dfda-48e6-b92c-71e0fdbf7131 admin admin] Authenticating user token {{(pid=80245) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:32:10.459617 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea202baa-c8f6-4969-9004-084e6518fe99 admin admin] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:32:10.525746 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG dbcounter [-] [80245] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80245) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:32:10.554013 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b97bbb81-dfda-48e6-b92c-71e0fdbf7131 admin admin] Validating token access rules against request {{(pid=80245) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:32:10.558281 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea202baa-c8f6-4969-9004-084e6518fe99 admin admin] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:32:10.559764 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af7a3c54-de85-4813-9c2b-a473053eba4e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '42d15fc0515945acacbbffcf2356c7a5', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80245) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:32:10.560614 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af7a3c54-de85-4813-9c2b-a473053eba4e admin admin] REQUEST_METHOD: `POST` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:32:10.560905 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af7a3c54-de85-4813-9c2b-a473053eba4e admin admin] SCRIPT_NAME: `/identity` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:32:10.561189 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af7a3c54-de85-4813-9c2b-a473053eba4e admin admin] PATH_INFO: `/v3/users` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:32:10.561881 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af7a3c54-de85-4813-9c2b-a473053eba4e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:32:10.563399 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc8ba73e-b283-48a5-8f45-b9915ecbe8a2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '42d15fc0515945acacbbffcf2356c7a5', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:32:10.563834 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af7a3c54-de85-4813-9c2b-a473053eba4e admin admin] RBAC: Authorization granted {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:32:10.564132 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc8ba73e-b283-48a5-8f45-b9915ecbe8a2 admin admin] REQUEST_METHOD: `POST` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:32:10.564415 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc8ba73e-b283-48a5-8f45-b9915ecbe8a2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:32:10.564697 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc8ba73e-b283-48a5-8f45-b9915ecbe8a2 admin admin] PATH_INFO: `/v3/users` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:32:10.565395 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc8ba73e-b283-48a5-8f45-b9915ecbe8a2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:32:10.567464 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc8ba73e-b283-48a5-8f45-b9915ecbe8a2 admin admin] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:32:10.591952 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: [pid: 80245|app: 0|req: 1185/2363] 10.222.0.47 () {60 vars in 1143 bytes} [Thu Jul 21 08:32:10 2022] POST /identity/v3/users => generated 288 bytes in 131 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 08:32:10.593092 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af7a3c54-de85-4813-9c2b-a473053eba4e admin admin] Authenticating user token {{(pid=80245) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:32:10.593208 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 1179/2364] 10.222.0.47 () {60 vars in 1143 bytes} [Thu Jul 21 08:32:10 2022] POST /identity/v3/users => generated 288 bytes in 135 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 08:32:10.597644 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc8ba73e-b283-48a5-8f45-b9915ecbe8a2 admin admin] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:32:10.604450 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af7a3c54-de85-4813-9c2b-a473053eba4e admin admin] Validating token access rules against request {{(pid=80245) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:32:10.608708 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc8ba73e-b283-48a5-8f45-b9915ecbe8a2 admin admin] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:32:10.610850 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47d90bf8-35fb-455a-bc7b-fc797d80a7de admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '42d15fc0515945acacbbffcf2356c7a5', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80245) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:32:10.611628 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47d90bf8-35fb-455a-bc7b-fc797d80a7de admin admin] REQUEST_METHOD: `GET` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:32:10.611919 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47d90bf8-35fb-455a-bc7b-fc797d80a7de admin admin] SCRIPT_NAME: `/identity` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:32:10.612230 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47d90bf8-35fb-455a-bc7b-fc797d80a7de admin admin] PATH_INFO: `/v3/roles` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:32:10.612980 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47d90bf8-35fb-455a-bc7b-fc797d80a7de admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:32:10.614590 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47d90bf8-35fb-455a-bc7b-fc797d80a7de admin admin] RBAC: Authorization granted {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:32:10.616057 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c798959-73a9-445e-a923-298e50680cd3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '42d15fc0515945acacbbffcf2356c7a5', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:32:10.617908 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c798959-73a9-445e-a923-298e50680cd3 admin admin] REQUEST_METHOD: `GET` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:32:10.618205 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c798959-73a9-445e-a923-298e50680cd3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:32:10.618486 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c798959-73a9-445e-a923-298e50680cd3 admin admin] PATH_INFO: `/v3/roles` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:32:10.619774 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c798959-73a9-445e-a923-298e50680cd3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:32:10.621956 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c798959-73a9-445e-a923-298e50680cd3 admin admin] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:32:10.625578 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: [pid: 80245|app: 0|req: 1186/2365] 10.222.0.47 () {58 vars in 1121 bytes} [Thu Jul 21 08:32:10 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 21 08:32:10.629583 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47d90bf8-35fb-455a-bc7b-fc797d80a7de admin admin] Authenticating user token {{(pid=80245) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:32:10.636208 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 1180/2366] 10.222.0.47 () {58 vars in 1121 bytes} [Thu Jul 21 08:32:10 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 21 08:32:10.640341 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c798959-73a9-445e-a923-298e50680cd3 admin admin] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:32:10.646733 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47d90bf8-35fb-455a-bc7b-fc797d80a7de admin admin] Validating token access rules against request {{(pid=80245) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:32:10.653792 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5141844-507f-4a8b-a5e1-6c11798fc300 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '42d15fc0515945acacbbffcf2356c7a5', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80245) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:32:10.654586 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5141844-507f-4a8b-a5e1-6c11798fc300 admin admin] REQUEST_METHOD: `GET` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:32:10.654787 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5141844-507f-4a8b-a5e1-6c11798fc300 admin admin] SCRIPT_NAME: `/identity` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:32:10.655125 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5141844-507f-4a8b-a5e1-6c11798fc300 admin admin] PATH_INFO: `/v3/roles` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:32:10.655990 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5141844-507f-4a8b-a5e1-6c11798fc300 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:32:10.657543 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c798959-73a9-445e-a923-298e50680cd3 admin admin] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:32:10.657777 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5141844-507f-4a8b-a5e1-6c11798fc300 admin admin] RBAC: Authorization granted {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:32:10.667201 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a011c1b-cd91-4662-89ef-b2bf21a7665b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '42d15fc0515945acacbbffcf2356c7a5', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:32:10.668081 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a011c1b-cd91-4662-89ef-b2bf21a7665b admin admin] REQUEST_METHOD: `GET` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:32:10.668456 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a011c1b-cd91-4662-89ef-b2bf21a7665b admin admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:32:10.668750 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a011c1b-cd91-4662-89ef-b2bf21a7665b admin admin] PATH_INFO: `/v3/roles` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:32:10.669823 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a011c1b-cd91-4662-89ef-b2bf21a7665b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:32:10.671704 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a011c1b-cd91-4662-89ef-b2bf21a7665b admin admin] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:32:10.677706 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: [pid: 80245|app: 0|req: 1187/2367] 10.222.0.47 () {58 vars in 1121 bytes} [Thu Jul 21 08:32:10 2022] GET /identity/v3/roles => generated 1420 bytes in 49 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 21 08:32:10.679297 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5141844-507f-4a8b-a5e1-6c11798fc300 admin admin] Authenticating user token {{(pid=80245) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:32:10.688953 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 1181/2368] 10.222.0.47 () {58 vars in 1121 bytes} [Thu Jul 21 08:32:10 2022] GET /identity/v3/roles => generated 1420 bytes in 49 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 08:32:10.691026 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5141844-507f-4a8b-a5e1-6c11798fc300 admin admin] Validating token access rules against request {{(pid=80245) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:32:10.693580 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a011c1b-cd91-4662-89ef-b2bf21a7665b admin admin] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:32:10.697183 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8086567-59b6-4541-a202-5075eaf3e4bd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '42d15fc0515945acacbbffcf2356c7a5', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80245) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:32:10.698758 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8086567-59b6-4541-a202-5075eaf3e4bd admin admin] REQUEST_METHOD: `PUT` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:32:10.699136 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8086567-59b6-4541-a202-5075eaf3e4bd admin admin] SCRIPT_NAME: `/identity` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:32:10.700023 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8086567-59b6-4541-a202-5075eaf3e4bd admin admin] PATH_INFO: `/v3/projects/b81fcc0271d64d308682d41e453ebe79/users/f019747f42954cba836a09b7649c3a91/roles/3a074738221e4601b5205715ba87d240` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:32:10.707052 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a011c1b-cd91-4662-89ef-b2bf21a7665b admin admin] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:32:10.714378 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d949cb88-6482-401d-ab72-48c6df6b989c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '42d15fc0515945acacbbffcf2356c7a5', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:32:10.715630 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d949cb88-6482-401d-ab72-48c6df6b989c admin admin] REQUEST_METHOD: `PUT` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:32:10.715936 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d949cb88-6482-401d-ab72-48c6df6b989c admin admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:32:10.716081 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d949cb88-6482-401d-ab72-48c6df6b989c admin admin] PATH_INFO: `/v3/projects/fbbdc782b5fe4c5ca61ba9e149fb94fe/users/d1ef300402cc473d8ccae6ee4d02f698/roles/3a074738221e4601b5205715ba87d240` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:32:10.719076 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8086567-59b6-4541-a202-5075eaf3e4bd admin admin] RBAC: Authorizing `identity:create_grant(project_id=b81fcc0271d64d308682d41e453ebe79, user_id=f019747f42954cba836a09b7649c3a91, role_id=3a074738221e4601b5205715ba87d240)` {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:32:10.722545 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8086567-59b6-4541-a202-5075eaf3e4bd admin admin] RBAC: Authorization granted {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:32:10.734342 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d949cb88-6482-401d-ab72-48c6df6b989c admin admin] RBAC: Authorizing `identity:create_grant(project_id=fbbdc782b5fe4c5ca61ba9e149fb94fe, user_id=d1ef300402cc473d8ccae6ee4d02f698, role_id=3a074738221e4601b5205715ba87d240)` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:32:10.737140 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d949cb88-6482-401d-ab72-48c6df6b989c admin admin] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:32:10.756447 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: [pid: 80245|app: 0|req: 1188/2369] 10.222.0.47 () {60 vars in 1482 bytes} [Thu Jul 21 08:32:10 2022] PUT /identity/v3/projects/b81fcc0271d64d308682d41e453ebe79/users/f019747f42954cba836a09b7649c3a91/roles/3a074738221e4601b5205715ba87d240 => generated 0 bytes in 69 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 08:32:10.756752 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 1182/2370] 10.222.0.47 () {60 vars in 1482 bytes} [Thu Jul 21 08:32:10 2022] PUT /identity/v3/projects/fbbdc782b5fe4c5ca61ba9e149fb94fe/users/d1ef300402cc473d8ccae6ee4d02f698/roles/3a074738221e4601b5205715ba87d240 => generated 0 bytes in 60 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 08:32:10.760686 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6af3b9e-01bc-4517-a0e7-fb19b067fa5a None None] REQUEST_METHOD: `POST` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:32:10.760883 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6af3b9e-01bc-4517-a0e7-fb19b067fa5a None None] SCRIPT_NAME: `/identity` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:32:10.761160 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6af3b9e-01bc-4517-a0e7-fb19b067fa5a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:32:10.766003 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab575a95-bdfb-40b6-94e6-1b855a31adf8 None None] REQUEST_METHOD: `POST` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:32:10.766198 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab575a95-bdfb-40b6-94e6-1b855a31adf8 None None] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:32:10.766478 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab575a95-bdfb-40b6-94e6-1b855a31adf8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:32:10.797877 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.auth.core [None req-f6af3b9e-01bc-4517-a0e7-fb19b067fa5a None None] MFA Rules not processed for user `f019747f42954cba836a09b7649c3a91`. Rule list: `[]` (Enabled: `True`). {{(pid=80245) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 08:32:10.800707 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.fernet_utils [None req-f6af3b9e-01bc-4517-a0e7-fb19b067fa5a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80245) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 08:32:10.818944 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.auth.core [None req-ab575a95-bdfb-40b6-94e6-1b855a31adf8 None None] MFA Rules not processed for user `d1ef300402cc473d8ccae6ee4d02f698`. Rule list: `[]` (Enabled: `True`). {{(pid=80244) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 08:32:10.820224 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.fernet_utils [None req-ab575a95-bdfb-40b6-94e6-1b855a31adf8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80244) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 08:32:10.838739 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG dbcounter [-] [80244] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80244) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:32:10.886179 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: [pid: 80245|app: 0|req: 1189/2371] 10.222.0.47 () {58 vars in 957 bytes} [Thu Jul 21 08:32:10 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 08:32:10.898627 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6af3b9e-01bc-4517-a0e7-fb19b067fa5a None None] Authenticating user token {{(pid=80245) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:32:10.900055 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.fernet_utils [None req-f6af3b9e-01bc-4517-a0e7-fb19b067fa5a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80245) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 08:32:10.914021 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 1183/2372] 10.222.0.47 () {58 vars in 957 bytes} [Thu Jul 21 08:32:10 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 149 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 08:32:10.919510 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab575a95-bdfb-40b6-94e6-1b855a31adf8 None None] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:32:10.991305 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6af3b9e-01bc-4517-a0e7-fb19b067fa5a None None] Validating token access rules against request {{(pid=80245) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:32:10.995519 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab575a95-bdfb-40b6-94e6-1b855a31adf8 None None] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:32:10.997305 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e28dff0e-705e-41e7-a73a-fbe600811bf7 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2b5b99a36442148fe743269054e991', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4faef13aa50e4e36b6c265fb1398fa14', 'project_domain_id': 'default', 'roles': ['member', 'service', 'ResellerAdmin', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80245) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:32:10.997972 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e28dff0e-705e-41e7-a73a-fbe600811bf7 service nova] REQUEST_METHOD: `GET` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:32:10.998257 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e28dff0e-705e-41e7-a73a-fbe600811bf7 service nova] SCRIPT_NAME: `/identity` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:32:10.998524 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e28dff0e-705e-41e7-a73a-fbe600811bf7 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:32:11.002094 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63bbf822-5b22-48eb-9793-8ee1426377da admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '42d15fc0515945acacbbffcf2356c7a5', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:32:11.002775 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63bbf822-5b22-48eb-9793-8ee1426377da admin admin] REQUEST_METHOD: `POST` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:32:11.003066 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63bbf822-5b22-48eb-9793-8ee1426377da admin admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:32:11.003392 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63bbf822-5b22-48eb-9793-8ee1426377da admin admin] PATH_INFO: `/v3/projects` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:32:11.004354 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63bbf822-5b22-48eb-9793-8ee1426377da admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:32:11.006295 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63bbf822-5b22-48eb-9793-8ee1426377da admin admin] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:32:11.007387 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e28dff0e-705e-41e7-a73a-fbe600811bf7 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:32:11.008975 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e28dff0e-705e-41e7-a73a-fbe600811bf7 service nova] RBAC: Authorization granted {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:32:11.019183 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: [pid: 80245|app: 0|req: 1190/2373] 10.222.0.47 () {60 vars in 1365 bytes} [Thu Jul 21 08:32:10 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 121 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 08:32:11.044826 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 1184/2374] 10.222.0.47 () {60 vars in 1152 bytes} [Thu Jul 21 08:32:10 2022] POST /identity/v3/projects => generated 377 bytes in 126 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 08:32:11.054013 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e28dff0e-705e-41e7-a73a-fbe600811bf7 service nova] Authenticating user token {{(pid=80245) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:32:11.069147 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG dbcounter [-] [80245] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80245) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:32:11.090631 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e28dff0e-705e-41e7-a73a-fbe600811bf7 service nova] Validating token access rules against request {{(pid=80245) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:32:11.175011 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e61fb57-2e66-465d-8147-7292bd41124c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '42d15fc0515945acacbbffcf2356c7a5', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80245) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:32:11.175011 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e61fb57-2e66-465d-8147-7292bd41124c admin admin] REQUEST_METHOD: `POST` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:32:11.175011 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e61fb57-2e66-465d-8147-7292bd41124c admin admin] SCRIPT_NAME: `/identity` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:32:11.175011 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e61fb57-2e66-465d-8147-7292bd41124c admin admin] PATH_INFO: `/v3/users` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:32:11.176194 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e61fb57-2e66-465d-8147-7292bd41124c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:32:11.178276 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e61fb57-2e66-465d-8147-7292bd41124c admin admin] RBAC: Authorization granted {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:32:11.203361 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: [pid: 80245|app: 0|req: 1191/2375] 10.222.0.47 () {60 vars in 1143 bytes} [Thu Jul 21 08:32:11 2022] POST /identity/v3/users => generated 289 bytes in 149 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 08:32:11.207657 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63bbf822-5b22-48eb-9793-8ee1426377da admin admin] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:32:11.222741 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63bbf822-5b22-48eb-9793-8ee1426377da admin admin] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:32:11.229438 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-945b1f4f-b641-460b-bf96-69b8b61fc7ca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '42d15fc0515945acacbbffcf2356c7a5', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:32:11.230121 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-945b1f4f-b641-460b-bf96-69b8b61fc7ca admin admin] REQUEST_METHOD: `GET` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:32:11.230405 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-945b1f4f-b641-460b-bf96-69b8b61fc7ca admin admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:32:11.230684 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-945b1f4f-b641-460b-bf96-69b8b61fc7ca admin admin] PATH_INFO: `/v3/roles` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:32:11.231478 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-945b1f4f-b641-460b-bf96-69b8b61fc7ca admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:32:11.236261 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-945b1f4f-b641-460b-bf96-69b8b61fc7ca admin admin] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:32:11.245821 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 1185/2376] 10.222.0.47 () {58 vars in 1121 bytes} [Thu Jul 21 08:32:11 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 08:32:11.250169 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e61fb57-2e66-465d-8147-7292bd41124c admin admin] Authenticating user token {{(pid=80245) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:32:11.267749 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e61fb57-2e66-465d-8147-7292bd41124c admin admin] Validating token access rules against request {{(pid=80245) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:32:11.278922 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90159760-6cc9-4664-b0e7-781b542cbbdb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '42d15fc0515945acacbbffcf2356c7a5', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80245) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:32:11.279654 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90159760-6cc9-4664-b0e7-781b542cbbdb admin admin] REQUEST_METHOD: `GET` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:32:11.281097 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90159760-6cc9-4664-b0e7-781b542cbbdb admin admin] SCRIPT_NAME: `/identity` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:32:11.281713 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90159760-6cc9-4664-b0e7-781b542cbbdb admin admin] PATH_INFO: `/v3/roles` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:32:11.282270 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90159760-6cc9-4664-b0e7-781b542cbbdb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:32:11.284095 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90159760-6cc9-4664-b0e7-781b542cbbdb admin admin] RBAC: Authorization granted {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:32:11.296695 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: [pid: 80245|app: 0|req: 1192/2377] 10.222.0.47 () {58 vars in 1121 bytes} [Thu Jul 21 08:32:11 2022] GET /identity/v3/roles => generated 1420 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 21 08:32:11.301011 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-945b1f4f-b641-460b-bf96-69b8b61fc7ca admin admin] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:32:11.314054 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-945b1f4f-b641-460b-bf96-69b8b61fc7ca admin admin] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:32:11.323390 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90159760-6cc9-4664-b0e7-781b542cbbdb admin admin] Authenticating user token {{(pid=80245) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:32:11.327099 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b819d5c-b804-4a36-9624-73e823c7b463 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '42d15fc0515945acacbbffcf2356c7a5', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:32:11.328260 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b819d5c-b804-4a36-9624-73e823c7b463 admin admin] REQUEST_METHOD: `PUT` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:32:11.329227 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b819d5c-b804-4a36-9624-73e823c7b463 admin admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:32:11.329512 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b819d5c-b804-4a36-9624-73e823c7b463 admin admin] PATH_INFO: `/v3/projects/67739db64f6044b19d5b8e5e007d3b66/users/7d24f72b48d04f3bb94401f05d838527/roles/3a074738221e4601b5205715ba87d240` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:32:11.334843 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90159760-6cc9-4664-b0e7-781b542cbbdb admin admin] Validating token access rules against request {{(pid=80245) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:32:11.340857 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edfcf0e6-593b-48b1-aa73-3af551d6d47b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '42d15fc0515945acacbbffcf2356c7a5', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80245) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:32:11.341759 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edfcf0e6-593b-48b1-aa73-3af551d6d47b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:32:11.342045 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edfcf0e6-593b-48b1-aa73-3af551d6d47b admin admin] SCRIPT_NAME: `/identity` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:32:11.342307 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edfcf0e6-593b-48b1-aa73-3af551d6d47b admin admin] PATH_INFO: `/v3/users/f019747f42954cba836a09b7649c3a91` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:32:11.343594 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edfcf0e6-593b-48b1-aa73-3af551d6d47b admin admin] RBAC: Authorizing `identity:delete_user(user_id=f019747f42954cba836a09b7649c3a91)` {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:32:11.346183 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edfcf0e6-593b-48b1-aa73-3af551d6d47b admin admin] RBAC: Authorization granted {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:32:11.347449 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b819d5c-b804-4a36-9624-73e823c7b463 admin admin] RBAC: Authorizing `identity:create_grant(project_id=67739db64f6044b19d5b8e5e007d3b66, user_id=7d24f72b48d04f3bb94401f05d838527, role_id=3a074738221e4601b5205715ba87d240)` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:32:11.350137 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b819d5c-b804-4a36-9624-73e823c7b463 admin admin] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:32:11.369060 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 1186/2378] 10.222.0.47 () {60 vars in 1482 bytes} [Thu Jul 21 08:32:11 2022] PUT /identity/v3/projects/67739db64f6044b19d5b8e5e007d3b66/users/7d24f72b48d04f3bb94401f05d838527/roles/3a074738221e4601b5205715ba87d240 => generated 0 bytes in 69 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 08:32:11.380682 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67227c14-813b-4fe1-beb6-7fa22a96130d None None] REQUEST_METHOD: `POST` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:32:11.381366 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67227c14-813b-4fe1-beb6-7fa22a96130d None None] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:32:11.381366 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67227c14-813b-4fe1-beb6-7fa22a96130d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:32:11.413303 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.auth.core [None req-67227c14-813b-4fe1-beb6-7fa22a96130d None None] MFA Rules not processed for user `7d24f72b48d04f3bb94401f05d838527`. Rule list: `[]` (Enabled: `True`). {{(pid=80244) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 08:32:11.414959 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.fernet_utils [None req-67227c14-813b-4fe1-beb6-7fa22a96130d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80244) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 08:32:11.418238 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.notifications [None req-edfcf0e6-593b-48b1-aa73-3af551d6d47b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f019747f42954cba836a09b7649c3a91'} {{(pid=80245) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 08:32:11.432320 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.notifications [None req-edfcf0e6-593b-48b1-aa73-3af551d6d47b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f019747f42954cba836a09b7649c3a91'} {{(pid=80245) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 08:32:11.465538 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.notifications [None req-edfcf0e6-593b-48b1-aa73-3af551d6d47b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f019747f42954cba836a09b7649c3a91'} {{(pid=80245) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 08:32:11.476917 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.notifications [None req-edfcf0e6-593b-48b1-aa73-3af551d6d47b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f019747f42954cba836a09b7649c3a91'} {{(pid=80245) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 08:32:11.478486 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.notifications [None req-edfcf0e6-593b-48b1-aa73-3af551d6d47b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f019747f42954cba836a09b7649c3a91'} {{(pid=80245) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 08:32:11.488085 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG dbcounter [-] [80244] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80244) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:32:11.490494 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: [pid: 80245|app: 0|req: 1193/2379] 10.222.0.47 () {58 vars in 1223 bytes} [Thu Jul 21 08:32:11 2022] DELETE /identity/v3/users/f019747f42954cba836a09b7649c3a91 => generated 0 bytes in 168 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 08:32:11.495913 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80d1befc-a168-4d90-ad2f-3a888b5c4d29 None None] REQUEST_METHOD: `POST` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:32:11.496593 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80d1befc-a168-4d90-ad2f-3a888b5c4d29 None None] SCRIPT_NAME: `/identity` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:32:11.497121 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80d1befc-a168-4d90-ad2f-3a888b5c4d29 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:32:11.518742 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 1187/2380] 10.222.0.47 () {58 vars in 957 bytes} [Thu Jul 21 08:32:11 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 140 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 08:32:11.523746 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67227c14-813b-4fe1-beb6-7fa22a96130d None None] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:32:11.524598 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.auth.core [None req-80d1befc-a168-4d90-ad2f-3a888b5c4d29 None None] MFA Rules not processed for user `2dce40b8d74c41ec94cc36e6ff05d412`. Rule list: `[]` (Enabled: `True`). {{(pid=80245) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 08:32:11.528233 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.fernet_utils [None req-67227c14-813b-4fe1-beb6-7fa22a96130d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80244) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 08:32:11.528368 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.fernet_utils [None req-80d1befc-a168-4d90-ad2f-3a888b5c4d29 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80245) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 08:32:11.589921 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG dbcounter [-] [80245] Writing DB stats keystone:SELECT=86,keystone:INSERT=4,keystone:DELETE=10 {{(pid=80245) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:32:11.606082 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67227c14-813b-4fe1-beb6-7fa22a96130d None None] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:32:11.606746 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: [pid: 80245|app: 0|req: 1194/2381] 10.222.0.47 () {58 vars in 957 bytes} [Thu Jul 21 08:32:11 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 08:32:11.615900 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e513e09-6efa-45be-9410-adadb7f5932d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '42d15fc0515945acacbbffcf2356c7a5', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:32:11.616196 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80d1befc-a168-4d90-ad2f-3a888b5c4d29 None None] Authenticating user token {{(pid=80245) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:32:11.616603 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e513e09-6efa-45be-9410-adadb7f5932d admin admin] REQUEST_METHOD: `POST` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:32:11.617005 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e513e09-6efa-45be-9410-adadb7f5932d admin admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:32:11.617851 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e513e09-6efa-45be-9410-adadb7f5932d admin admin] PATH_INFO: `/v3/projects` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:32:11.618020 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.fernet_utils [None req-80d1befc-a168-4d90-ad2f-3a888b5c4d29 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80245) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 08:32:11.618598 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e513e09-6efa-45be-9410-adadb7f5932d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:32:11.621166 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e513e09-6efa-45be-9410-adadb7f5932d admin admin] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:32:11.659538 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 1188/2382] 10.222.0.47 () {60 vars in 1152 bytes} [Thu Jul 21 08:32:11 2022] POST /identity/v3/projects => generated 377 bytes in 135 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 08:32:11.662295 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e513e09-6efa-45be-9410-adadb7f5932d admin admin] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:32:11.689950 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80d1befc-a168-4d90-ad2f-3a888b5c4d29 None None] Validating token access rules against request {{(pid=80245) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:32:11.728223 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e513e09-6efa-45be-9410-adadb7f5932d admin admin] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:32:11.735085 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d50f5cb-edcc-43f9-816d-05459d4b3207 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '42d15fc0515945acacbbffcf2356c7a5', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:32:11.737138 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d50f5cb-edcc-43f9-816d-05459d4b3207 admin admin] REQUEST_METHOD: `POST` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:32:11.737439 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d50f5cb-edcc-43f9-816d-05459d4b3207 admin admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:32:11.737747 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d50f5cb-edcc-43f9-816d-05459d4b3207 admin admin] PATH_INFO: `/v3/users` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:32:11.738474 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d50f5cb-edcc-43f9-816d-05459d4b3207 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:32:11.740704 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d50f5cb-edcc-43f9-816d-05459d4b3207 admin admin] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:32:11.752469 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-beb3c719-4425-48a1-82e7-0476a2e09c70 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20ca3c3bd3774691b3374cfbfec6e494', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4faef13aa50e4e36b6c265fb1398fa14', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80245) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:32:11.753125 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beb3c719-4425-48a1-82e7-0476a2e09c70 service neutron] REQUEST_METHOD: `GET` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:32:11.753407 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beb3c719-4425-48a1-82e7-0476a2e09c70 service neutron] SCRIPT_NAME: `/identity` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:32:11.753679 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beb3c719-4425-48a1-82e7-0476a2e09c70 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:32:11.765209 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 1189/2383] 10.222.0.47 () {60 vars in 1143 bytes} [Thu Jul 21 08:32:11 2022] POST /identity/v3/users => generated 295 bytes in 101 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 08:32:11.765384 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-beb3c719-4425-48a1-82e7-0476a2e09c70 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:32:11.765480 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-beb3c719-4425-48a1-82e7-0476a2e09c70 service neutron] RBAC: Authorization granted {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:32:11.766265 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d50f5cb-edcc-43f9-816d-05459d4b3207 admin admin] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:32:11.775998 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: [pid: 80245|app: 0|req: 1195/2384] 10.222.0.47 () {60 vars in 1365 bytes} [Thu Jul 21 08:32:11 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 160 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 08:32:11.776818 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d50f5cb-edcc-43f9-816d-05459d4b3207 admin admin] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:32:11.781874 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7da6b411-d44d-4b2b-b303-53a7b0025741 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '42d15fc0515945acacbbffcf2356c7a5', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:32:11.782562 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7da6b411-d44d-4b2b-b303-53a7b0025741 admin admin] REQUEST_METHOD: `GET` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:32:11.782841 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7da6b411-d44d-4b2b-b303-53a7b0025741 admin admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:32:11.783160 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7da6b411-d44d-4b2b-b303-53a7b0025741 admin admin] PATH_INFO: `/v3/roles` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:32:11.783904 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7da6b411-d44d-4b2b-b303-53a7b0025741 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:32:11.785512 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7da6b411-d44d-4b2b-b303-53a7b0025741 admin admin] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:32:11.802823 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-beb3c719-4425-48a1-82e7-0476a2e09c70 service neutron] Authenticating user token {{(pid=80245) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:32:11.807711 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 1190/2385] 10.222.0.47 () {58 vars in 1121 bytes} [Thu Jul 21 08:32:11 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 21 08:32:11.814554 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-beb3c719-4425-48a1-82e7-0476a2e09c70 service neutron] Validating token access rules against request {{(pid=80245) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:32:11.819721 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-648ff58b-62b4-43a9-8419-f00776baf527 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '42d15fc0515945acacbbffcf2356c7a5', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80245) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:32:11.820859 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-648ff58b-62b4-43a9-8419-f00776baf527 admin admin] REQUEST_METHOD: `PUT` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:32:11.820980 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-648ff58b-62b4-43a9-8419-f00776baf527 admin admin] SCRIPT_NAME: `/identity` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:32:11.821227 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-648ff58b-62b4-43a9-8419-f00776baf527 admin admin] PATH_INFO: `/v3/projects/6fc370bce50947ea8fb7e7e0f0d68912/users/0c9b621a1c8b46cc941d9e5d81d859f4/roles/e29870f6e9e2411a9db0d5bf45f32ca5` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:32:11.839261 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-648ff58b-62b4-43a9-8419-f00776baf527 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6fc370bce50947ea8fb7e7e0f0d68912, user_id=0c9b621a1c8b46cc941d9e5d81d859f4, role_id=e29870f6e9e2411a9db0d5bf45f32ca5)` {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:32:11.843833 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-648ff58b-62b4-43a9-8419-f00776baf527 admin admin] RBAC: Authorization granted {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:32:11.872116 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: [pid: 80245|app: 0|req: 1196/2386] 10.222.0.47 () {60 vars in 1482 bytes} [Thu Jul 21 08:32:11 2022] PUT /identity/v3/projects/6fc370bce50947ea8fb7e7e0f0d68912/users/0c9b621a1c8b46cc941d9e5d81d859f4/roles/e29870f6e9e2411a9db0d5bf45f32ca5 => generated 0 bytes in 66 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 08:32:11.873473 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7da6b411-d44d-4b2b-b303-53a7b0025741 admin admin] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:32:11.885838 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7da6b411-d44d-4b2b-b303-53a7b0025741 admin admin] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:32:11.976025 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG dbcounter [-] [80244] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80244) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:32:11.988077 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ccc985e-622b-4a07-8ce0-ae4f014e8947 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '42d15fc0515945acacbbffcf2356c7a5', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:32:11.988848 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ccc985e-622b-4a07-8ce0-ae4f014e8947 admin admin] REQUEST_METHOD: `GET` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:32:11.989061 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ccc985e-622b-4a07-8ce0-ae4f014e8947 admin admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:32:11.989404 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ccc985e-622b-4a07-8ce0-ae4f014e8947 admin admin] PATH_INFO: `/v3/roles` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:32:11.990090 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ccc985e-622b-4a07-8ce0-ae4f014e8947 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:32:11.991906 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ccc985e-622b-4a07-8ce0-ae4f014e8947 admin admin] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:32:12.005785 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 1191/2387] 10.222.0.47 () {58 vars in 1121 bytes} [Thu Jul 21 08:32:11 2022] GET /identity/v3/roles => generated 1420 bytes in 133 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 08:32:12.009997 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-648ff58b-62b4-43a9-8419-f00776baf527 admin admin] Authenticating user token {{(pid=80245) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:32:12.020983 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-648ff58b-62b4-43a9-8419-f00776baf527 admin admin] Validating token access rules against request {{(pid=80245) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:32:12.026652 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fe17d66-6830-4a04-a8ae-796833e5f4bc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '42d15fc0515945acacbbffcf2356c7a5', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80245) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:32:12.027779 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fe17d66-6830-4a04-a8ae-796833e5f4bc admin admin] REQUEST_METHOD: `PUT` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:32:12.028243 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fe17d66-6830-4a04-a8ae-796833e5f4bc admin admin] SCRIPT_NAME: `/identity` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:32:12.028672 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fe17d66-6830-4a04-a8ae-796833e5f4bc admin admin] PATH_INFO: `/v3/projects/6fc370bce50947ea8fb7e7e0f0d68912/users/0c9b621a1c8b46cc941d9e5d81d859f4/roles/e29870f6e9e2411a9db0d5bf45f32ca5` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:32:12.031034 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fe17d66-6830-4a04-a8ae-796833e5f4bc admin admin] RBAC: Authorizing `identity:create_grant(project_id=6fc370bce50947ea8fb7e7e0f0d68912, user_id=0c9b621a1c8b46cc941d9e5d81d859f4, role_id=e29870f6e9e2411a9db0d5bf45f32ca5)` {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:32:12.033919 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fe17d66-6830-4a04-a8ae-796833e5f4bc admin admin] RBAC: Authorization granted {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:32:12.046334 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: [pid: 80245|app: 0|req: 1197/2388] 10.222.0.47 () {60 vars in 1482 bytes} [Thu Jul 21 08:32:12 2022] PUT /identity/v3/projects/6fc370bce50947ea8fb7e7e0f0d68912/users/0c9b621a1c8b46cc941d9e5d81d859f4/roles/e29870f6e9e2411a9db0d5bf45f32ca5 => generated 0 bytes in 37 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 08:32:12.056437 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da39d6c0-ebf7-41f6-b114-7175b376200c None None] REQUEST_METHOD: `POST` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:32:12.057278 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da39d6c0-ebf7-41f6-b114-7175b376200c None None] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:32:12.057559 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da39d6c0-ebf7-41f6-b114-7175b376200c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:32:12.089949 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.auth.core [None req-da39d6c0-ebf7-41f6-b114-7175b376200c None None] MFA Rules not processed for user `0c9b621a1c8b46cc941d9e5d81d859f4`. Rule list: `[]` (Enabled: `True`). {{(pid=80244) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 08:32:12.091804 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.fernet_utils [None req-da39d6c0-ebf7-41f6-b114-7175b376200c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80244) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 08:32:12.137022 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fe17d66-6830-4a04-a8ae-796833e5f4bc admin admin] Authenticating user token {{(pid=80245) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:32:12.138966 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.fernet_utils [None req-1fe17d66-6830-4a04-a8ae-796833e5f4bc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80245) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 08:32:12.233237 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fe17d66-6830-4a04-a8ae-796833e5f4bc admin admin] Validating token access rules against request {{(pid=80245) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:32:12.238556 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89cd5093-8df3-45c2-96f7-901e63bdbb93 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '42d15fc0515945acacbbffcf2356c7a5', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80245) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:32:12.239525 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89cd5093-8df3-45c2-96f7-901e63bdbb93 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:32:12.239811 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89cd5093-8df3-45c2-96f7-901e63bdbb93 admin admin] SCRIPT_NAME: `/identity` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:32:12.240141 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 1192/2389] 10.222.0.47 () {58 vars in 957 bytes} [Thu Jul 21 08:32:12 2022] POST /identity/v3/auth/tokens => generated 3311 bytes in 185 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 08:32:12.244462 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89cd5093-8df3-45c2-96f7-901e63bdbb93 admin admin] PATH_INFO: `/v3/projects/b81fcc0271d64d308682d41e453ebe79` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:32:12.245162 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89cd5093-8df3-45c2-96f7-901e63bdbb93 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b81fcc0271d64d308682d41e453ebe79)` {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:32:12.247838 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89cd5093-8df3-45c2-96f7-901e63bdbb93 admin admin] RBAC: Authorization granted {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:32:12.255686 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da39d6c0-ebf7-41f6-b114-7175b376200c None None] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:32:12.258289 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.fernet_utils [None req-da39d6c0-ebf7-41f6-b114-7175b376200c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80244) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 08:32:12.286945 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG dbcounter [-] [80245] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80245) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:32:12.341453 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da39d6c0-ebf7-41f6-b114-7175b376200c None None] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:32:12.346130 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4783b378-e4de-4ce3-9e6c-44116485509c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4f40410b7674687ab5362ac982248ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4faef13aa50e4e36b6c265fb1398fa14', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:32:12.346905 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4783b378-e4de-4ce3-9e6c-44116485509c service cinder] REQUEST_METHOD: `GET` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:32:12.347192 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4783b378-e4de-4ce3-9e6c-44116485509c service cinder] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:32:12.347517 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4783b378-e4de-4ce3-9e6c-44116485509c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:32:12.350524 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.fernet_utils [None req-4783b378-e4de-4ce3-9e6c-44116485509c service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80244) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 08:32:12.379644 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.notifications [None req-89cd5093-8df3-45c2-96f7-901e63bdbb93 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b81fcc0271d64d308682d41e453ebe79', 'request_id': 'req-89cd5093-8df3-45c2-96f7-901e63bdbb93'} {{(pid=80245) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 08:32:12.393449 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.notifications [None req-89cd5093-8df3-45c2-96f7-901e63bdbb93 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b81fcc0271d64d308682d41e453ebe79', 'request_id': 'req-89cd5093-8df3-45c2-96f7-901e63bdbb93'} {{(pid=80245) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 08:32:12.408732 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.notifications [None req-89cd5093-8df3-45c2-96f7-901e63bdbb93 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b81fcc0271d64d308682d41e453ebe79', 'request_id': 'req-89cd5093-8df3-45c2-96f7-901e63bdbb93'} {{(pid=80245) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 08:32:12.434916 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.notifications [None req-89cd5093-8df3-45c2-96f7-901e63bdbb93 admin admin] The token cache is being invalidate because project b81fcc0271d64d308682d41e453ebe79 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80245) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 21 08:32:12.435226 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.notifications [None req-89cd5093-8df3-45c2-96f7-901e63bdbb93 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80245) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 08:32:12.435957 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.notifications [None req-89cd5093-8df3-45c2-96f7-901e63bdbb93 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80245) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 08:32:12.440778 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: [pid: 80245|app: 0|req: 1198/2390] 10.222.0.47 () {58 vars in 1232 bytes} [Thu Jul 21 08:32:12 2022] DELETE /identity/v3/projects/b81fcc0271d64d308682d41e453ebe79 => generated 0 bytes in 302 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 08:32:12.442494 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34e058a4-b46b-464a-b78b-dee7d9891dca None None] REQUEST_METHOD: `POST` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:32:12.442792 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34e058a4-b46b-464a-b78b-dee7d9891dca None None] SCRIPT_NAME: `/identity` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:32:12.443070 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34e058a4-b46b-464a-b78b-dee7d9891dca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:32:12.448238 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4783b378-e4de-4ce3-9e6c-44116485509c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:32:12.449778 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4783b378-e4de-4ce3-9e6c-44116485509c service cinder] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:32:12.469153 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.auth.core [None req-34e058a4-b46b-464a-b78b-dee7d9891dca None None] MFA Rules not processed for user `2dce40b8d74c41ec94cc36e6ff05d412`. Rule list: `[]` (Enabled: `True`). {{(pid=80245) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 08:32:12.471498 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.fernet_utils [None req-34e058a4-b46b-464a-b78b-dee7d9891dca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80245) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 08:32:12.478288 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 1193/2391] 10.222.0.47 () {60 vars in 1365 bytes} [Thu Jul 21 08:32:12 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 224 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 21 08:32:12.549352 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: [pid: 80245|app: 0|req: 1199/2392] 10.222.0.47 () {58 vars in 957 bytes} [Thu Jul 21 08:32:12 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 08:32:12.556962 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ea56fb6-086d-4e50-9515-6932f1239504 None None] REQUEST_METHOD: `POST` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:32:12.557261 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ea56fb6-086d-4e50-9515-6932f1239504 None None] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:32:12.557546 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ea56fb6-086d-4e50-9515-6932f1239504 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:32:12.576943 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.auth.core [None req-7ea56fb6-086d-4e50-9515-6932f1239504 None None] MFA Rules not processed for user `2dce40b8d74c41ec94cc36e6ff05d412`. Rule list: `[]` (Enabled: `True`). {{(pid=80244) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 08:32:12.578165 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.fernet_utils [None req-7ea56fb6-086d-4e50-9515-6932f1239504 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80244) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 08:32:12.591394 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 1194/2393] 10.222.0.47 () {58 vars in 957 bytes} [Thu Jul 21 08:32:12 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 21 08:32:12.592580 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34e058a4-b46b-464a-b78b-dee7d9891dca None None] Authenticating user token {{(pid=80245) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:32:12.603236 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34e058a4-b46b-464a-b78b-dee7d9891dca None None] Validating token access rules against request {{(pid=80245) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:32:12.608352 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdd7cf97-44d1-4851-8b43-f289de4f8f87 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '42d15fc0515945acacbbffcf2356c7a5', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80245) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:32:12.609178 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdd7cf97-44d1-4851-8b43-f289de4f8f87 admin admin] REQUEST_METHOD: `GET` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:32:12.609493 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdd7cf97-44d1-4851-8b43-f289de4f8f87 admin admin] SCRIPT_NAME: `/identity` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:32:12.609935 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdd7cf97-44d1-4851-8b43-f289de4f8f87 admin admin] PATH_INFO: `/v3/domains` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:32:12.610735 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdd7cf97-44d1-4851-8b43-f289de4f8f87 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80245) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 08:32:12.611181 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdd7cf97-44d1-4851-8b43-f289de4f8f87 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:32:12.612798 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdd7cf97-44d1-4851-8b43-f289de4f8f87 admin admin] RBAC: Authorization granted {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:32:12.628148 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: [pid: 80245|app: 0|req: 1200/2394] 10.222.0.47 () {58 vars in 1152 bytes} [Thu Jul 21 08:32:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 08:32:12.633570 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ea56fb6-086d-4e50-9515-6932f1239504 None None] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:32:12.643646 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ea56fb6-086d-4e50-9515-6932f1239504 None None] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:32:12.648577 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aab2c826-4b96-43b3-977f-c811c6010200 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '42d15fc0515945acacbbffcf2356c7a5', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:32:12.649243 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aab2c826-4b96-43b3-977f-c811c6010200 admin admin] REQUEST_METHOD: `POST` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:32:12.649529 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aab2c826-4b96-43b3-977f-c811c6010200 admin admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:32:12.649834 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aab2c826-4b96-43b3-977f-c811c6010200 admin admin] PATH_INFO: `/v3/projects` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:32:12.651051 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aab2c826-4b96-43b3-977f-c811c6010200 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:32:12.653015 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aab2c826-4b96-43b3-977f-c811c6010200 admin admin] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:32:12.675369 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG dbcounter [-] [80244] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80244) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:32:12.685045 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 1195/2395] 10.222.0.47 () {60 vars in 1152 bytes} [Thu Jul 21 08:32:12 2022] POST /identity/v3/projects => generated 403 bytes in 52 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 21 08:32:12.691378 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdd7cf97-44d1-4851-8b43-f289de4f8f87 admin admin] Authenticating user token {{(pid=80245) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:32:12.705375 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdd7cf97-44d1-4851-8b43-f289de4f8f87 admin admin] Validating token access rules against request {{(pid=80245) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:32:12.811188 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c87d6213-9543-472c-ad60-adb95ab945d0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '42d15fc0515945acacbbffcf2356c7a5', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80245) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:32:12.811915 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c87d6213-9543-472c-ad60-adb95ab945d0 admin admin] REQUEST_METHOD: `POST` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:32:12.812372 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c87d6213-9543-472c-ad60-adb95ab945d0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:32:12.812755 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c87d6213-9543-472c-ad60-adb95ab945d0 admin admin] PATH_INFO: `/v3/users` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:32:12.821131 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c87d6213-9543-472c-ad60-adb95ab945d0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:32:12.821131 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c87d6213-9543-472c-ad60-adb95ab945d0 admin admin] RBAC: Authorization granted {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:32:12.847175 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: [pid: 80245|app: 0|req: 1201/2396] 10.222.0.47 () {60 vars in 1143 bytes} [Thu Jul 21 08:32:12 2022] POST /identity/v3/users => generated 302 bytes in 155 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 08:32:12.848227 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aab2c826-4b96-43b3-977f-c811c6010200 admin admin] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:32:12.859190 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aab2c826-4b96-43b3-977f-c811c6010200 admin admin] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:32:12.864226 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cb6c16e-ea0c-4e10-a5c8-eb440d63d113 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '42d15fc0515945acacbbffcf2356c7a5', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:32:12.864912 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cb6c16e-ea0c-4e10-a5c8-eb440d63d113 admin admin] REQUEST_METHOD: `GET` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:32:12.865286 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cb6c16e-ea0c-4e10-a5c8-eb440d63d113 admin admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:32:12.865465 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cb6c16e-ea0c-4e10-a5c8-eb440d63d113 admin admin] PATH_INFO: `/v3/roles` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:32:12.866194 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cb6c16e-ea0c-4e10-a5c8-eb440d63d113 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:32:12.868014 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cb6c16e-ea0c-4e10-a5c8-eb440d63d113 admin admin] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:32:12.882498 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 1196/2397] 10.222.0.47 () {58 vars in 1121 bytes} [Thu Jul 21 08:32:12 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 08:32:12.887464 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c87d6213-9543-472c-ad60-adb95ab945d0 admin admin] Authenticating user token {{(pid=80245) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:32:12.897544 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c87d6213-9543-472c-ad60-adb95ab945d0 admin admin] Validating token access rules against request {{(pid=80245) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:32:12.902297 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85edf5fb-2a16-43ca-8b74-e37a8e4b8775 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '42d15fc0515945acacbbffcf2356c7a5', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80245) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:32:12.902956 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85edf5fb-2a16-43ca-8b74-e37a8e4b8775 admin admin] REQUEST_METHOD: `GET` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:32:12.903239 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85edf5fb-2a16-43ca-8b74-e37a8e4b8775 admin admin] SCRIPT_NAME: `/identity` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:32:12.903556 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85edf5fb-2a16-43ca-8b74-e37a8e4b8775 admin admin] PATH_INFO: `/v3/roles` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:32:12.904310 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85edf5fb-2a16-43ca-8b74-e37a8e4b8775 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:32:12.906025 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85edf5fb-2a16-43ca-8b74-e37a8e4b8775 admin admin] RBAC: Authorization granted {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:32:12.916498 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: [pid: 80245|app: 0|req: 1202/2398] 10.222.0.47 () {58 vars in 1121 bytes} [Thu Jul 21 08:32:12 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 08:32:12.920649 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cb6c16e-ea0c-4e10-a5c8-eb440d63d113 admin admin] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:32:12.930576 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cb6c16e-ea0c-4e10-a5c8-eb440d63d113 admin admin] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:32:12.935685 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d540919-9829-4f15-b1a3-627091d4eccb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '42d15fc0515945acacbbffcf2356c7a5', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:32:12.936562 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d540919-9829-4f15-b1a3-627091d4eccb admin admin] REQUEST_METHOD: `PUT` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:32:12.936845 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d540919-9829-4f15-b1a3-627091d4eccb admin admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:32:12.937120 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d540919-9829-4f15-b1a3-627091d4eccb admin admin] PATH_INFO: `/v3/projects/181f3e9749e94e98a6bcbff5be5d36d9/users/ef99762d947244edbf9514de634ed46f/roles/3a074738221e4601b5205715ba87d240` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:32:12.952012 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d540919-9829-4f15-b1a3-627091d4eccb admin admin] RBAC: Authorizing `identity:create_grant(project_id=181f3e9749e94e98a6bcbff5be5d36d9, user_id=ef99762d947244edbf9514de634ed46f, role_id=3a074738221e4601b5205715ba87d240)` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:32:12.954673 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d540919-9829-4f15-b1a3-627091d4eccb admin admin] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:32:12.967427 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 1197/2399] 10.222.0.47 () {60 vars in 1482 bytes} [Thu Jul 21 08:32:12 2022] PUT /identity/v3/projects/181f3e9749e94e98a6bcbff5be5d36d9/users/ef99762d947244edbf9514de634ed46f/roles/3a074738221e4601b5205715ba87d240 => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 08:32:12.978974 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e67d464-d2bf-4425-9fe9-8c2d4d3ab196 None None] REQUEST_METHOD: `POST` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:32:12.979268 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e67d464-d2bf-4425-9fe9-8c2d4d3ab196 None None] SCRIPT_NAME: `/identity` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:32:12.979605 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e67d464-d2bf-4425-9fe9-8c2d4d3ab196 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:32:12.999060 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG dbcounter [-] [80245] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=80245) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:32:13.008595 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.auth.core [None req-6e67d464-d2bf-4425-9fe9-8c2d4d3ab196 None None] MFA Rules not processed for user `ef99762d947244edbf9514de634ed46f`. Rule list: `[]` (Enabled: `True`). {{(pid=80245) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 08:32:13.011114 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.fernet_utils [None req-6e67d464-d2bf-4425-9fe9-8c2d4d3ab196 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80245) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 08:32:13.076655 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: [pid: 80245|app: 0|req: 1203/2400] 10.222.0.47 () {58 vars in 957 bytes} [Thu Jul 21 08:32:12 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 08:32:13.082850 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d540919-9829-4f15-b1a3-627091d4eccb admin admin] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:32:13.146747 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d540919-9829-4f15-b1a3-627091d4eccb admin admin] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:32:13.151506 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5830989-4e41-4675-ad47-f9634b3da8d8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '42d15fc0515945acacbbffcf2356c7a5', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:32:13.152109 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5830989-4e41-4675-ad47-f9634b3da8d8 admin admin] REQUEST_METHOD: `POST` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:32:13.152395 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5830989-4e41-4675-ad47-f9634b3da8d8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:32:13.152674 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5830989-4e41-4675-ad47-f9634b3da8d8 admin admin] PATH_INFO: `/v3/projects` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:32:13.153383 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5830989-4e41-4675-ad47-f9634b3da8d8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:32:13.155285 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5830989-4e41-4675-ad47-f9634b3da8d8 admin admin] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:32:13.186626 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 1198/2401] 10.222.0.47 () {60 vars in 1152 bytes} [Thu Jul 21 08:32:13 2022] POST /identity/v3/projects => generated 405 bytes in 104 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 08:32:13.190829 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e67d464-d2bf-4425-9fe9-8c2d4d3ab196 None None] Authenticating user token {{(pid=80245) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:32:13.204243 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e67d464-d2bf-4425-9fe9-8c2d4d3ab196 None None] Validating token access rules against request {{(pid=80245) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:32:13.276332 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-775bb059-c33c-4a96-8861-7305959419a3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '42d15fc0515945acacbbffcf2356c7a5', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80245) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:32:13.277008 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-775bb059-c33c-4a96-8861-7305959419a3 admin admin] REQUEST_METHOD: `POST` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:32:13.277294 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-775bb059-c33c-4a96-8861-7305959419a3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:32:13.277572 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-775bb059-c33c-4a96-8861-7305959419a3 admin admin] PATH_INFO: `/v3/users` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:32:13.278268 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-775bb059-c33c-4a96-8861-7305959419a3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:32:13.280236 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-775bb059-c33c-4a96-8861-7305959419a3 admin admin] RBAC: Authorization granted {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:32:13.305007 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: [pid: 80245|app: 0|req: 1204/2402] 10.222.0.47 () {60 vars in 1143 bytes} [Thu Jul 21 08:32:13 2022] POST /identity/v3/users => generated 309 bytes in 112 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 08:32:13.306243 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5830989-4e41-4675-ad47-f9634b3da8d8 admin admin] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:32:13.317345 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5830989-4e41-4675-ad47-f9634b3da8d8 admin admin] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:32:13.322111 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b301c12-3bf2-47f6-b6af-74ffafe993f8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '42d15fc0515945acacbbffcf2356c7a5', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:32:13.322814 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b301c12-3bf2-47f6-b6af-74ffafe993f8 admin admin] REQUEST_METHOD: `GET` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:32:13.323101 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b301c12-3bf2-47f6-b6af-74ffafe993f8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:32:13.323413 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b301c12-3bf2-47f6-b6af-74ffafe993f8 admin admin] PATH_INFO: `/v3/roles` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:32:13.324180 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b301c12-3bf2-47f6-b6af-74ffafe993f8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:32:13.325818 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b301c12-3bf2-47f6-b6af-74ffafe993f8 admin admin] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:32:13.337283 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 1199/2403] 10.222.0.47 () {58 vars in 1121 bytes} [Thu Jul 21 08:32:13 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 08:32:13.342724 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-775bb059-c33c-4a96-8861-7305959419a3 admin admin] Authenticating user token {{(pid=80245) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:32:13.354325 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-775bb059-c33c-4a96-8861-7305959419a3 admin admin] Validating token access rules against request {{(pid=80245) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:32:13.361407 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdbc6d01-f12c-4c32-97a6-3e070510c852 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '42d15fc0515945acacbbffcf2356c7a5', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80245) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:32:13.363977 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdbc6d01-f12c-4c32-97a6-3e070510c852 admin admin] REQUEST_METHOD: `PUT` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:32:13.364270 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdbc6d01-f12c-4c32-97a6-3e070510c852 admin admin] SCRIPT_NAME: `/identity` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:32:13.364856 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdbc6d01-f12c-4c32-97a6-3e070510c852 admin admin] PATH_INFO: `/v3/projects/d9023c942c2b442da1f0196e837fcddc/users/7dd619369a334560a0d8ce2cd06c3a48/roles/e29870f6e9e2411a9db0d5bf45f32ca5` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:32:13.389019 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdbc6d01-f12c-4c32-97a6-3e070510c852 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d9023c942c2b442da1f0196e837fcddc, user_id=7dd619369a334560a0d8ce2cd06c3a48, role_id=e29870f6e9e2411a9db0d5bf45f32ca5)` {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:32:13.392425 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdbc6d01-f12c-4c32-97a6-3e070510c852 admin admin] RBAC: Authorization granted {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:32:13.408654 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: [pid: 80245|app: 0|req: 1205/2404] 10.222.0.47 () {60 vars in 1482 bytes} [Thu Jul 21 08:32:13 2022] PUT /identity/v3/projects/d9023c942c2b442da1f0196e837fcddc/users/7dd619369a334560a0d8ce2cd06c3a48/roles/e29870f6e9e2411a9db0d5bf45f32ca5 => generated 0 bytes in 67 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 08:32:13.413675 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b301c12-3bf2-47f6-b6af-74ffafe993f8 admin admin] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:32:13.444144 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b301c12-3bf2-47f6-b6af-74ffafe993f8 admin admin] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:32:13.552185 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b1abfa0-d65f-41fa-8ad8-036c079f17f6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '42d15fc0515945acacbbffcf2356c7a5', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:32:13.552873 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b1abfa0-d65f-41fa-8ad8-036c079f17f6 admin admin] REQUEST_METHOD: `GET` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:32:13.553162 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b1abfa0-d65f-41fa-8ad8-036c079f17f6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:32:13.553438 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b1abfa0-d65f-41fa-8ad8-036c079f17f6 admin admin] PATH_INFO: `/v3/roles` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:32:13.554224 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b1abfa0-d65f-41fa-8ad8-036c079f17f6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:32:13.555882 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b1abfa0-d65f-41fa-8ad8-036c079f17f6 admin admin] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:32:13.570712 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 1200/2405] 10.222.0.47 () {58 vars in 1121 bytes} [Thu Jul 21 08:32:13 2022] GET /identity/v3/roles => generated 1420 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 08:32:13.571559 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdbc6d01-f12c-4c32-97a6-3e070510c852 admin admin] Authenticating user token {{(pid=80245) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:32:13.586411 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdbc6d01-f12c-4c32-97a6-3e070510c852 admin admin] Validating token access rules against request {{(pid=80245) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:32:13.592918 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a471857-3355-45a7-bd51-51f58d9d8d91 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '42d15fc0515945acacbbffcf2356c7a5', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80245) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:32:13.594232 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a471857-3355-45a7-bd51-51f58d9d8d91 admin admin] REQUEST_METHOD: `PUT` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:32:13.594513 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a471857-3355-45a7-bd51-51f58d9d8d91 admin admin] SCRIPT_NAME: `/identity` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:32:13.594792 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a471857-3355-45a7-bd51-51f58d9d8d91 admin admin] PATH_INFO: `/v3/projects/d9023c942c2b442da1f0196e837fcddc/users/7dd619369a334560a0d8ce2cd06c3a48/roles/e29870f6e9e2411a9db0d5bf45f32ca5` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:32:13.601485 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a471857-3355-45a7-bd51-51f58d9d8d91 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d9023c942c2b442da1f0196e837fcddc, user_id=7dd619369a334560a0d8ce2cd06c3a48, role_id=e29870f6e9e2411a9db0d5bf45f32ca5)` {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:32:13.605325 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a471857-3355-45a7-bd51-51f58d9d8d91 admin admin] RBAC: Authorization granted {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:32:13.619869 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: [pid: 80245|app: 0|req: 1206/2406] 10.222.0.47 () {60 vars in 1482 bytes} [Thu Jul 21 08:32:13 2022] PUT /identity/v3/projects/d9023c942c2b442da1f0196e837fcddc/users/7dd619369a334560a0d8ce2cd06c3a48/roles/e29870f6e9e2411a9db0d5bf45f32ca5 => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 21 08:32:13.638348 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12fdff56-f137-42f2-9de8-697df3b89847 None None] REQUEST_METHOD: `POST` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:32:13.638811 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12fdff56-f137-42f2-9de8-697df3b89847 None None] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:32:13.639392 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12fdff56-f137-42f2-9de8-697df3b89847 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:32:13.683368 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.auth.core [None req-12fdff56-f137-42f2-9de8-697df3b89847 None None] MFA Rules not processed for user `7dd619369a334560a0d8ce2cd06c3a48`. Rule list: `[]` (Enabled: `True`). {{(pid=80244) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 08:32:13.684293 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.fernet_utils [None req-12fdff56-f137-42f2-9de8-697df3b89847 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80244) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 08:32:13.715684 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG dbcounter [-] [80244] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80244) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:32:13.778604 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 1201/2407] 10.222.0.47 () {58 vars in 957 bytes} [Thu Jul 21 08:32:13 2022] POST /identity/v3/auth/tokens => generated 3339 bytes in 137 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 08:32:13.788602 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a471857-3355-45a7-bd51-51f58d9d8d91 admin admin] Authenticating user token {{(pid=80245) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:32:13.790395 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.fernet_utils [None req-3a471857-3355-45a7-bd51-51f58d9d8d91 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80245) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 08:32:13.859093 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a471857-3355-45a7-bd51-51f58d9d8d91 admin admin] Validating token access rules against request {{(pid=80245) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:32:13.873381 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7a02799-a9d0-4d9c-b1e9-722fa588dc8b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4f40410b7674687ab5362ac982248ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4faef13aa50e4e36b6c265fb1398fa14', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80245) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:32:13.873381 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7a02799-a9d0-4d9c-b1e9-722fa588dc8b service cinder] REQUEST_METHOD: `GET` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:32:13.873381 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7a02799-a9d0-4d9c-b1e9-722fa588dc8b service cinder] SCRIPT_NAME: `/identity` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:32:13.873381 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7a02799-a9d0-4d9c-b1e9-722fa588dc8b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:32:13.887828 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG dbcounter [-] [80245] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80245) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:32:13.894064 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7a02799-a9d0-4d9c-b1e9-722fa588dc8b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:32:13.897487 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7a02799-a9d0-4d9c-b1e9-722fa588dc8b service cinder] RBAC: Authorization granted {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:32:13.912113 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: [pid: 80245|app: 0|req: 1207/2408] 10.222.0.47 () {60 vars in 1365 bytes} [Thu Jul 21 08:32:13 2022] GET /identity/v3/auth/tokens => generated 3339 bytes in 124 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 08:32:14.357459 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12fdff56-f137-42f2-9de8-697df3b89847 None None] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:32:14.412316 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7a02799-a9d0-4d9c-b1e9-722fa588dc8b service cinder] Authenticating user token {{(pid=80245) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:32:14.413869 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.fernet_utils [None req-d7a02799-a9d0-4d9c-b1e9-722fa588dc8b service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80245) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 08:32:14.429054 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7a02799-a9d0-4d9c-b1e9-722fa588dc8b service cinder] Validating token access rules against request {{(pid=80245) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:32:14.433555 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9208e3a8-2121-421e-97ef-27eb4c23f63f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4f40410b7674687ab5362ac982248ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4faef13aa50e4e36b6c265fb1398fa14', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80245) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:32:14.434312 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9208e3a8-2121-421e-97ef-27eb4c23f63f service cinder] REQUEST_METHOD: `GET` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:32:14.434669 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9208e3a8-2121-421e-97ef-27eb4c23f63f service cinder] SCRIPT_NAME: `/identity` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:32:14.435016 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9208e3a8-2121-421e-97ef-27eb4c23f63f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:32:14.436927 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.fernet_utils [None req-9208e3a8-2121-421e-97ef-27eb4c23f63f service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80245) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 08:32:14.442202 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12fdff56-f137-42f2-9de8-697df3b89847 None None] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:32:14.447580 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08239fac-0300-4ad2-85d0-e3a3b8c9167c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '42d15fc0515945acacbbffcf2356c7a5', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:32:14.448467 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08239fac-0300-4ad2-85d0-e3a3b8c9167c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:32:14.448796 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08239fac-0300-4ad2-85d0-e3a3b8c9167c admin admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:32:14.449089 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08239fac-0300-4ad2-85d0-e3a3b8c9167c admin admin] PATH_INFO: `/v3/users/ef99762d947244edbf9514de634ed46f` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:32:14.450320 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08239fac-0300-4ad2-85d0-e3a3b8c9167c admin admin] RBAC: Authorizing `identity:delete_user(user_id=ef99762d947244edbf9514de634ed46f)` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:32:14.452392 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08239fac-0300-4ad2-85d0-e3a3b8c9167c admin admin] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:32:14.514124 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9208e3a8-2121-421e-97ef-27eb4c23f63f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:32:14.514388 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9208e3a8-2121-421e-97ef-27eb4c23f63f service cinder] RBAC: Authorization granted {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:32:14.517226 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.notifications [None req-08239fac-0300-4ad2-85d0-e3a3b8c9167c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ef99762d947244edbf9514de634ed46f'} {{(pid=80244) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 08:32:14.524461 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: [pid: 80245|app: 0|req: 1208/2409] 10.222.0.47 () {60 vars in 1365 bytes} [Thu Jul 21 08:32:14 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 113 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 21 08:32:14.528733 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.notifications [None req-08239fac-0300-4ad2-85d0-e3a3b8c9167c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ef99762d947244edbf9514de634ed46f'} {{(pid=80244) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 08:32:14.543139 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.notifications [None req-08239fac-0300-4ad2-85d0-e3a3b8c9167c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ef99762d947244edbf9514de634ed46f'} {{(pid=80244) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 08:32:14.543994 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.notifications [None req-08239fac-0300-4ad2-85d0-e3a3b8c9167c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ef99762d947244edbf9514de634ed46f'} {{(pid=80244) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 08:32:14.554232 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.notifications [None req-08239fac-0300-4ad2-85d0-e3a3b8c9167c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ef99762d947244edbf9514de634ed46f'} {{(pid=80244) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 08:32:14.566401 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 1202/2410] 10.222.0.47 () {58 vars in 1223 bytes} [Thu Jul 21 08:32:14 2022] DELETE /identity/v3/users/ef99762d947244edbf9514de634ed46f => generated 0 bytes in 206 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 08:32:14.568113 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cc4f0a0-fbb1-469c-9b9e-5af8e6cff51b None None] REQUEST_METHOD: `POST` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:32:14.568408 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cc4f0a0-fbb1-469c-9b9e-5af8e6cff51b None None] SCRIPT_NAME: `/identity` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:32:14.570135 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cc4f0a0-fbb1-469c-9b9e-5af8e6cff51b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:32:14.596795 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.auth.core [None req-6cc4f0a0-fbb1-469c-9b9e-5af8e6cff51b None None] MFA Rules not processed for user `2dce40b8d74c41ec94cc36e6ff05d412`. Rule list: `[]` (Enabled: `True`). {{(pid=80245) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 08:32:14.598130 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.fernet_utils [None req-6cc4f0a0-fbb1-469c-9b9e-5af8e6cff51b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80245) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 08:32:14.697022 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: [pid: 80245|app: 0|req: 1209/2411] 10.222.0.47 () {58 vars in 957 bytes} [Thu Jul 21 08:32:14 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 08:32:14.707822 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08239fac-0300-4ad2-85d0-e3a3b8c9167c admin admin] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:32:14.710517 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.fernet_utils [None req-08239fac-0300-4ad2-85d0-e3a3b8c9167c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80244) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 08:32:14.769867 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG dbcounter [-] [80244] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80244) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:32:14.804423 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08239fac-0300-4ad2-85d0-e3a3b8c9167c admin admin] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:32:14.809372 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f7358b6-dbe8-41b0-8e94-f8426b51120e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20ca3c3bd3774691b3374cfbfec6e494', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4faef13aa50e4e36b6c265fb1398fa14', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:32:14.810124 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f7358b6-dbe8-41b0-8e94-f8426b51120e service neutron] REQUEST_METHOD: `GET` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:32:14.810516 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f7358b6-dbe8-41b0-8e94-f8426b51120e service neutron] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:32:14.810896 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f7358b6-dbe8-41b0-8e94-f8426b51120e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:32:14.821486 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f7358b6-dbe8-41b0-8e94-f8426b51120e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:32:14.823577 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f7358b6-dbe8-41b0-8e94-f8426b51120e service neutron] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:32:14.836847 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 1203/2412] 10.222.0.47 () {60 vars in 1365 bytes} [Thu Jul 21 08:32:14 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 129 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 08:32:15.255489 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cc4f0a0-fbb1-469c-9b9e-5af8e6cff51b None None] Authenticating user token {{(pid=80245) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:32:15.277615 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.fernet_utils [None req-6cc4f0a0-fbb1-469c-9b9e-5af8e6cff51b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80245) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 08:32:15.305035 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cc4f0a0-fbb1-469c-9b9e-5af8e6cff51b None None] Validating token access rules against request {{(pid=80245) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:32:15.312730 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f266758-a8f0-4dc3-b6b6-dfcf42f821c4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '42d15fc0515945acacbbffcf2356c7a5', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80245) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:32:15.315546 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f266758-a8f0-4dc3-b6b6-dfcf42f821c4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:32:15.316638 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f266758-a8f0-4dc3-b6b6-dfcf42f821c4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:32:15.317080 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f266758-a8f0-4dc3-b6b6-dfcf42f821c4 admin admin] PATH_INFO: `/v3/projects/181f3e9749e94e98a6bcbff5be5d36d9` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:32:15.318538 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f266758-a8f0-4dc3-b6b6-dfcf42f821c4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=181f3e9749e94e98a6bcbff5be5d36d9)` {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:32:15.321565 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f266758-a8f0-4dc3-b6b6-dfcf42f821c4 admin admin] RBAC: Authorization granted {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:32:15.423412 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.notifications [None req-8f266758-a8f0-4dc3-b6b6-dfcf42f821c4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '181f3e9749e94e98a6bcbff5be5d36d9', 'request_id': 'req-8f266758-a8f0-4dc3-b6b6-dfcf42f821c4'} {{(pid=80245) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 08:32:15.430235 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.notifications [None req-8f266758-a8f0-4dc3-b6b6-dfcf42f821c4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '181f3e9749e94e98a6bcbff5be5d36d9', 'request_id': 'req-8f266758-a8f0-4dc3-b6b6-dfcf42f821c4'} {{(pid=80245) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 08:32:15.437514 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.notifications [None req-8f266758-a8f0-4dc3-b6b6-dfcf42f821c4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '181f3e9749e94e98a6bcbff5be5d36d9', 'request_id': 'req-8f266758-a8f0-4dc3-b6b6-dfcf42f821c4'} {{(pid=80245) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 08:32:15.451732 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.notifications [None req-8f266758-a8f0-4dc3-b6b6-dfcf42f821c4 admin admin] The token cache is being invalidate because project 181f3e9749e94e98a6bcbff5be5d36d9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80245) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 21 08:32:15.452026 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.notifications [None req-8f266758-a8f0-4dc3-b6b6-dfcf42f821c4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80245) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 08:32:15.452685 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.notifications [None req-8f266758-a8f0-4dc3-b6b6-dfcf42f821c4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80245) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 08:32:15.458751 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: [pid: 80245|app: 0|req: 1210/2413] 10.222.0.47 () {58 vars in 1232 bytes} [Thu Jul 21 08:32:15 2022] DELETE /identity/v3/projects/181f3e9749e94e98a6bcbff5be5d36d9 => generated 0 bytes in 207 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 08:32:15.459665 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f7358b6-dbe8-41b0-8e94-f8426b51120e service neutron] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:32:15.471963 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f7358b6-dbe8-41b0-8e94-f8426b51120e service neutron] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:32:15.506719 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f266758-a8f0-4dc3-b6b6-dfcf42f821c4 admin admin] Authenticating user token {{(pid=80245) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:32:15.508087 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.fernet_utils [None req-8f266758-a8f0-4dc3-b6b6-dfcf42f821c4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80245) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 08:32:15.531186 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG dbcounter [-] [80245] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80245) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:32:15.569974 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ef10a04-442f-461b-b9f9-d2e48322d70d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '42d15fc0515945acacbbffcf2356c7a5', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:32:15.571176 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ef10a04-442f-461b-b9f9-d2e48322d70d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:32:15.571508 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ef10a04-442f-461b-b9f9-d2e48322d70d admin admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:32:15.572456 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ef10a04-442f-461b-b9f9-d2e48322d70d admin admin] PATH_INFO: `/v3/users/7dd619369a334560a0d8ce2cd06c3a48` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:32:15.573987 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ef10a04-442f-461b-b9f9-d2e48322d70d admin admin] RBAC: Authorizing `identity:delete_user(user_id=7dd619369a334560a0d8ce2cd06c3a48)` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:32:15.576854 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ef10a04-442f-461b-b9f9-d2e48322d70d admin admin] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:32:15.584531 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f266758-a8f0-4dc3-b6b6-dfcf42f821c4 admin admin] Validating token access rules against request {{(pid=80245) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:32:15.590460 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eabd6d2-339b-4431-91f2-a56a2cb4f386 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4f40410b7674687ab5362ac982248ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4faef13aa50e4e36b6c265fb1398fa14', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80245) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:32:15.591208 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eabd6d2-339b-4431-91f2-a56a2cb4f386 service cinder] REQUEST_METHOD: `GET` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:32:15.592937 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eabd6d2-339b-4431-91f2-a56a2cb4f386 service cinder] SCRIPT_NAME: `/identity` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:32:15.596029 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eabd6d2-339b-4431-91f2-a56a2cb4f386 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:32:15.599538 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.fernet_utils [None req-3eabd6d2-339b-4431-91f2-a56a2cb4f386 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80245) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 08:32:15.658655 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.notifications [None req-8ef10a04-442f-461b-b9f9-d2e48322d70d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7dd619369a334560a0d8ce2cd06c3a48'} {{(pid=80244) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 08:32:15.681020 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.notifications [None req-8ef10a04-442f-461b-b9f9-d2e48322d70d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7dd619369a334560a0d8ce2cd06c3a48'} {{(pid=80244) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 08:32:15.701910 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3eabd6d2-339b-4431-91f2-a56a2cb4f386 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:32:15.704235 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3eabd6d2-339b-4431-91f2-a56a2cb4f386 service cinder] RBAC: Authorization granted {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:32:15.706569 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.notifications [None req-8ef10a04-442f-461b-b9f9-d2e48322d70d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7dd619369a334560a0d8ce2cd06c3a48'} {{(pid=80244) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 08:32:15.707444 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.notifications [None req-8ef10a04-442f-461b-b9f9-d2e48322d70d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7dd619369a334560a0d8ce2cd06c3a48'} {{(pid=80244) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 08:32:15.721526 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.notifications [None req-8ef10a04-442f-461b-b9f9-d2e48322d70d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7dd619369a334560a0d8ce2cd06c3a48'} {{(pid=80244) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 08:32:15.737875 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 1204/2414] 10.222.0.47 () {58 vars in 1223 bytes} [Thu Jul 21 08:32:15 2022] DELETE /identity/v3/users/7dd619369a334560a0d8ce2cd06c3a48 => generated 0 bytes in 275 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 21 08:32:15.741662 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: [pid: 80245|app: 0|req: 1211/2415] 10.222.0.47 () {60 vars in 1365 bytes} [Thu Jul 21 08:32:15 2022] GET /identity/v3/auth/tokens => generated 3311 bytes in 234 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 08:32:16.561078 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ef10a04-442f-461b-b9f9-d2e48322d70d admin admin] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:32:16.563529 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.fernet_utils [None req-8ef10a04-442f-461b-b9f9-d2e48322d70d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80244) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 08:32:16.663834 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ef10a04-442f-461b-b9f9-d2e48322d70d admin admin] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:32:16.669840 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbd41b0c-0eff-44a8-b60c-9ffa7dd54ee6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '42d15fc0515945acacbbffcf2356c7a5', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:32:16.672369 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbd41b0c-0eff-44a8-b60c-9ffa7dd54ee6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:32:16.673090 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbd41b0c-0eff-44a8-b60c-9ffa7dd54ee6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:32:16.673507 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbd41b0c-0eff-44a8-b60c-9ffa7dd54ee6 admin admin] PATH_INFO: `/v3/projects/d9023c942c2b442da1f0196e837fcddc` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:32:16.674996 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbd41b0c-0eff-44a8-b60c-9ffa7dd54ee6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d9023c942c2b442da1f0196e837fcddc)` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:32:16.676919 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbd41b0c-0eff-44a8-b60c-9ffa7dd54ee6 admin admin] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:32:16.679555 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG dbcounter [-] [80244] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80244) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:32:16.743261 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.notifications [None req-fbd41b0c-0eff-44a8-b60c-9ffa7dd54ee6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd9023c942c2b442da1f0196e837fcddc', 'request_id': 'req-fbd41b0c-0eff-44a8-b60c-9ffa7dd54ee6'} {{(pid=80244) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 08:32:16.758192 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.notifications [None req-fbd41b0c-0eff-44a8-b60c-9ffa7dd54ee6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd9023c942c2b442da1f0196e837fcddc', 'request_id': 'req-fbd41b0c-0eff-44a8-b60c-9ffa7dd54ee6'} {{(pid=80244) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 08:32:16.768380 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.notifications [None req-fbd41b0c-0eff-44a8-b60c-9ffa7dd54ee6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd9023c942c2b442da1f0196e837fcddc', 'request_id': 'req-fbd41b0c-0eff-44a8-b60c-9ffa7dd54ee6'} {{(pid=80244) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 08:32:16.796611 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.notifications [None req-fbd41b0c-0eff-44a8-b60c-9ffa7dd54ee6 admin admin] The token cache is being invalidate because project d9023c942c2b442da1f0196e837fcddc was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80244) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 21 08:32:16.796943 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.notifications [None req-fbd41b0c-0eff-44a8-b60c-9ffa7dd54ee6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80244) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 08:32:16.797693 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.notifications [None req-fbd41b0c-0eff-44a8-b60c-9ffa7dd54ee6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80244) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 08:32:16.799789 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 1205/2416] 10.222.0.47 () {58 vars in 1232 bytes} [Thu Jul 21 08:32:16 2022] DELETE /identity/v3/projects/d9023c942c2b442da1f0196e837fcddc => generated 0 bytes in 240 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 08:32:16.805349 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-919bd1b3-34c6-4fac-aecf-7f9d7bafc41b None None] REQUEST_METHOD: `POST` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:32:16.805829 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-919bd1b3-34c6-4fac-aecf-7f9d7bafc41b None None] SCRIPT_NAME: `/identity` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:32:16.805944 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-919bd1b3-34c6-4fac-aecf-7f9d7bafc41b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:32:16.827168 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.auth.core [None req-919bd1b3-34c6-4fac-aecf-7f9d7bafc41b None None] MFA Rules not processed for user `2dce40b8d74c41ec94cc36e6ff05d412`. Rule list: `[]` (Enabled: `True`). {{(pid=80245) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 08:32:16.829292 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.fernet_utils [None req-919bd1b3-34c6-4fac-aecf-7f9d7bafc41b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80245) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 08:32:16.958785 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: [pid: 80245|app: 0|req: 1212/2417] 10.222.0.47 () {58 vars in 957 bytes} [Thu Jul 21 08:32:16 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 155 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 08:32:16.966599 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53466152-0877-439c-8e44-250e08d1f046 None None] REQUEST_METHOD: `POST` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:32:16.966982 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53466152-0877-439c-8e44-250e08d1f046 None None] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:32:16.967361 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53466152-0877-439c-8e44-250e08d1f046 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:32:16.996082 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.auth.core [None req-53466152-0877-439c-8e44-250e08d1f046 None None] MFA Rules not processed for user `2dce40b8d74c41ec94cc36e6ff05d412`. Rule list: `[]` (Enabled: `True`). {{(pid=80244) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 08:32:16.998182 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.fernet_utils [None req-53466152-0877-439c-8e44-250e08d1f046 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80244) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 08:32:17.010588 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 1206/2418] 10.222.0.47 () {58 vars in 957 bytes} [Thu Jul 21 08:32:16 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 45 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 21 08:32:17.015038 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-919bd1b3-34c6-4fac-aecf-7f9d7bafc41b None None] Authenticating user token {{(pid=80245) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:32:17.025522 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-919bd1b3-34c6-4fac-aecf-7f9d7bafc41b None None] Validating token access rules against request {{(pid=80245) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:32:17.030930 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77857afc-c1ea-4b3a-90ba-591f05b1a06c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '42d15fc0515945acacbbffcf2356c7a5', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80245) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:32:17.031651 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77857afc-c1ea-4b3a-90ba-591f05b1a06c admin admin] REQUEST_METHOD: `GET` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:32:17.031936 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77857afc-c1ea-4b3a-90ba-591f05b1a06c admin admin] SCRIPT_NAME: `/identity` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:32:17.032208 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77857afc-c1ea-4b3a-90ba-591f05b1a06c admin admin] PATH_INFO: `/v3/domains` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:32:17.032937 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77857afc-c1ea-4b3a-90ba-591f05b1a06c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80245) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 08:32:17.033591 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77857afc-c1ea-4b3a-90ba-591f05b1a06c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:32:17.035350 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77857afc-c1ea-4b3a-90ba-591f05b1a06c admin admin] RBAC: Authorization granted {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:32:17.048426 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: [pid: 80245|app: 0|req: 1213/2419] 10.222.0.47 () {58 vars in 1152 bytes} [Thu Jul 21 08:32:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 08:32:17.053108 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53466152-0877-439c-8e44-250e08d1f046 None None] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:32:17.067569 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53466152-0877-439c-8e44-250e08d1f046 None None] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:32:17.072881 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0164e43-2010-4f80-92a8-9511b0d7835b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '42d15fc0515945acacbbffcf2356c7a5', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:32:17.073669 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0164e43-2010-4f80-92a8-9511b0d7835b admin admin] REQUEST_METHOD: `POST` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:32:17.074048 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0164e43-2010-4f80-92a8-9511b0d7835b admin admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:32:17.074434 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0164e43-2010-4f80-92a8-9511b0d7835b admin admin] PATH_INFO: `/v3/projects` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:32:17.075231 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0164e43-2010-4f80-92a8-9511b0d7835b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:32:17.077326 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0164e43-2010-4f80-92a8-9511b0d7835b admin admin] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:32:17.128581 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 1207/2420] 10.222.0.47 () {60 vars in 1152 bytes} [Thu Jul 21 08:32:17 2022] POST /identity/v3/projects => generated 403 bytes in 76 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 08:32:17.132641 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77857afc-c1ea-4b3a-90ba-591f05b1a06c admin admin] Authenticating user token {{(pid=80245) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:32:17.146288 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77857afc-c1ea-4b3a-90ba-591f05b1a06c admin admin] Validating token access rules against request {{(pid=80245) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:32:17.264658 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG dbcounter [-] [80245] Writing DB stats keystone:SELECT=100 {{(pid=80245) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:32:17.290072 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cf1138d-c903-4dba-8e8d-8f59d83acbb8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '42d15fc0515945acacbbffcf2356c7a5', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80245) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:32:17.309871 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cf1138d-c903-4dba-8e8d-8f59d83acbb8 admin admin] REQUEST_METHOD: `POST` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:32:17.309871 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cf1138d-c903-4dba-8e8d-8f59d83acbb8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:32:17.309871 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cf1138d-c903-4dba-8e8d-8f59d83acbb8 admin admin] PATH_INFO: `/v3/users` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:32:17.315012 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cf1138d-c903-4dba-8e8d-8f59d83acbb8 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:32:17.320357 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cf1138d-c903-4dba-8e8d-8f59d83acbb8 admin admin] RBAC: Authorization granted {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:32:17.375631 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: [pid: 80245|app: 0|req: 1214/2421] 10.222.0.47 () {60 vars in 1143 bytes} [Thu Jul 21 08:32:17 2022] POST /identity/v3/users => generated 302 bytes in 241 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 08:32:17.376931 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0164e43-2010-4f80-92a8-9511b0d7835b admin admin] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:32:17.390711 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0164e43-2010-4f80-92a8-9511b0d7835b admin admin] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:32:17.397304 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55748d30-b290-4a3a-b05f-d8862df680fb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '42d15fc0515945acacbbffcf2356c7a5', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:32:17.398117 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55748d30-b290-4a3a-b05f-d8862df680fb admin admin] REQUEST_METHOD: `GET` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:32:17.398501 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55748d30-b290-4a3a-b05f-d8862df680fb admin admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:32:17.398879 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55748d30-b290-4a3a-b05f-d8862df680fb admin admin] PATH_INFO: `/v3/roles` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:32:17.399762 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55748d30-b290-4a3a-b05f-d8862df680fb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:32:17.402004 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55748d30-b290-4a3a-b05f-d8862df680fb admin admin] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:32:17.413886 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 1208/2422] 10.222.0.47 () {58 vars in 1121 bytes} [Thu Jul 21 08:32:17 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 08:32:17.418452 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cf1138d-c903-4dba-8e8d-8f59d83acbb8 admin admin] Authenticating user token {{(pid=80245) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:32:17.431303 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cf1138d-c903-4dba-8e8d-8f59d83acbb8 admin admin] Validating token access rules against request {{(pid=80245) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:32:17.436261 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3933db35-f8f6-4024-9c0e-5f789bd4f625 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '42d15fc0515945acacbbffcf2356c7a5', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80245) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:32:17.437113 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3933db35-f8f6-4024-9c0e-5f789bd4f625 admin admin] REQUEST_METHOD: `GET` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:32:17.437500 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3933db35-f8f6-4024-9c0e-5f789bd4f625 admin admin] SCRIPT_NAME: `/identity` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:32:17.437987 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3933db35-f8f6-4024-9c0e-5f789bd4f625 admin admin] PATH_INFO: `/v3/roles` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:32:17.438944 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3933db35-f8f6-4024-9c0e-5f789bd4f625 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:32:17.440936 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3933db35-f8f6-4024-9c0e-5f789bd4f625 admin admin] RBAC: Authorization granted {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:32:17.466266 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: [pid: 80245|app: 0|req: 1215/2423] 10.222.0.47 () {58 vars in 1121 bytes} [Thu Jul 21 08:32:17 2022] GET /identity/v3/roles => generated 1420 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 08:32:17.475585 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55748d30-b290-4a3a-b05f-d8862df680fb admin admin] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:32:17.490724 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55748d30-b290-4a3a-b05f-d8862df680fb admin admin] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:32:17.501258 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa6a0324-9a26-49c3-bfbe-d8c9e1d11867 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '42d15fc0515945acacbbffcf2356c7a5', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:32:17.502581 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa6a0324-9a26-49c3-bfbe-d8c9e1d11867 admin admin] REQUEST_METHOD: `PUT` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:32:17.503298 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa6a0324-9a26-49c3-bfbe-d8c9e1d11867 admin admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:32:17.503766 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa6a0324-9a26-49c3-bfbe-d8c9e1d11867 admin admin] PATH_INFO: `/v3/projects/c4df09e49b19415683519a57c34bdcc9/users/867e21619d6447d78f87058f77b1f90a/roles/3a074738221e4601b5205715ba87d240` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:32:17.529208 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa6a0324-9a26-49c3-bfbe-d8c9e1d11867 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c4df09e49b19415683519a57c34bdcc9, user_id=867e21619d6447d78f87058f77b1f90a, role_id=3a074738221e4601b5205715ba87d240)` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:32:17.532777 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa6a0324-9a26-49c3-bfbe-d8c9e1d11867 admin admin] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:32:17.549829 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 1209/2424] 10.222.0.47 () {60 vars in 1482 bytes} [Thu Jul 21 08:32:17 2022] PUT /identity/v3/projects/c4df09e49b19415683519a57c34bdcc9/users/867e21619d6447d78f87058f77b1f90a/roles/3a074738221e4601b5205715ba87d240 => generated 0 bytes in 80 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 08:32:17.554856 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6751b5ff-066c-4a61-8b4c-d4a570606221 None None] REQUEST_METHOD: `POST` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:32:17.555153 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6751b5ff-066c-4a61-8b4c-d4a570606221 None None] SCRIPT_NAME: `/identity` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:32:17.555533 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6751b5ff-066c-4a61-8b4c-d4a570606221 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:32:17.574464 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.auth.core [None req-6751b5ff-066c-4a61-8b4c-d4a570606221 None None] MFA Rules not processed for user `2dce40b8d74c41ec94cc36e6ff05d412`. Rule list: `[]` (Enabled: `True`). {{(pid=80245) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 08:32:17.575709 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.fernet_utils [None req-6751b5ff-066c-4a61-8b4c-d4a570606221 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80245) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 08:32:17.643910 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: [pid: 80245|app: 0|req: 1216/2425] 10.222.0.47 () {58 vars in 957 bytes} [Thu Jul 21 08:32:17 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 08:32:17.654958 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa6a0324-9a26-49c3-bfbe-d8c9e1d11867 admin admin] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:32:17.656828 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.fernet_utils [None req-fa6a0324-9a26-49c3-bfbe-d8c9e1d11867 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80244) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 08:32:17.718768 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa6a0324-9a26-49c3-bfbe-d8c9e1d11867 admin admin] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:32:17.722553 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1443c691-7afc-46b7-ae4a-49b03d2a7a83 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20ca3c3bd3774691b3374cfbfec6e494', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4faef13aa50e4e36b6c265fb1398fa14', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:32:17.723202 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1443c691-7afc-46b7-ae4a-49b03d2a7a83 service neutron] REQUEST_METHOD: `GET` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:32:17.723518 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1443c691-7afc-46b7-ae4a-49b03d2a7a83 service neutron] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:32:17.723793 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1443c691-7afc-46b7-ae4a-49b03d2a7a83 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:32:17.733535 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1443c691-7afc-46b7-ae4a-49b03d2a7a83 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:32:17.734964 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1443c691-7afc-46b7-ae4a-49b03d2a7a83 service neutron] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:32:17.744955 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 1210/2426] 10.222.0.47 () {60 vars in 1365 bytes} [Thu Jul 21 08:32:17 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 08:32:18.411663 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8998b4b7-8810-4c4f-8b88-c4a385c99711 None None] REQUEST_METHOD: `POST` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:32:18.412639 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8998b4b7-8810-4c4f-8b88-c4a385c99711 None None] SCRIPT_NAME: `/identity` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:32:18.413177 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8998b4b7-8810-4c4f-8b88-c4a385c99711 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:32:18.454707 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.auth.core [None req-8998b4b7-8810-4c4f-8b88-c4a385c99711 None None] MFA Rules not processed for user `867e21619d6447d78f87058f77b1f90a`. Rule list: `[]` (Enabled: `True`). {{(pid=80245) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 08:32:18.456763 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.fernet_utils [None req-8998b4b7-8810-4c4f-8b88-c4a385c99711 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80245) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 08:32:18.571374 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: [pid: 80245|app: 0|req: 1217/2427] 10.222.0.47 () {58 vars in 957 bytes} [Thu Jul 21 08:32:18 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 158 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 08:32:18.574646 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1443c691-7afc-46b7-ae4a-49b03d2a7a83 service neutron] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:32:18.589007 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1443c691-7afc-46b7-ae4a-49b03d2a7a83 service neutron] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:32:18.595557 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1840a1ef-ebea-4bf2-b0ba-c7d62d0bf989 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '42d15fc0515945acacbbffcf2356c7a5', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:32:18.596429 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1840a1ef-ebea-4bf2-b0ba-c7d62d0bf989 admin admin] REQUEST_METHOD: `POST` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:32:18.597245 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1840a1ef-ebea-4bf2-b0ba-c7d62d0bf989 admin admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:32:18.597558 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1840a1ef-ebea-4bf2-b0ba-c7d62d0bf989 admin admin] PATH_INFO: `/v3/projects` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:32:18.598456 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1840a1ef-ebea-4bf2-b0ba-c7d62d0bf989 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:32:18.600842 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1840a1ef-ebea-4bf2-b0ba-c7d62d0bf989 admin admin] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:32:18.651159 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 1211/2428] 10.222.0.47 () {60 vars in 1152 bytes} [Thu Jul 21 08:32:18 2022] POST /identity/v3/projects => generated 403 bytes in 74 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 08:32:18.652354 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8998b4b7-8810-4c4f-8b88-c4a385c99711 None None] Authenticating user token {{(pid=80245) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:32:18.664210 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8998b4b7-8810-4c4f-8b88-c4a385c99711 None None] Validating token access rules against request {{(pid=80245) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:32:18.764386 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b65b36b8-6de1-4ca6-801e-fc172a711ca9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '42d15fc0515945acacbbffcf2356c7a5', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80245) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:32:18.767876 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b65b36b8-6de1-4ca6-801e-fc172a711ca9 admin admin] REQUEST_METHOD: `POST` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:32:18.768301 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b65b36b8-6de1-4ca6-801e-fc172a711ca9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:32:18.768786 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b65b36b8-6de1-4ca6-801e-fc172a711ca9 admin admin] PATH_INFO: `/v3/users` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:32:18.769656 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b65b36b8-6de1-4ca6-801e-fc172a711ca9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:32:18.771746 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b65b36b8-6de1-4ca6-801e-fc172a711ca9 admin admin] RBAC: Authorization granted {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:32:18.783111 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG dbcounter [-] [80245] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80245) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:32:18.798389 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: [pid: 80245|app: 0|req: 1218/2429] 10.222.0.47 () {60 vars in 1143 bytes} [Thu Jul 21 08:32:18 2022] POST /identity/v3/users => generated 308 bytes in 147 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 21 08:32:18.802387 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1840a1ef-ebea-4bf2-b0ba-c7d62d0bf989 admin admin] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:32:18.816048 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1840a1ef-ebea-4bf2-b0ba-c7d62d0bf989 admin admin] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:32:18.821517 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a330370-3982-49dc-aa7e-bd2e03a0ca0b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '42d15fc0515945acacbbffcf2356c7a5', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:32:18.822420 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a330370-3982-49dc-aa7e-bd2e03a0ca0b admin admin] REQUEST_METHOD: `GET` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:32:18.822734 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a330370-3982-49dc-aa7e-bd2e03a0ca0b admin admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:32:18.823010 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a330370-3982-49dc-aa7e-bd2e03a0ca0b admin admin] PATH_INFO: `/v3/roles` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:32:18.823812 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a330370-3982-49dc-aa7e-bd2e03a0ca0b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:32:18.825411 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a330370-3982-49dc-aa7e-bd2e03a0ca0b admin admin] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:32:18.828226 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG dbcounter [-] [80244] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=80244) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:32:18.840839 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 1212/2430] 10.222.0.47 () {58 vars in 1121 bytes} [Thu Jul 21 08:32:18 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 08:32:18.843813 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b65b36b8-6de1-4ca6-801e-fc172a711ca9 admin admin] Authenticating user token {{(pid=80245) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:32:18.856998 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b65b36b8-6de1-4ca6-801e-fc172a711ca9 admin admin] Validating token access rules against request {{(pid=80245) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:32:18.862174 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf2a9e5c-faeb-4329-9cfb-1889ad6c941a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '42d15fc0515945acacbbffcf2356c7a5', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80245) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:32:18.863179 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf2a9e5c-faeb-4329-9cfb-1889ad6c941a admin admin] REQUEST_METHOD: `PUT` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:32:18.863434 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf2a9e5c-faeb-4329-9cfb-1889ad6c941a admin admin] SCRIPT_NAME: `/identity` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:32:18.864008 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf2a9e5c-faeb-4329-9cfb-1889ad6c941a admin admin] PATH_INFO: `/v3/projects/3207b455ea9245a49e83eaf5e9e1cd2f/users/e35f2683d0634657a1326fe92d1e9a61/roles/e29870f6e9e2411a9db0d5bf45f32ca5` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:32:18.883388 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf2a9e5c-faeb-4329-9cfb-1889ad6c941a admin admin] RBAC: Authorizing `identity:create_grant(project_id=3207b455ea9245a49e83eaf5e9e1cd2f, user_id=e35f2683d0634657a1326fe92d1e9a61, role_id=e29870f6e9e2411a9db0d5bf45f32ca5)` {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:32:18.888361 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf2a9e5c-faeb-4329-9cfb-1889ad6c941a admin admin] RBAC: Authorization granted {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:32:18.904935 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: [pid: 80245|app: 0|req: 1219/2431] 10.222.0.47 () {60 vars in 1482 bytes} [Thu Jul 21 08:32:18 2022] PUT /identity/v3/projects/3207b455ea9245a49e83eaf5e9e1cd2f/users/e35f2683d0634657a1326fe92d1e9a61/roles/e29870f6e9e2411a9db0d5bf45f32ca5 => generated 0 bytes in 62 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 08:32:18.926330 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a330370-3982-49dc-aa7e-bd2e03a0ca0b admin admin] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:32:18.937228 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a330370-3982-49dc-aa7e-bd2e03a0ca0b admin admin] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:32:19.000757 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3378869c-a7ee-4abe-8de6-aaff2cacae14 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '42d15fc0515945acacbbffcf2356c7a5', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:32:19.001438 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3378869c-a7ee-4abe-8de6-aaff2cacae14 admin admin] REQUEST_METHOD: `GET` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:32:19.001724 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3378869c-a7ee-4abe-8de6-aaff2cacae14 admin admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:32:19.002007 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3378869c-a7ee-4abe-8de6-aaff2cacae14 admin admin] PATH_INFO: `/v3/roles` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:32:19.002787 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3378869c-a7ee-4abe-8de6-aaff2cacae14 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:32:19.004695 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3378869c-a7ee-4abe-8de6-aaff2cacae14 admin admin] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:32:19.014935 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 1213/2432] 10.222.0.47 () {58 vars in 1121 bytes} [Thu Jul 21 08:32:18 2022] GET /identity/v3/roles => generated 1420 bytes in 106 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 08:32:19.019262 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf2a9e5c-faeb-4329-9cfb-1889ad6c941a admin admin] Authenticating user token {{(pid=80245) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:32:19.031028 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf2a9e5c-faeb-4329-9cfb-1889ad6c941a admin admin] Validating token access rules against request {{(pid=80245) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:32:19.035734 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d69973c0-b670-4b5a-95f9-4f9dcb3fbcf3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '42d15fc0515945acacbbffcf2356c7a5', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80245) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:32:19.036624 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d69973c0-b670-4b5a-95f9-4f9dcb3fbcf3 admin admin] REQUEST_METHOD: `PUT` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:32:19.036910 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d69973c0-b670-4b5a-95f9-4f9dcb3fbcf3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:32:19.037187 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d69973c0-b670-4b5a-95f9-4f9dcb3fbcf3 admin admin] PATH_INFO: `/v3/projects/3207b455ea9245a49e83eaf5e9e1cd2f/users/e35f2683d0634657a1326fe92d1e9a61/roles/e29870f6e9e2411a9db0d5bf45f32ca5` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:32:19.039147 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d69973c0-b670-4b5a-95f9-4f9dcb3fbcf3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3207b455ea9245a49e83eaf5e9e1cd2f, user_id=e35f2683d0634657a1326fe92d1e9a61, role_id=e29870f6e9e2411a9db0d5bf45f32ca5)` {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:32:19.042547 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d69973c0-b670-4b5a-95f9-4f9dcb3fbcf3 admin admin] RBAC: Authorization granted {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:32:19.053022 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: [pid: 80245|app: 0|req: 1220/2433] 10.222.0.47 () {60 vars in 1482 bytes} [Thu Jul 21 08:32:19 2022] PUT /identity/v3/projects/3207b455ea9245a49e83eaf5e9e1cd2f/users/e35f2683d0634657a1326fe92d1e9a61/roles/e29870f6e9e2411a9db0d5bf45f32ca5 => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 08:32:19.745113 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e370d35-bc49-40a5-94b5-5835cb93633d None None] REQUEST_METHOD: `POST` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:32:19.746573 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e370d35-bc49-40a5-94b5-5835cb93633d None None] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:32:19.747218 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e370d35-bc49-40a5-94b5-5835cb93633d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:32:19.794409 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.auth.core [None req-8e370d35-bc49-40a5-94b5-5835cb93633d None None] MFA Rules not processed for user `e35f2683d0634657a1326fe92d1e9a61`. Rule list: `[]` (Enabled: `True`). {{(pid=80244) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 08:32:19.796475 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.fernet_utils [None req-8e370d35-bc49-40a5-94b5-5835cb93633d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80244) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 08:32:19.911665 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 1214/2434] 10.222.0.47 () {58 vars in 957 bytes} [Thu Jul 21 08:32:19 2022] POST /identity/v3/auth/tokens => generated 3337 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 08:32:20.056459 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d69973c0-b670-4b5a-95f9-4f9dcb3fbcf3 admin admin] Authenticating user token {{(pid=80245) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:32:20.058701 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.fernet_utils [None req-d69973c0-b670-4b5a-95f9-4f9dcb3fbcf3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80245) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 08:32:20.140072 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d69973c0-b670-4b5a-95f9-4f9dcb3fbcf3 admin admin] Validating token access rules against request {{(pid=80245) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:32:20.146100 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38cc080d-84fc-4bbb-b275-3ce19f30071a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '42d15fc0515945acacbbffcf2356c7a5', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80245) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:32:20.147311 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38cc080d-84fc-4bbb-b275-3ce19f30071a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:32:20.147848 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38cc080d-84fc-4bbb-b275-3ce19f30071a admin admin] SCRIPT_NAME: `/identity` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:32:20.148321 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38cc080d-84fc-4bbb-b275-3ce19f30071a admin admin] PATH_INFO: `/v3/users/d1ef300402cc473d8ccae6ee4d02f698` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:32:20.149886 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38cc080d-84fc-4bbb-b275-3ce19f30071a admin admin] RBAC: Authorizing `identity:delete_user(user_id=d1ef300402cc473d8ccae6ee4d02f698)` {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:32:20.152987 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38cc080d-84fc-4bbb-b275-3ce19f30071a admin admin] RBAC: Authorization granted {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:32:20.224222 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.notifications [None req-38cc080d-84fc-4bbb-b275-3ce19f30071a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd1ef300402cc473d8ccae6ee4d02f698'} {{(pid=80245) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 08:32:20.232908 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.notifications [None req-38cc080d-84fc-4bbb-b275-3ce19f30071a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd1ef300402cc473d8ccae6ee4d02f698'} {{(pid=80245) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 08:32:20.247764 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.notifications [None req-38cc080d-84fc-4bbb-b275-3ce19f30071a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd1ef300402cc473d8ccae6ee4d02f698'} {{(pid=80245) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 08:32:20.272410 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.notifications [None req-38cc080d-84fc-4bbb-b275-3ce19f30071a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd1ef300402cc473d8ccae6ee4d02f698'} {{(pid=80245) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 08:32:20.281303 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.notifications [None req-38cc080d-84fc-4bbb-b275-3ce19f30071a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd1ef300402cc473d8ccae6ee4d02f698'} {{(pid=80245) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 08:32:20.289131 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: [pid: 80245|app: 0|req: 1221/2435] 10.222.0.47 () {58 vars in 1223 bytes} [Thu Jul 21 08:32:20 2022] DELETE /identity/v3/users/d1ef300402cc473d8ccae6ee4d02f698 => generated 0 bytes in 234 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 08:32:20.294633 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e4a6588-3bb4-455c-9709-bc0fb3304815 None None] REQUEST_METHOD: `POST` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:32:20.294973 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e4a6588-3bb4-455c-9709-bc0fb3304815 None None] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:32:20.295260 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e4a6588-3bb4-455c-9709-bc0fb3304815 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:32:20.326113 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.auth.core [None req-8e4a6588-3bb4-455c-9709-bc0fb3304815 None None] MFA Rules not processed for user `2dce40b8d74c41ec94cc36e6ff05d412`. Rule list: `[]` (Enabled: `True`). {{(pid=80244) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 08:32:20.331041 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.fernet_utils [None req-8e4a6588-3bb4-455c-9709-bc0fb3304815 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80244) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 08:32:20.417834 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 1215/2436] 10.222.0.47 () {58 vars in 957 bytes} [Thu Jul 21 08:32:20 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 08:32:20.428071 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38cc080d-84fc-4bbb-b275-3ce19f30071a admin admin] Authenticating user token {{(pid=80245) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:32:20.429577 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.fernet_utils [None req-38cc080d-84fc-4bbb-b275-3ce19f30071a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80245) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 08:32:20.477257 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG dbcounter [-] [80245] Writing DB stats keystone:INSERT=4,keystone:SELECT=86,keystone:DELETE=10 {{(pid=80245) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:32:20.502917 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38cc080d-84fc-4bbb-b275-3ce19f30071a admin admin] Validating token access rules against request {{(pid=80245) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:32:20.508226 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c8484aa-19bb-42ed-8e6b-feb810c9022a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20ca3c3bd3774691b3374cfbfec6e494', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4faef13aa50e4e36b6c265fb1398fa14', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80245) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:32:20.508928 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c8484aa-19bb-42ed-8e6b-feb810c9022a service neutron] REQUEST_METHOD: `GET` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:32:20.509185 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c8484aa-19bb-42ed-8e6b-feb810c9022a service neutron] SCRIPT_NAME: `/identity` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:32:20.510040 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c8484aa-19bb-42ed-8e6b-feb810c9022a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:32:20.524555 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c8484aa-19bb-42ed-8e6b-feb810c9022a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:32:20.526061 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c8484aa-19bb-42ed-8e6b-feb810c9022a service neutron] RBAC: Authorization granted {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:32:20.537074 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: [pid: 80245|app: 0|req: 1222/2437] 10.222.0.47 () {60 vars in 1365 bytes} [Thu Jul 21 08:32:20 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 110 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 21 08:32:20.872064 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e4a6588-3bb4-455c-9709-bc0fb3304815 None None] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:32:20.873791 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.fernet_utils [None req-8e4a6588-3bb4-455c-9709-bc0fb3304815 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80244) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 08:32:20.889403 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e4a6588-3bb4-455c-9709-bc0fb3304815 None None] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:32:20.895178 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faffc3ed-a7f7-4002-a1b5-43353316dc11 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '42d15fc0515945acacbbffcf2356c7a5', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:32:20.896342 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faffc3ed-a7f7-4002-a1b5-43353316dc11 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:32:20.896979 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faffc3ed-a7f7-4002-a1b5-43353316dc11 admin admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:32:20.897236 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faffc3ed-a7f7-4002-a1b5-43353316dc11 admin admin] PATH_INFO: `/v3/projects/fbbdc782b5fe4c5ca61ba9e149fb94fe` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:32:20.912574 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faffc3ed-a7f7-4002-a1b5-43353316dc11 admin admin] RBAC: Authorizing `identity:delete_project(project_id=fbbdc782b5fe4c5ca61ba9e149fb94fe)` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:32:20.918409 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faffc3ed-a7f7-4002-a1b5-43353316dc11 admin admin] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:32:20.938699 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG dbcounter [-] [80244] Writing DB stats keystone:SELECT=100 {{(pid=80244) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:32:20.955863 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c8484aa-19bb-42ed-8e6b-feb810c9022a service neutron] Authenticating user token {{(pid=80245) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:32:20.957551 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.fernet_utils [None req-2c8484aa-19bb-42ed-8e6b-feb810c9022a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80245) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 08:32:20.981413 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c8484aa-19bb-42ed-8e6b-feb810c9022a service neutron] Validating token access rules against request {{(pid=80245) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:32:20.991790 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdff16d9-68be-4615-82b8-2480c957a5b0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '42d15fc0515945acacbbffcf2356c7a5', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80245) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:32:20.993083 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdff16d9-68be-4615-82b8-2480c957a5b0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:32:20.994171 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdff16d9-68be-4615-82b8-2480c957a5b0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:32:20.994604 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdff16d9-68be-4615-82b8-2480c957a5b0 admin admin] PATH_INFO: `/v3/users/1b3e7b164a7a49a9a7318072f5ba523c` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:32:20.996549 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdff16d9-68be-4615-82b8-2480c957a5b0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=1b3e7b164a7a49a9a7318072f5ba523c)` {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:32:20.998623 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdff16d9-68be-4615-82b8-2480c957a5b0 admin admin] RBAC: Authorization granted {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:32:21.018195 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.notifications [None req-faffc3ed-a7f7-4002-a1b5-43353316dc11 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'fbbdc782b5fe4c5ca61ba9e149fb94fe', 'request_id': 'req-faffc3ed-a7f7-4002-a1b5-43353316dc11'} {{(pid=80244) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 08:32:21.041049 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.notifications [None req-faffc3ed-a7f7-4002-a1b5-43353316dc11 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'fbbdc782b5fe4c5ca61ba9e149fb94fe', 'request_id': 'req-faffc3ed-a7f7-4002-a1b5-43353316dc11'} {{(pid=80244) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 08:32:21.054145 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.notifications [None req-faffc3ed-a7f7-4002-a1b5-43353316dc11 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'fbbdc782b5fe4c5ca61ba9e149fb94fe', 'request_id': 'req-faffc3ed-a7f7-4002-a1b5-43353316dc11'} {{(pid=80244) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 08:32:21.071583 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.notifications [None req-faffc3ed-a7f7-4002-a1b5-43353316dc11 admin admin] The token cache is being invalidate because project fbbdc782b5fe4c5ca61ba9e149fb94fe was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80244) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 21 08:32:21.072318 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.notifications [None req-faffc3ed-a7f7-4002-a1b5-43353316dc11 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80244) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 08:32:21.073156 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.notifications [None req-faffc3ed-a7f7-4002-a1b5-43353316dc11 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80244) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 08:32:21.076950 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 1216/2438] 10.222.0.47 () {58 vars in 1232 bytes} [Thu Jul 21 08:32:20 2022] DELETE /identity/v3/projects/fbbdc782b5fe4c5ca61ba9e149fb94fe => generated 0 bytes in 206 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 08:32:21.081199 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faffc3ed-a7f7-4002-a1b5-43353316dc11 admin admin] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:32:21.081453 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.notifications [None req-cdff16d9-68be-4615-82b8-2480c957a5b0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1b3e7b164a7a49a9a7318072f5ba523c'} {{(pid=80245) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 08:32:21.083431 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.fernet_utils [None req-faffc3ed-a7f7-4002-a1b5-43353316dc11 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80244) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 08:32:21.094588 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.notifications [None req-cdff16d9-68be-4615-82b8-2480c957a5b0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1b3e7b164a7a49a9a7318072f5ba523c'} {{(pid=80245) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 08:32:21.112966 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.notifications [None req-cdff16d9-68be-4615-82b8-2480c957a5b0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1b3e7b164a7a49a9a7318072f5ba523c'} {{(pid=80245) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 08:32:21.122260 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.notifications [None req-cdff16d9-68be-4615-82b8-2480c957a5b0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1b3e7b164a7a49a9a7318072f5ba523c'} {{(pid=80245) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 08:32:21.122936 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.notifications [None req-cdff16d9-68be-4615-82b8-2480c957a5b0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1b3e7b164a7a49a9a7318072f5ba523c'} {{(pid=80245) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 08:32:21.130844 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: [pid: 80245|app: 0|req: 1223/2439] 10.222.0.47 () {58 vars in 1223 bytes} [Thu Jul 21 08:32:20 2022] DELETE /identity/v3/users/1b3e7b164a7a49a9a7318072f5ba523c => generated 0 bytes in 176 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 21 08:32:21.138212 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73b63306-53b9-49f7-8976-3b5af312c2f1 None None] REQUEST_METHOD: `POST` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:32:21.138792 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73b63306-53b9-49f7-8976-3b5af312c2f1 None None] SCRIPT_NAME: `/identity` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:32:21.139799 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73b63306-53b9-49f7-8976-3b5af312c2f1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:32:21.159477 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.auth.core [None req-73b63306-53b9-49f7-8976-3b5af312c2f1 None None] MFA Rules not processed for user `2dce40b8d74c41ec94cc36e6ff05d412`. Rule list: `[]` (Enabled: `True`). {{(pid=80245) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 08:32:21.160936 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.fernet_utils [None req-73b63306-53b9-49f7-8976-3b5af312c2f1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80245) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 08:32:21.161270 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faffc3ed-a7f7-4002-a1b5-43353316dc11 admin admin] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:32:21.232747 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d649cdcf-804b-4411-b4a9-b85885fa02c4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '42d15fc0515945acacbbffcf2356c7a5', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:32:21.233806 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d649cdcf-804b-4411-b4a9-b85885fa02c4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:32:21.234085 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d649cdcf-804b-4411-b4a9-b85885fa02c4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:32:21.234423 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d649cdcf-804b-4411-b4a9-b85885fa02c4 admin admin] PATH_INFO: `/v3/users/7d24f72b48d04f3bb94401f05d838527` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:32:21.235714 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d649cdcf-804b-4411-b4a9-b85885fa02c4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=7d24f72b48d04f3bb94401f05d838527)` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:32:21.240545 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: [pid: 80245|app: 0|req: 1224/2440] 10.222.0.47 () {58 vars in 957 bytes} [Thu Jul 21 08:32:21 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 21 08:32:21.240740 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d649cdcf-804b-4411-b4a9-b85885fa02c4 admin admin] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:32:21.248297 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73b63306-53b9-49f7-8976-3b5af312c2f1 None None] Authenticating user token {{(pid=80245) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:32:21.251408 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.fernet_utils [None req-73b63306-53b9-49f7-8976-3b5af312c2f1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80245) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 08:32:21.324305 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.notifications [None req-d649cdcf-804b-4411-b4a9-b85885fa02c4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7d24f72b48d04f3bb94401f05d838527'} {{(pid=80244) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 08:32:21.332361 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG dbcounter [-] [80244] Writing DB stats keystone:SELECT=83,keystone:DELETE=15,keystone:INSERT=2 {{(pid=80244) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:32:21.339048 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73b63306-53b9-49f7-8976-3b5af312c2f1 None None] Validating token access rules against request {{(pid=80245) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:32:21.342308 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.notifications [None req-d649cdcf-804b-4411-b4a9-b85885fa02c4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7d24f72b48d04f3bb94401f05d838527'} {{(pid=80244) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 08:32:21.348203 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a409b99e-8a1c-4d31-a339-81229351c67c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20ca3c3bd3774691b3374cfbfec6e494', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4faef13aa50e4e36b6c265fb1398fa14', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80245) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:32:21.349356 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a409b99e-8a1c-4d31-a339-81229351c67c service neutron] REQUEST_METHOD: `GET` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:32:21.350517 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a409b99e-8a1c-4d31-a339-81229351c67c service neutron] SCRIPT_NAME: `/identity` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:32:21.351589 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a409b99e-8a1c-4d31-a339-81229351c67c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:32:21.358995 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.notifications [None req-d649cdcf-804b-4411-b4a9-b85885fa02c4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7d24f72b48d04f3bb94401f05d838527'} {{(pid=80244) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 08:32:21.371863 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.notifications [None req-d649cdcf-804b-4411-b4a9-b85885fa02c4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7d24f72b48d04f3bb94401f05d838527'} {{(pid=80244) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 08:32:21.375172 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG dbcounter [-] [80245] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80245) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:32:21.382349 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a409b99e-8a1c-4d31-a339-81229351c67c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:32:21.385542 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a409b99e-8a1c-4d31-a339-81229351c67c service neutron] RBAC: Authorization granted {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:32:21.393786 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.notifications [None req-d649cdcf-804b-4411-b4a9-b85885fa02c4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7d24f72b48d04f3bb94401f05d838527'} {{(pid=80244) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 08:32:21.402596 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 1217/2441] 10.222.0.47 () {58 vars in 1223 bytes} [Thu Jul 21 08:32:21 2022] DELETE /identity/v3/users/7d24f72b48d04f3bb94401f05d838527 => generated 0 bytes in 322 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 21 08:32:21.404135 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4cab0c9-22e1-4791-8eba-aeb7e90411ca None None] REQUEST_METHOD: `POST` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:32:21.405304 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4cab0c9-22e1-4791-8eba-aeb7e90411ca None None] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:32:21.408283 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: [pid: 80245|app: 0|req: 1225/2442] 10.222.0.47 () {60 vars in 1365 bytes} [Thu Jul 21 08:32:21 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 161 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 08:32:21.417681 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4cab0c9-22e1-4791-8eba-aeb7e90411ca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:32:21.422026 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a409b99e-8a1c-4d31-a339-81229351c67c service neutron] Authenticating user token {{(pid=80245) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:32:21.423885 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.fernet_utils [None req-a409b99e-8a1c-4d31-a339-81229351c67c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80245) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 08:32:21.442770 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a409b99e-8a1c-4d31-a339-81229351c67c service neutron] Validating token access rules against request {{(pid=80245) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:32:21.444369 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.auth.core [None req-f4cab0c9-22e1-4791-8eba-aeb7e90411ca None None] MFA Rules not processed for user `aa156584880d4449bd89a7bcdcfe403f`. Rule list: `[]` (Enabled: `True`). {{(pid=80244) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 08:32:21.445825 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.fernet_utils [None req-f4cab0c9-22e1-4791-8eba-aeb7e90411ca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80244) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 08:32:21.519264 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 1218/2443] 10.222.0.47 () {58 vars in 957 bytes} [Thu Jul 21 08:32:21 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 08:32:21.521682 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2497b195-4a5c-4875-960c-ae41229d276e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '42d15fc0515945acacbbffcf2356c7a5', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80245) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:32:21.522642 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2497b195-4a5c-4875-960c-ae41229d276e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:32:21.522984 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2497b195-4a5c-4875-960c-ae41229d276e admin admin] SCRIPT_NAME: `/identity` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:32:21.523574 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2497b195-4a5c-4875-960c-ae41229d276e admin admin] PATH_INFO: `/v3/users/867e21619d6447d78f87058f77b1f90a` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:32:21.524823 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2497b195-4a5c-4875-960c-ae41229d276e admin admin] RBAC: Authorizing `identity:delete_user(user_id=867e21619d6447d78f87058f77b1f90a)` {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:32:21.529236 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2497b195-4a5c-4875-960c-ae41229d276e admin admin] RBAC: Authorization granted {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:32:21.600297 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.notifications [None req-2497b195-4a5c-4875-960c-ae41229d276e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '867e21619d6447d78f87058f77b1f90a'} {{(pid=80245) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 08:32:21.608224 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.notifications [None req-2497b195-4a5c-4875-960c-ae41229d276e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '867e21619d6447d78f87058f77b1f90a'} {{(pid=80245) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 08:32:21.624602 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.notifications [None req-2497b195-4a5c-4875-960c-ae41229d276e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '867e21619d6447d78f87058f77b1f90a'} {{(pid=80245) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 08:32:21.632383 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.notifications [None req-2497b195-4a5c-4875-960c-ae41229d276e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '867e21619d6447d78f87058f77b1f90a'} {{(pid=80245) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 08:32:21.633059 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.notifications [None req-2497b195-4a5c-4875-960c-ae41229d276e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '867e21619d6447d78f87058f77b1f90a'} {{(pid=80245) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 08:32:21.639351 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: [pid: 80245|app: 0|req: 1226/2444] 10.222.0.47 () {58 vars in 1223 bytes} [Thu Jul 21 08:32:21 2022] DELETE /identity/v3/users/867e21619d6447d78f87058f77b1f90a => generated 0 bytes in 218 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 08:32:21.822074 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4cab0c9-22e1-4791-8eba-aeb7e90411ca None None] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:32:21.824171 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.fernet_utils [None req-f4cab0c9-22e1-4791-8eba-aeb7e90411ca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80244) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 08:32:21.856252 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4cab0c9-22e1-4791-8eba-aeb7e90411ca None None] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:32:21.955991 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7be285c-6fe0-4fcb-8d11-caacbd150029 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '42d15fc0515945acacbbffcf2356c7a5', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:32:21.957056 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7be285c-6fe0-4fcb-8d11-caacbd150029 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:32:21.957404 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7be285c-6fe0-4fcb-8d11-caacbd150029 admin admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:32:21.957743 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7be285c-6fe0-4fcb-8d11-caacbd150029 admin admin] PATH_INFO: `/v3/projects/c4df09e49b19415683519a57c34bdcc9` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:32:21.959114 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7be285c-6fe0-4fcb-8d11-caacbd150029 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c4df09e49b19415683519a57c34bdcc9)` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:32:21.961085 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7be285c-6fe0-4fcb-8d11-caacbd150029 admin admin] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:32:22.013607 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.notifications [None req-e7be285c-6fe0-4fcb-8d11-caacbd150029 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c4df09e49b19415683519a57c34bdcc9', 'request_id': 'req-e7be285c-6fe0-4fcb-8d11-caacbd150029'} {{(pid=80244) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 08:32:22.021949 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.notifications [None req-e7be285c-6fe0-4fcb-8d11-caacbd150029 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c4df09e49b19415683519a57c34bdcc9', 'request_id': 'req-e7be285c-6fe0-4fcb-8d11-caacbd150029'} {{(pid=80244) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 08:32:22.028880 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.notifications [None req-e7be285c-6fe0-4fcb-8d11-caacbd150029 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c4df09e49b19415683519a57c34bdcc9', 'request_id': 'req-e7be285c-6fe0-4fcb-8d11-caacbd150029'} {{(pid=80244) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 08:32:22.039586 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.notifications [None req-e7be285c-6fe0-4fcb-8d11-caacbd150029 admin admin] The token cache is being invalidate because project c4df09e49b19415683519a57c34bdcc9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80244) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 21 08:32:22.039871 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.notifications [None req-e7be285c-6fe0-4fcb-8d11-caacbd150029 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80244) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 08:32:22.040529 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.notifications [None req-e7be285c-6fe0-4fcb-8d11-caacbd150029 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80244) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 08:32:22.042608 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 1219/2445] 10.222.0.47 () {58 vars in 1232 bytes} [Thu Jul 21 08:32:21 2022] DELETE /identity/v3/projects/c4df09e49b19415683519a57c34bdcc9 => generated 0 bytes in 222 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 08:32:22.046782 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2497b195-4a5c-4875-960c-ae41229d276e admin admin] Authenticating user token {{(pid=80245) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:32:22.048468 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.fernet_utils [None req-2497b195-4a5c-4875-960c-ae41229d276e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80245) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 08:32:22.061857 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2497b195-4a5c-4875-960c-ae41229d276e admin admin] Validating token access rules against request {{(pid=80245) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:32:22.120405 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d505a19e-8cd9-414b-9d9d-b64424f18cf0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '42d15fc0515945acacbbffcf2356c7a5', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80245) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:32:22.121294 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d505a19e-8cd9-414b-9d9d-b64424f18cf0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:32:22.121614 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d505a19e-8cd9-414b-9d9d-b64424f18cf0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:32:22.121859 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d505a19e-8cd9-414b-9d9d-b64424f18cf0 admin admin] PATH_INFO: `/v3/users/e35f2683d0634657a1326fe92d1e9a61` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:32:22.123275 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d505a19e-8cd9-414b-9d9d-b64424f18cf0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e35f2683d0634657a1326fe92d1e9a61)` {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:32:22.125423 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d505a19e-8cd9-414b-9d9d-b64424f18cf0 admin admin] RBAC: Authorization granted {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:32:22.144459 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG dbcounter [-] [80245] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=80245) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:32:22.190107 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7be285c-6fe0-4fcb-8d11-caacbd150029 admin admin] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:32:22.191981 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.fernet_utils [None req-e7be285c-6fe0-4fcb-8d11-caacbd150029 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80244) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 08:32:22.193572 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.notifications [None req-d505a19e-8cd9-414b-9d9d-b64424f18cf0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e35f2683d0634657a1326fe92d1e9a61'} {{(pid=80245) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 08:32:22.205333 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG dbcounter [-] [80244] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=80244) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:32:22.207615 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.notifications [None req-d505a19e-8cd9-414b-9d9d-b64424f18cf0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e35f2683d0634657a1326fe92d1e9a61'} {{(pid=80245) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 08:32:22.209671 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7be285c-6fe0-4fcb-8d11-caacbd150029 admin admin] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:32:22.216301 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d488b5b-b0e7-42df-82a2-f3065c9ca60f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '42d15fc0515945acacbbffcf2356c7a5', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:32:22.217469 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d488b5b-b0e7-42df-82a2-f3065c9ca60f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:32:22.218356 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d488b5b-b0e7-42df-82a2-f3065c9ca60f admin admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:32:22.218685 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d488b5b-b0e7-42df-82a2-f3065c9ca60f admin admin] PATH_INFO: `/v3/projects/67739db64f6044b19d5b8e5e007d3b66` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:32:22.220836 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d488b5b-b0e7-42df-82a2-f3065c9ca60f admin admin] RBAC: Authorizing `identity:delete_project(project_id=67739db64f6044b19d5b8e5e007d3b66)` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:32:22.225486 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d488b5b-b0e7-42df-82a2-f3065c9ca60f admin admin] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:32:22.232548 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.notifications [None req-d505a19e-8cd9-414b-9d9d-b64424f18cf0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e35f2683d0634657a1326fe92d1e9a61'} {{(pid=80245) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 08:32:22.243093 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.notifications [None req-d505a19e-8cd9-414b-9d9d-b64424f18cf0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e35f2683d0634657a1326fe92d1e9a61'} {{(pid=80245) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 08:32:22.244553 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.notifications [None req-d505a19e-8cd9-414b-9d9d-b64424f18cf0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e35f2683d0634657a1326fe92d1e9a61'} {{(pid=80245) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 08:32:22.259197 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: [pid: 80245|app: 0|req: 1227/2446] 10.222.0.47 () {58 vars in 1223 bytes} [Thu Jul 21 08:32:22 2022] DELETE /identity/v3/users/e35f2683d0634657a1326fe92d1e9a61 => generated 0 bytes in 210 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 21 08:32:22.259197 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d505a19e-8cd9-414b-9d9d-b64424f18cf0 admin admin] Authenticating user token {{(pid=80245) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:32:22.259924 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.fernet_utils [None req-d505a19e-8cd9-414b-9d9d-b64424f18cf0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80245) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 08:32:22.319787 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.notifications [None req-5d488b5b-b0e7-42df-82a2-f3065c9ca60f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '67739db64f6044b19d5b8e5e007d3b66', 'request_id': 'req-5d488b5b-b0e7-42df-82a2-f3065c9ca60f'} {{(pid=80244) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 08:32:22.331954 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.notifications [None req-5d488b5b-b0e7-42df-82a2-f3065c9ca60f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '67739db64f6044b19d5b8e5e007d3b66', 'request_id': 'req-5d488b5b-b0e7-42df-82a2-f3065c9ca60f'} {{(pid=80244) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 08:32:22.338029 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d505a19e-8cd9-414b-9d9d-b64424f18cf0 admin admin] Validating token access rules against request {{(pid=80245) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:32:22.356497 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.notifications [None req-5d488b5b-b0e7-42df-82a2-f3065c9ca60f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '67739db64f6044b19d5b8e5e007d3b66', 'request_id': 'req-5d488b5b-b0e7-42df-82a2-f3065c9ca60f'} {{(pid=80244) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 08:32:22.384280 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.notifications [None req-5d488b5b-b0e7-42df-82a2-f3065c9ca60f admin admin] The token cache is being invalidate because project 67739db64f6044b19d5b8e5e007d3b66 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80244) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 21 08:32:22.384597 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.notifications [None req-5d488b5b-b0e7-42df-82a2-f3065c9ca60f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80244) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 08:32:22.385443 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.notifications [None req-5d488b5b-b0e7-42df-82a2-f3065c9ca60f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80244) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 08:32:22.387958 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 1220/2447] 10.222.0.47 () {58 vars in 1232 bytes} [Thu Jul 21 08:32:22 2022] DELETE /identity/v3/projects/67739db64f6044b19d5b8e5e007d3b66 => generated 0 bytes in 198 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 08:32:22.393019 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d488b5b-b0e7-42df-82a2-f3065c9ca60f admin admin] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:32:22.394435 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.fernet_utils [None req-5d488b5b-b0e7-42df-82a2-f3065c9ca60f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80244) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 08:32:22.417466 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c34b0ae-150a-481b-9b9b-13e03ba5bf90 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '42d15fc0515945acacbbffcf2356c7a5', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80245) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:32:22.418744 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c34b0ae-150a-481b-9b9b-13e03ba5bf90 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:32:22.418881 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c34b0ae-150a-481b-9b9b-13e03ba5bf90 admin admin] SCRIPT_NAME: `/identity` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:32:22.419250 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c34b0ae-150a-481b-9b9b-13e03ba5bf90 admin admin] PATH_INFO: `/v3/projects/16dc806bd5c842258c652409fa4bc353` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:32:22.421756 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c34b0ae-150a-481b-9b9b-13e03ba5bf90 admin admin] RBAC: Authorizing `identity:delete_project(project_id=16dc806bd5c842258c652409fa4bc353)` {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:32:22.422978 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c34b0ae-150a-481b-9b9b-13e03ba5bf90 admin admin] RBAC: Authorization granted {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:32:22.485295 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG dbcounter [-] [80245] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=80245) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:32:22.497808 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.notifications [None req-9c34b0ae-150a-481b-9b9b-13e03ba5bf90 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '16dc806bd5c842258c652409fa4bc353', 'request_id': 'req-9c34b0ae-150a-481b-9b9b-13e03ba5bf90'} {{(pid=80245) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 08:32:22.507421 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.notifications [None req-9c34b0ae-150a-481b-9b9b-13e03ba5bf90 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '16dc806bd5c842258c652409fa4bc353', 'request_id': 'req-9c34b0ae-150a-481b-9b9b-13e03ba5bf90'} {{(pid=80245) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 08:32:22.515980 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.notifications [None req-9c34b0ae-150a-481b-9b9b-13e03ba5bf90 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '16dc806bd5c842258c652409fa4bc353', 'request_id': 'req-9c34b0ae-150a-481b-9b9b-13e03ba5bf90'} {{(pid=80245) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 08:32:22.547405 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.notifications [None req-9c34b0ae-150a-481b-9b9b-13e03ba5bf90 admin admin] The token cache is being invalidate because project 16dc806bd5c842258c652409fa4bc353 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80245) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 21 08:32:22.547637 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.notifications [None req-9c34b0ae-150a-481b-9b9b-13e03ba5bf90 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80245) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 08:32:22.548412 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.notifications [None req-9c34b0ae-150a-481b-9b9b-13e03ba5bf90 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80245) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 08:32:22.557976 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d488b5b-b0e7-42df-82a2-f3065c9ca60f admin admin] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:32:22.568635 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: [pid: 80245|app: 0|req: 1228/2448] 10.222.0.47 () {58 vars in 1232 bytes} [Thu Jul 21 08:32:22 2022] DELETE /identity/v3/projects/16dc806bd5c842258c652409fa4bc353 => generated 0 bytes in 293 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 08:32:22.569867 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c34b0ae-150a-481b-9b9b-13e03ba5bf90 admin admin] Authenticating user token {{(pid=80245) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:32:22.571278 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.fernet_utils [None req-9c34b0ae-150a-481b-9b9b-13e03ba5bf90 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80245) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 08:32:22.728013 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d35d889-9858-4068-9761-d168c152e10d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '42d15fc0515945acacbbffcf2356c7a5', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:32:22.729947 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d35d889-9858-4068-9761-d168c152e10d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:32:22.733059 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d35d889-9858-4068-9761-d168c152e10d admin admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:32:22.734866 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d35d889-9858-4068-9761-d168c152e10d admin admin] PATH_INFO: `/v3/users/0c9b621a1c8b46cc941d9e5d81d859f4` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:32:22.738561 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d35d889-9858-4068-9761-d168c152e10d admin admin] RBAC: Authorizing `identity:delete_user(user_id=0c9b621a1c8b46cc941d9e5d81d859f4)` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:32:22.743425 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d35d889-9858-4068-9761-d168c152e10d admin admin] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:32:22.754268 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c34b0ae-150a-481b-9b9b-13e03ba5bf90 admin admin] Validating token access rules against request {{(pid=80245) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:32:22.767515 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed86ffcf-5ae9-419d-a092-64ad3c304bf5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '42d15fc0515945acacbbffcf2356c7a5', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80245) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:32:22.769269 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed86ffcf-5ae9-419d-a092-64ad3c304bf5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:32:22.771659 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed86ffcf-5ae9-419d-a092-64ad3c304bf5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:32:22.771659 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed86ffcf-5ae9-419d-a092-64ad3c304bf5 admin admin] PATH_INFO: `/v3/projects/3207b455ea9245a49e83eaf5e9e1cd2f` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:32:22.780267 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed86ffcf-5ae9-419d-a092-64ad3c304bf5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=3207b455ea9245a49e83eaf5e9e1cd2f)` {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:32:22.785822 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed86ffcf-5ae9-419d-a092-64ad3c304bf5 admin admin] RBAC: Authorization granted {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:32:22.828980 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG dbcounter [-] [80244] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=80244) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:32:22.843182 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.notifications [None req-6d35d889-9858-4068-9761-d168c152e10d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0c9b621a1c8b46cc941d9e5d81d859f4'} {{(pid=80244) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 08:32:22.851649 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.notifications [None req-6d35d889-9858-4068-9761-d168c152e10d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0c9b621a1c8b46cc941d9e5d81d859f4'} {{(pid=80244) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 08:32:22.856046 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.notifications [None req-ed86ffcf-5ae9-419d-a092-64ad3c304bf5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3207b455ea9245a49e83eaf5e9e1cd2f', 'request_id': 'req-ed86ffcf-5ae9-419d-a092-64ad3c304bf5'} {{(pid=80245) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 08:32:22.862524 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.notifications [None req-ed86ffcf-5ae9-419d-a092-64ad3c304bf5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3207b455ea9245a49e83eaf5e9e1cd2f', 'request_id': 'req-ed86ffcf-5ae9-419d-a092-64ad3c304bf5'} {{(pid=80245) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 08:32:22.869713 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.notifications [None req-6d35d889-9858-4068-9761-d168c152e10d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0c9b621a1c8b46cc941d9e5d81d859f4'} {{(pid=80244) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 08:32:22.870383 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.notifications [None req-ed86ffcf-5ae9-419d-a092-64ad3c304bf5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3207b455ea9245a49e83eaf5e9e1cd2f', 'request_id': 'req-ed86ffcf-5ae9-419d-a092-64ad3c304bf5'} {{(pid=80245) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 08:32:22.870683 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.notifications [None req-6d35d889-9858-4068-9761-d168c152e10d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0c9b621a1c8b46cc941d9e5d81d859f4'} {{(pid=80244) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 08:32:22.879391 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.notifications [None req-6d35d889-9858-4068-9761-d168c152e10d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0c9b621a1c8b46cc941d9e5d81d859f4'} {{(pid=80244) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 08:32:22.887396 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 1221/2449] 10.222.0.47 () {58 vars in 1223 bytes} [Thu Jul 21 08:32:22 2022] DELETE /identity/v3/users/0c9b621a1c8b46cc941d9e5d81d859f4 => generated 0 bytes in 496 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 21 08:32:22.889230 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.notifications [None req-ed86ffcf-5ae9-419d-a092-64ad3c304bf5 admin admin] The token cache is being invalidate because project 3207b455ea9245a49e83eaf5e9e1cd2f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80245) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 21 08:32:22.898180 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.notifications [None req-ed86ffcf-5ae9-419d-a092-64ad3c304bf5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80245) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 08:32:22.900353 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.notifications [None req-ed86ffcf-5ae9-419d-a092-64ad3c304bf5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80245) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 08:32:22.903017 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: [pid: 80245|app: 0|req: 1229/2450] 10.222.0.47 () {58 vars in 1232 bytes} [Thu Jul 21 08:32:22 2022] DELETE /identity/v3/projects/3207b455ea9245a49e83eaf5e9e1cd2f => generated 0 bytes in 334 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 08:32:22.908081 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f720cee5-9ceb-43f3-8849-5375366f657b None None] REQUEST_METHOD: `POST` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:32:22.908397 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f720cee5-9ceb-43f3-8849-5375366f657b None None] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:32:22.908869 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f720cee5-9ceb-43f3-8849-5375366f657b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:32:22.930504 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.auth.core [None req-f720cee5-9ceb-43f3-8849-5375366f657b None None] MFA Rules not processed for user `2dce40b8d74c41ec94cc36e6ff05d412`. Rule list: `[]` (Enabled: `True`). {{(pid=80244) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 08:32:22.931888 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.fernet_utils [None req-f720cee5-9ceb-43f3-8849-5375366f657b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80244) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 08:32:23.038517 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 1222/2451] 10.222.0.47 () {58 vars in 957 bytes} [Thu Jul 21 08:32:22 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 08:32:23.041148 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1c82fca-aa78-46d1-a3b4-6a82c0f08ac5 None None] REQUEST_METHOD: `POST` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:32:23.041522 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1c82fca-aa78-46d1-a3b4-6a82c0f08ac5 None None] SCRIPT_NAME: `/identity` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:32:23.042063 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1c82fca-aa78-46d1-a3b4-6a82c0f08ac5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:32:23.065473 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.auth.core [None req-c1c82fca-aa78-46d1-a3b4-6a82c0f08ac5 None None] MFA Rules not processed for user `2dce40b8d74c41ec94cc36e6ff05d412`. Rule list: `[]` (Enabled: `True`). {{(pid=80245) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 08:32:23.067122 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.fernet_utils [None req-c1c82fca-aa78-46d1-a3b4-6a82c0f08ac5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80245) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 08:32:23.083229 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: [pid: 80245|app: 0|req: 1230/2452] 10.222.0.47 () {58 vars in 957 bytes} [Thu Jul 21 08:32:23 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 44 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 08:32:23.088968 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f720cee5-9ceb-43f3-8849-5375366f657b None None] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:32:23.105311 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f720cee5-9ceb-43f3-8849-5375366f657b None None] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:32:23.120238 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0788d812-5598-4999-81c0-30d14016c796 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '42d15fc0515945acacbbffcf2356c7a5', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:32:23.120949 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0788d812-5598-4999-81c0-30d14016c796 admin admin] REQUEST_METHOD: `GET` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:32:23.121235 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0788d812-5598-4999-81c0-30d14016c796 admin admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:32:23.121511 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0788d812-5598-4999-81c0-30d14016c796 admin admin] PATH_INFO: `/v3/domains` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:32:23.122312 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0788d812-5598-4999-81c0-30d14016c796 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80244) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 08:32:23.122637 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0788d812-5598-4999-81c0-30d14016c796 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:32:23.124364 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0788d812-5598-4999-81c0-30d14016c796 admin admin] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:32:23.141450 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 1223/2453] 10.222.0.47 () {58 vars in 1152 bytes} [Thu Jul 21 08:32:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 51 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 08:32:23.142636 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1c82fca-aa78-46d1-a3b4-6a82c0f08ac5 None None] Authenticating user token {{(pid=80245) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:32:23.154287 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1c82fca-aa78-46d1-a3b4-6a82c0f08ac5 None None] Validating token access rules against request {{(pid=80245) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:32:23.163026 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9625aa3c-0528-407e-84b5-e7c040e82f2b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '42d15fc0515945acacbbffcf2356c7a5', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80245) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:32:23.164457 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9625aa3c-0528-407e-84b5-e7c040e82f2b admin admin] REQUEST_METHOD: `POST` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:32:23.165201 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9625aa3c-0528-407e-84b5-e7c040e82f2b admin admin] SCRIPT_NAME: `/identity` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:32:23.165515 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9625aa3c-0528-407e-84b5-e7c040e82f2b admin admin] PATH_INFO: `/v3/projects` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:32:23.166318 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9625aa3c-0528-407e-84b5-e7c040e82f2b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:32:23.168646 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9625aa3c-0528-407e-84b5-e7c040e82f2b admin admin] RBAC: Authorization granted {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:32:23.201060 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: [pid: 80245|app: 0|req: 1231/2454] 10.222.0.47 () {60 vars in 1152 bytes} [Thu Jul 21 08:32:23 2022] POST /identity/v3/projects => generated 393 bytes in 59 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 21 08:32:23.205601 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0788d812-5598-4999-81c0-30d14016c796 admin admin] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:32:23.218939 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0788d812-5598-4999-81c0-30d14016c796 admin admin] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:32:23.279413 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9625aa3c-0528-407e-84b5-e7c040e82f2b admin admin] Authenticating user token {{(pid=80245) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:32:23.281216 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.fernet_utils [None req-9625aa3c-0528-407e-84b5-e7c040e82f2b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80245) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 08:32:23.307583 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b90a481-4686-4cb3-9dbe-96d192c5d557 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '42d15fc0515945acacbbffcf2356c7a5', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:32:23.308474 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b90a481-4686-4cb3-9dbe-96d192c5d557 admin admin] REQUEST_METHOD: `POST` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:32:23.308840 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b90a481-4686-4cb3-9dbe-96d192c5d557 admin admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:32:23.309398 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b90a481-4686-4cb3-9dbe-96d192c5d557 admin admin] PATH_INFO: `/v3/users` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:32:23.310191 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b90a481-4686-4cb3-9dbe-96d192c5d557 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:32:23.312263 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b90a481-4686-4cb3-9dbe-96d192c5d557 admin admin] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:32:23.328967 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG dbcounter [-] [80245] Writing DB stats keystone:SELECT=86,keystone:DELETE=11,keystone:INSERT=3 {{(pid=80245) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:32:23.338593 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 1224/2455] 10.222.0.47 () {60 vars in 1143 bytes} [Thu Jul 21 08:32:23 2022] POST /identity/v3/users => generated 297 bytes in 131 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 08:32:23.340007 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b90a481-4686-4cb3-9dbe-96d192c5d557 admin admin] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:32:23.351891 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b90a481-4686-4cb3-9dbe-96d192c5d557 admin admin] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:32:23.358358 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28d539e2-71bf-4162-9a71-5f62439dfdfe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '42d15fc0515945acacbbffcf2356c7a5', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:32:23.359133 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28d539e2-71bf-4162-9a71-5f62439dfdfe admin admin] REQUEST_METHOD: `GET` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:32:23.359426 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28d539e2-71bf-4162-9a71-5f62439dfdfe admin admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:32:23.359708 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28d539e2-71bf-4162-9a71-5f62439dfdfe admin admin] PATH_INFO: `/v3/roles` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:32:23.360494 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28d539e2-71bf-4162-9a71-5f62439dfdfe admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:32:23.362239 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28d539e2-71bf-4162-9a71-5f62439dfdfe admin admin] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:32:23.371149 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9625aa3c-0528-407e-84b5-e7c040e82f2b admin admin] Validating token access rules against request {{(pid=80245) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:32:23.376623 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68bffc24-80da-415c-bd9d-1727811e0a24 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '42d15fc0515945acacbbffcf2356c7a5', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80245) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:32:23.379368 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 1225/2456] 10.222.0.47 () {58 vars in 1121 bytes} [Thu Jul 21 08:32:23 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 08:32:23.379507 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68bffc24-80da-415c-bd9d-1727811e0a24 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:32:23.379507 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68bffc24-80da-415c-bd9d-1727811e0a24 admin admin] SCRIPT_NAME: `/identity` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:32:23.379507 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68bffc24-80da-415c-bd9d-1727811e0a24 admin admin] PATH_INFO: `/v3/projects/6fc370bce50947ea8fb7e7e0f0d68912` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:32:23.379507 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68bffc24-80da-415c-bd9d-1727811e0a24 admin admin] RBAC: Authorizing `identity:delete_project(project_id=6fc370bce50947ea8fb7e7e0f0d68912)` {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:32:23.381756 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68bffc24-80da-415c-bd9d-1727811e0a24 admin admin] RBAC: Authorization granted {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:32:23.383979 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28d539e2-71bf-4162-9a71-5f62439dfdfe admin admin] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:32:23.396353 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28d539e2-71bf-4162-9a71-5f62439dfdfe admin admin] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:32:23.401358 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-860b551b-ab0c-40bf-89cb-d3550d77667b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '42d15fc0515945acacbbffcf2356c7a5', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:32:23.402364 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-860b551b-ab0c-40bf-89cb-d3550d77667b admin admin] REQUEST_METHOD: `GET` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:32:23.402364 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-860b551b-ab0c-40bf-89cb-d3550d77667b admin admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:32:23.402754 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-860b551b-ab0c-40bf-89cb-d3550d77667b admin admin] PATH_INFO: `/v3/roles` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:32:23.403348 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-860b551b-ab0c-40bf-89cb-d3550d77667b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:32:23.406089 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-860b551b-ab0c-40bf-89cb-d3550d77667b admin admin] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:32:23.419058 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 1226/2457] 10.222.0.47 () {58 vars in 1121 bytes} [Thu Jul 21 08:32:23 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 21 08:32:23.424083 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-860b551b-ab0c-40bf-89cb-d3550d77667b admin admin] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:32:23.436423 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-860b551b-ab0c-40bf-89cb-d3550d77667b admin admin] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:32:23.437625 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.notifications [None req-68bffc24-80da-415c-bd9d-1727811e0a24 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6fc370bce50947ea8fb7e7e0f0d68912', 'request_id': 'req-68bffc24-80da-415c-bd9d-1727811e0a24'} {{(pid=80245) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 08:32:23.441545 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG dbcounter [-] [80244] Writing DB stats keystone:SELECT=91,keystone:DELETE=5,keystone:INSERT=4 {{(pid=80244) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:32:23.444659 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.notifications [None req-68bffc24-80da-415c-bd9d-1727811e0a24 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6fc370bce50947ea8fb7e7e0f0d68912', 'request_id': 'req-68bffc24-80da-415c-bd9d-1727811e0a24'} {{(pid=80245) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 08:32:23.451553 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.notifications [None req-68bffc24-80da-415c-bd9d-1727811e0a24 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6fc370bce50947ea8fb7e7e0f0d68912', 'request_id': 'req-68bffc24-80da-415c-bd9d-1727811e0a24'} {{(pid=80245) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 08:32:23.465357 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.notifications [None req-68bffc24-80da-415c-bd9d-1727811e0a24 admin admin] The token cache is being invalidate because project 6fc370bce50947ea8fb7e7e0f0d68912 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80245) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 21 08:32:23.465357 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.notifications [None req-68bffc24-80da-415c-bd9d-1727811e0a24 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80245) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 08:32:23.465967 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.notifications [None req-68bffc24-80da-415c-bd9d-1727811e0a24 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80245) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 08:32:23.467607 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: [pid: 80245|app: 0|req: 1232/2458] 10.222.0.47 () {58 vars in 1232 bytes} [Thu Jul 21 08:32:23 2022] DELETE /identity/v3/projects/6fc370bce50947ea8fb7e7e0f0d68912 => generated 0 bytes in 189 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 21 08:32:23.472427 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a477d49a-41ee-4130-b94c-2b835047b9d8 None None] REQUEST_METHOD: `POST` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:32:23.472720 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a477d49a-41ee-4130-b94c-2b835047b9d8 None None] SCRIPT_NAME: `/identity` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:32:23.473026 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a477d49a-41ee-4130-b94c-2b835047b9d8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:32:23.493098 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.auth.core [None req-a477d49a-41ee-4130-b94c-2b835047b9d8 None None] MFA Rules not processed for user `2dce40b8d74c41ec94cc36e6ff05d412`. Rule list: `[]` (Enabled: `True`). {{(pid=80245) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 08:32:23.494790 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.fernet_utils [None req-a477d49a-41ee-4130-b94c-2b835047b9d8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80245) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 08:32:23.511052 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0340f910-0a76-4065-89cb-e6f73ba48e7e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '42d15fc0515945acacbbffcf2356c7a5', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:32:23.511991 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0340f910-0a76-4065-89cb-e6f73ba48e7e admin admin] REQUEST_METHOD: `PUT` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:32:23.512276 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0340f910-0a76-4065-89cb-e6f73ba48e7e admin admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:32:23.512552 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0340f910-0a76-4065-89cb-e6f73ba48e7e admin admin] PATH_INFO: `/v3/projects/221cee8a42424d84b6e5629d8bcfd5e0/users/9669d6d508344f1da1fa98c6bbc75854/roles/3a074738221e4601b5205715ba87d240` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:32:23.528109 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0340f910-0a76-4065-89cb-e6f73ba48e7e admin admin] RBAC: Authorizing `identity:create_grant(project_id=221cee8a42424d84b6e5629d8bcfd5e0, user_id=9669d6d508344f1da1fa98c6bbc75854, role_id=3a074738221e4601b5205715ba87d240)` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:32:23.530797 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0340f910-0a76-4065-89cb-e6f73ba48e7e admin admin] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:32:23.546755 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 1227/2459] 10.222.0.47 () {60 vars in 1482 bytes} [Thu Jul 21 08:32:23 2022] PUT /identity/v3/projects/221cee8a42424d84b6e5629d8bcfd5e0/users/9669d6d508344f1da1fa98c6bbc75854/roles/3a074738221e4601b5205715ba87d240 => generated 0 bytes in 124 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 21 08:32:23.558406 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dd82fbe-168c-46dd-838d-35cfb65fb7ef None None] REQUEST_METHOD: `POST` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:32:23.559098 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dd82fbe-168c-46dd-838d-35cfb65fb7ef None None] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:32:23.559416 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dd82fbe-168c-46dd-838d-35cfb65fb7ef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:32:23.560698 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: [pid: 80245|app: 0|req: 1233/2460] 10.222.0.47 () {58 vars in 957 bytes} [Thu Jul 21 08:32:23 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 21 08:32:23.569340 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9e732dd-4d63-41ff-8681-2b6c21f8ffb7 None None] REQUEST_METHOD: `POST` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:32:23.569633 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9e732dd-4d63-41ff-8681-2b6c21f8ffb7 None None] SCRIPT_NAME: `/identity` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:32:23.569915 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9e732dd-4d63-41ff-8681-2b6c21f8ffb7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:32:23.588618 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.auth.core [None req-5dd82fbe-168c-46dd-838d-35cfb65fb7ef None None] MFA Rules not processed for user `9669d6d508344f1da1fa98c6bbc75854`. Rule list: `[]` (Enabled: `True`). {{(pid=80244) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 08:32:23.589838 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.fernet_utils [None req-5dd82fbe-168c-46dd-838d-35cfb65fb7ef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80244) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 08:32:23.590091 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.auth.core [None req-f9e732dd-4d63-41ff-8681-2b6c21f8ffb7 None None] MFA Rules not processed for user `2dce40b8d74c41ec94cc36e6ff05d412`. Rule list: `[]` (Enabled: `True`). {{(pid=80245) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 08:32:23.591244 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.fernet_utils [None req-f9e732dd-4d63-41ff-8681-2b6c21f8ffb7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80245) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 08:32:23.652035 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: [pid: 80245|app: 0|req: 1234/2461] 10.222.0.47 () {58 vars in 957 bytes} [Thu Jul 21 08:32:23 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 21 08:32:23.655186 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 1228/2462] 10.222.0.47 () {58 vars in 957 bytes} [Thu Jul 21 08:32:23 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 21 08:32:23.658625 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9e732dd-4d63-41ff-8681-2b6c21f8ffb7 None None] Authenticating user token {{(pid=80245) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:32:23.659312 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dd82fbe-168c-46dd-838d-35cfb65fb7ef None None] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:32:23.660645 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.fernet_utils [None req-5dd82fbe-168c-46dd-838d-35cfb65fb7ef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80244) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 08:32:23.661056 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG dbcounter [-] [80245] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80245) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:32:23.667837 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9e732dd-4d63-41ff-8681-2b6c21f8ffb7 None None] Validating token access rules against request {{(pid=80245) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:32:23.673916 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef7ebbc6-4d3a-4c21-9064-934a3fd2ba58 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '42d15fc0515945acacbbffcf2356c7a5', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80245) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:32:23.674618 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef7ebbc6-4d3a-4c21-9064-934a3fd2ba58 admin admin] REQUEST_METHOD: `GET` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:32:23.674910 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef7ebbc6-4d3a-4c21-9064-934a3fd2ba58 admin admin] SCRIPT_NAME: `/identity` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:32:23.675285 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef7ebbc6-4d3a-4c21-9064-934a3fd2ba58 admin admin] PATH_INFO: `/v3/domains` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:32:23.675985 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dd82fbe-168c-46dd-838d-35cfb65fb7ef None None] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:32:23.676105 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef7ebbc6-4d3a-4c21-9064-934a3fd2ba58 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80245) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 08:32:23.676352 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef7ebbc6-4d3a-4c21-9064-934a3fd2ba58 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:32:23.678010 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef7ebbc6-4d3a-4c21-9064-934a3fd2ba58 admin admin] RBAC: Authorization granted {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:32:23.680623 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb7b0daa-069c-4700-8c0f-d6033d36d8bf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '42d15fc0515945acacbbffcf2356c7a5', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:32:23.681840 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb7b0daa-069c-4700-8c0f-d6033d36d8bf admin admin] REQUEST_METHOD: `POST` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:32:23.682124 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb7b0daa-069c-4700-8c0f-d6033d36d8bf admin admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:32:23.682446 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb7b0daa-069c-4700-8c0f-d6033d36d8bf admin admin] PATH_INFO: `/v3/projects` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:32:23.683334 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb7b0daa-069c-4700-8c0f-d6033d36d8bf admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:32:23.685472 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb7b0daa-069c-4700-8c0f-d6033d36d8bf admin admin] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:32:23.700673 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: [pid: 80245|app: 0|req: 1235/2463] 10.222.0.47 () {58 vars in 1152 bytes} [Thu Jul 21 08:32:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 21 08:32:23.701723 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef7ebbc6-4d3a-4c21-9064-934a3fd2ba58 admin admin] Authenticating user token {{(pid=80245) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:32:23.713151 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef7ebbc6-4d3a-4c21-9064-934a3fd2ba58 admin admin] Validating token access rules against request {{(pid=80245) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:32:23.719552 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 1229/2464] 10.222.0.47 () {60 vars in 1152 bytes} [Thu Jul 21 08:32:23 2022] POST /identity/v3/projects => generated 391 bytes in 61 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 21 08:32:23.722874 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de509551-9bcc-4dae-8568-8b39e6bc0628 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '42d15fc0515945acacbbffcf2356c7a5', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80245) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:32:23.723642 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de509551-9bcc-4dae-8568-8b39e6bc0628 admin admin] REQUEST_METHOD: `POST` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:32:23.723925 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de509551-9bcc-4dae-8568-8b39e6bc0628 admin admin] SCRIPT_NAME: `/identity` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:32:23.724057 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb7b0daa-069c-4700-8c0f-d6033d36d8bf admin admin] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:32:23.724221 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de509551-9bcc-4dae-8568-8b39e6bc0628 admin admin] PATH_INFO: `/v3/projects` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:32:23.724916 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de509551-9bcc-4dae-8568-8b39e6bc0628 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:32:23.727123 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de509551-9bcc-4dae-8568-8b39e6bc0628 admin admin] RBAC: Authorization granted {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:32:23.759447 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: [pid: 80245|app: 0|req: 1236/2465] 10.222.0.47 () {60 vars in 1152 bytes} [Thu Jul 21 08:32:23 2022] POST /identity/v3/projects => generated 399 bytes in 57 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 08:32:23.762161 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de509551-9bcc-4dae-8568-8b39e6bc0628 admin admin] Authenticating user token {{(pid=80245) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:32:23.794666 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb7b0daa-069c-4700-8c0f-d6033d36d8bf admin admin] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:32:23.799222 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG dbcounter [-] [80244] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80244) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:32:23.848288 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de509551-9bcc-4dae-8568-8b39e6bc0628 admin admin] Validating token access rules against request {{(pid=80245) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:32:23.853850 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d22a83ee-8752-4374-b88c-4869b3f8fdda admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '42d15fc0515945acacbbffcf2356c7a5', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80245) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:32:23.854528 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d22a83ee-8752-4374-b88c-4869b3f8fdda admin admin] REQUEST_METHOD: `POST` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:32:23.854808 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d22a83ee-8752-4374-b88c-4869b3f8fdda admin admin] SCRIPT_NAME: `/identity` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:32:23.855086 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d22a83ee-8752-4374-b88c-4869b3f8fdda admin admin] PATH_INFO: `/v3/users` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:32:23.855813 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d22a83ee-8752-4374-b88c-4869b3f8fdda admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:32:23.857922 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d22a83ee-8752-4374-b88c-4869b3f8fdda admin admin] RBAC: Authorization granted {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:32:23.874440 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d7f60db-53db-49b8-8105-ae703adcdb9c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '42d15fc0515945acacbbffcf2356c7a5', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:32:23.875500 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d7f60db-53db-49b8-8105-ae703adcdb9c admin admin] REQUEST_METHOD: `POST` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:32:23.876172 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d7f60db-53db-49b8-8105-ae703adcdb9c admin admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:32:23.877548 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d7f60db-53db-49b8-8105-ae703adcdb9c admin admin] PATH_INFO: `/v3/users` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:32:23.879180 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d7f60db-53db-49b8-8105-ae703adcdb9c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:32:23.881535 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d7f60db-53db-49b8-8105-ae703adcdb9c admin admin] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:32:23.883102 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: [pid: 80245|app: 0|req: 1237/2466] 10.222.0.47 () {60 vars in 1143 bytes} [Thu Jul 21 08:32:23 2022] POST /identity/v3/users => generated 300 bytes in 121 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 21 08:32:23.887207 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d22a83ee-8752-4374-b88c-4869b3f8fdda admin admin] Authenticating user token {{(pid=80245) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:32:23.900986 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d22a83ee-8752-4374-b88c-4869b3f8fdda admin admin] Validating token access rules against request {{(pid=80245) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:32:23.911933 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 1230/2467] 10.222.0.47 () {60 vars in 1143 bytes} [Thu Jul 21 08:32:23 2022] POST /identity/v3/users => generated 302 bytes in 188 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 08:32:23.912123 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a329c8c-a6f9-4406-9a4a-35f804b43153 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '42d15fc0515945acacbbffcf2356c7a5', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80245) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:32:23.913417 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d7f60db-53db-49b8-8105-ae703adcdb9c admin admin] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:32:23.916288 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a329c8c-a6f9-4406-9a4a-35f804b43153 admin admin] REQUEST_METHOD: `GET` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:32:23.916629 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a329c8c-a6f9-4406-9a4a-35f804b43153 admin admin] SCRIPT_NAME: `/identity` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:32:23.916915 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a329c8c-a6f9-4406-9a4a-35f804b43153 admin admin] PATH_INFO: `/v3/roles` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:32:23.919777 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a329c8c-a6f9-4406-9a4a-35f804b43153 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:32:23.924738 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d7f60db-53db-49b8-8105-ae703adcdb9c admin admin] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:32:23.925512 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a329c8c-a6f9-4406-9a4a-35f804b43153 admin admin] RBAC: Authorization granted {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:32:23.929535 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b51d15f1-4e12-4031-947e-72d0273aaf24 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '42d15fc0515945acacbbffcf2356c7a5', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:32:23.930210 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b51d15f1-4e12-4031-947e-72d0273aaf24 admin admin] REQUEST_METHOD: `GET` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:32:23.930490 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b51d15f1-4e12-4031-947e-72d0273aaf24 admin admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:32:23.930764 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b51d15f1-4e12-4031-947e-72d0273aaf24 admin admin] PATH_INFO: `/v3/roles` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:32:23.932411 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b51d15f1-4e12-4031-947e-72d0273aaf24 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:32:23.934547 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b51d15f1-4e12-4031-947e-72d0273aaf24 admin admin] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:32:23.948120 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: [pid: 80245|app: 0|req: 1238/2468] 10.222.0.47 () {58 vars in 1121 bytes} [Thu Jul 21 08:32:23 2022] GET /identity/v3/roles => generated 1420 bytes in 61 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 21 08:32:23.951020 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 1231/2469] 10.222.0.47 () {58 vars in 1121 bytes} [Thu Jul 21 08:32:23 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 08:32:23.951970 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b51d15f1-4e12-4031-947e-72d0273aaf24 admin admin] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:32:23.952197 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a329c8c-a6f9-4406-9a4a-35f804b43153 admin admin] Authenticating user token {{(pid=80245) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:32:23.965262 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a329c8c-a6f9-4406-9a4a-35f804b43153 admin admin] Validating token access rules against request {{(pid=80245) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:32:23.967476 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b51d15f1-4e12-4031-947e-72d0273aaf24 admin admin] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:32:23.970579 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43c42cf9-9cac-4394-8cff-d8d371774e1d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '42d15fc0515945acacbbffcf2356c7a5', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80245) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:32:23.971592 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43c42cf9-9cac-4394-8cff-d8d371774e1d admin admin] REQUEST_METHOD: `PUT` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:32:23.971940 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43c42cf9-9cac-4394-8cff-d8d371774e1d admin admin] SCRIPT_NAME: `/identity` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:32:23.972284 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43c42cf9-9cac-4394-8cff-d8d371774e1d admin admin] PATH_INFO: `/v3/projects/b6ecbd2527fa430481e48c210e6a9f64/users/436a6f89d8654f57958ea8e07badfb93/roles/e29870f6e9e2411a9db0d5bf45f32ca5` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:32:23.973664 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51483581-4606-41e9-8ae5-f3530801c5fc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '42d15fc0515945acacbbffcf2356c7a5', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:32:23.974547 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51483581-4606-41e9-8ae5-f3530801c5fc admin admin] REQUEST_METHOD: `GET` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:32:23.974896 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51483581-4606-41e9-8ae5-f3530801c5fc admin admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:32:23.975240 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51483581-4606-41e9-8ae5-f3530801c5fc admin admin] PATH_INFO: `/v3/roles` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:32:23.976088 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51483581-4606-41e9-8ae5-f3530801c5fc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:32:23.977779 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51483581-4606-41e9-8ae5-f3530801c5fc admin admin] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:32:23.990075 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 1232/2470] 10.222.0.47 () {58 vars in 1121 bytes} [Thu Jul 21 08:32:23 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 08:32:23.994850 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51483581-4606-41e9-8ae5-f3530801c5fc admin admin] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:32:23.997305 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43c42cf9-9cac-4394-8cff-d8d371774e1d admin admin] RBAC: Authorizing `identity:create_grant(project_id=b6ecbd2527fa430481e48c210e6a9f64, user_id=436a6f89d8654f57958ea8e07badfb93, role_id=e29870f6e9e2411a9db0d5bf45f32ca5)` {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:32:24.000379 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43c42cf9-9cac-4394-8cff-d8d371774e1d admin admin] RBAC: Authorization granted {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:32:24.021052 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51483581-4606-41e9-8ae5-f3530801c5fc admin admin] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:32:24.028427 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: [pid: 80245|app: 0|req: 1239/2471] 10.222.0.47 () {60 vars in 1482 bytes} [Thu Jul 21 08:32:23 2022] PUT /identity/v3/projects/b6ecbd2527fa430481e48c210e6a9f64/users/436a6f89d8654f57958ea8e07badfb93/roles/e29870f6e9e2411a9db0d5bf45f32ca5 => generated 0 bytes in 77 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 08:32:24.032882 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43c42cf9-9cac-4394-8cff-d8d371774e1d admin admin] Authenticating user token {{(pid=80245) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:32:24.149766 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43c42cf9-9cac-4394-8cff-d8d371774e1d admin admin] Validating token access rules against request {{(pid=80245) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:32:24.152742 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e24333d1-5c84-4ca3-beb6-a6fe060a2939 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '42d15fc0515945acacbbffcf2356c7a5', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:32:24.153715 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e24333d1-5c84-4ca3-beb6-a6fe060a2939 admin admin] REQUEST_METHOD: `PUT` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:32:24.154088 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e24333d1-5c84-4ca3-beb6-a6fe060a2939 admin admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:32:24.154397 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e24333d1-5c84-4ca3-beb6-a6fe060a2939 admin admin] PATH_INFO: `/v3/projects/85beacd8d4d144faa3ad01231de4578d/users/2c530c830bd8483282c78c595bbebf50/roles/3a074738221e4601b5205715ba87d240` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:32:24.156003 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3717329f-5f0b-459c-987e-615430571c67 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '42d15fc0515945acacbbffcf2356c7a5', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80245) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:32:24.156931 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3717329f-5f0b-459c-987e-615430571c67 admin admin] REQUEST_METHOD: `GET` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:32:24.157096 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3717329f-5f0b-459c-987e-615430571c67 admin admin] SCRIPT_NAME: `/identity` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:32:24.157494 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3717329f-5f0b-459c-987e-615430571c67 admin admin] PATH_INFO: `/v3/roles` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:32:24.158553 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3717329f-5f0b-459c-987e-615430571c67 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:32:24.160126 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3717329f-5f0b-459c-987e-615430571c67 admin admin] RBAC: Authorization granted {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:32:24.174479 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e24333d1-5c84-4ca3-beb6-a6fe060a2939 admin admin] RBAC: Authorizing `identity:create_grant(project_id=85beacd8d4d144faa3ad01231de4578d, user_id=2c530c830bd8483282c78c595bbebf50, role_id=3a074738221e4601b5205715ba87d240)` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:32:24.177299 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e24333d1-5c84-4ca3-beb6-a6fe060a2939 admin admin] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:32:24.181558 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: [pid: 80245|app: 0|req: 1240/2472] 10.222.0.47 () {58 vars in 1121 bytes} [Thu Jul 21 08:32:24 2022] GET /identity/v3/roles => generated 1420 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 08:32:24.182625 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3717329f-5f0b-459c-987e-615430571c67 admin admin] Authenticating user token {{(pid=80245) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:32:24.196335 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3717329f-5f0b-459c-987e-615430571c67 admin admin] Validating token access rules against request {{(pid=80245) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:32:24.198377 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 1233/2473] 10.222.0.47 () {60 vars in 1482 bytes} [Thu Jul 21 08:32:23 2022] PUT /identity/v3/projects/85beacd8d4d144faa3ad01231de4578d/users/2c530c830bd8483282c78c595bbebf50/roles/3a074738221e4601b5205715ba87d240 => generated 0 bytes in 204 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 21 08:32:24.210133 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8c969ed-3803-4a17-a963-af72b8501772 None None] REQUEST_METHOD: `POST` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:32:24.210437 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8c969ed-3803-4a17-a963-af72b8501772 None None] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:32:24.211629 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG dbcounter [-] [80245] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80245) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:32:24.214654 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8c969ed-3803-4a17-a963-af72b8501772 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:32:24.258528 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.auth.core [None req-a8c969ed-3803-4a17-a963-af72b8501772 None None] MFA Rules not processed for user `2c530c830bd8483282c78c595bbebf50`. Rule list: `[]` (Enabled: `True`). {{(pid=80244) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 08:32:24.260829 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.fernet_utils [None req-a8c969ed-3803-4a17-a963-af72b8501772 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80244) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 08:32:24.281766 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c421f413-c0e4-4a19-9e82-e2d4cb56c940 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '42d15fc0515945acacbbffcf2356c7a5', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80245) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:32:24.282728 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c421f413-c0e4-4a19-9e82-e2d4cb56c940 admin admin] REQUEST_METHOD: `PUT` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:32:24.283051 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c421f413-c0e4-4a19-9e82-e2d4cb56c940 admin admin] SCRIPT_NAME: `/identity` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:32:24.283359 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c421f413-c0e4-4a19-9e82-e2d4cb56c940 admin admin] PATH_INFO: `/v3/projects/b6ecbd2527fa430481e48c210e6a9f64/users/436a6f89d8654f57958ea8e07badfb93/roles/e29870f6e9e2411a9db0d5bf45f32ca5` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:32:24.286706 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c421f413-c0e4-4a19-9e82-e2d4cb56c940 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b6ecbd2527fa430481e48c210e6a9f64, user_id=436a6f89d8654f57958ea8e07badfb93, role_id=e29870f6e9e2411a9db0d5bf45f32ca5)` {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:32:24.290110 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c421f413-c0e4-4a19-9e82-e2d4cb56c940 admin admin] RBAC: Authorization granted {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:32:24.309490 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: [pid: 80245|app: 0|req: 1241/2474] 10.222.0.47 () {60 vars in 1482 bytes} [Thu Jul 21 08:32:24 2022] PUT /identity/v3/projects/b6ecbd2527fa430481e48c210e6a9f64/users/436a6f89d8654f57958ea8e07badfb93/roles/e29870f6e9e2411a9db0d5bf45f32ca5 => generated 0 bytes in 127 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 08:32:24.320228 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f4b3b40-127f-4d44-a7aa-6e62d3d91df5 None None] REQUEST_METHOD: `POST` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:32:24.322672 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f4b3b40-127f-4d44-a7aa-6e62d3d91df5 None None] SCRIPT_NAME: `/identity` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:32:24.322975 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f4b3b40-127f-4d44-a7aa-6e62d3d91df5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:32:24.323199 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG dbcounter [-] [80244] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80244) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:32:24.357582 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 1234/2475] 10.222.0.47 () {58 vars in 957 bytes} [Thu Jul 21 08:32:24 2022] POST /identity/v3/auth/tokens => generated 3266 bytes in 150 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 08:32:24.362127 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.auth.core [None req-4f4b3b40-127f-4d44-a7aa-6e62d3d91df5 None None] MFA Rules not processed for user `436a6f89d8654f57958ea8e07badfb93`. Rule list: `[]` (Enabled: `True`). {{(pid=80245) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 08:32:24.363720 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.fernet_utils [None req-4f4b3b40-127f-4d44-a7aa-6e62d3d91df5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80245) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 08:32:24.371702 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8c969ed-3803-4a17-a963-af72b8501772 None None] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:32:24.373461 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.fernet_utils [None req-a8c969ed-3803-4a17-a963-af72b8501772 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80244) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 08:32:24.480188 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8c969ed-3803-4a17-a963-af72b8501772 None None] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:32:24.484645 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: [pid: 80245|app: 0|req: 1242/2476] 10.222.0.47 () {58 vars in 957 bytes} [Thu Jul 21 08:32:24 2022] POST /identity/v3/auth/tokens => generated 3325 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 08:32:24.493428 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f4b3b40-127f-4d44-a7aa-6e62d3d91df5 None None] Authenticating user token {{(pid=80245) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:32:24.495363 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.fernet_utils [None req-4f4b3b40-127f-4d44-a7aa-6e62d3d91df5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80245) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 08:32:24.550017 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44148c6d-b0a0-423d-b800-392d583ff433 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4f40410b7674687ab5362ac982248ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4faef13aa50e4e36b6c265fb1398fa14', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:32:24.551105 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44148c6d-b0a0-423d-b800-392d583ff433 service cinder] REQUEST_METHOD: `GET` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:32:24.551777 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44148c6d-b0a0-423d-b800-392d583ff433 service cinder] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:32:24.552309 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44148c6d-b0a0-423d-b800-392d583ff433 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:32:24.574469 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f4b3b40-127f-4d44-a7aa-6e62d3d91df5 None None] Validating token access rules against request {{(pid=80245) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:32:24.579512 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cea2b641-f8cc-45dd-836e-09927d97c006 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4f40410b7674687ab5362ac982248ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4faef13aa50e4e36b6c265fb1398fa14', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80245) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:32:24.580208 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cea2b641-f8cc-45dd-836e-09927d97c006 service cinder] REQUEST_METHOD: `GET` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:32:24.580491 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cea2b641-f8cc-45dd-836e-09927d97c006 service cinder] SCRIPT_NAME: `/identity` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:32:24.580766 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cea2b641-f8cc-45dd-836e-09927d97c006 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:32:24.600896 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cea2b641-f8cc-45dd-836e-09927d97c006 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:32:24.602349 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cea2b641-f8cc-45dd-836e-09927d97c006 service cinder] RBAC: Authorization granted {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:32:24.622792 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: [pid: 80245|app: 0|req: 1243/2477] 10.222.0.47 () {60 vars in 1365 bytes} [Thu Jul 21 08:32:24 2022] GET /identity/v3/auth/tokens => generated 3325 bytes in 130 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 21 08:32:24.653911 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44148c6d-b0a0-423d-b800-392d583ff433 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:32:24.655861 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44148c6d-b0a0-423d-b800-392d583ff433 service cinder] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:32:24.666927 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 1235/2478] 10.222.0.47 () {60 vars in 1365 bytes} [Thu Jul 21 08:32:24 2022] GET /identity/v3/auth/tokens => generated 3266 bytes in 296 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 08:32:24.675152 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cea2b641-f8cc-45dd-836e-09927d97c006 service cinder] Authenticating user token {{(pid=80245) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:32:24.684019 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cea2b641-f8cc-45dd-836e-09927d97c006 service cinder] Validating token access rules against request {{(pid=80245) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:32:24.687785 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e66d338-26b1-4ea0-8781-37b2d7a16ebe service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4f40410b7674687ab5362ac982248ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4faef13aa50e4e36b6c265fb1398fa14', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80245) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:32:24.688682 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e66d338-26b1-4ea0-8781-37b2d7a16ebe service cinder] REQUEST_METHOD: `GET` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:32:24.689001 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e66d338-26b1-4ea0-8781-37b2d7a16ebe service cinder] SCRIPT_NAME: `/identity` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:32:24.689287 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e66d338-26b1-4ea0-8781-37b2d7a16ebe service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:32:24.745886 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG dbcounter [-] [80245] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80245) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:32:24.774133 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e66d338-26b1-4ea0-8781-37b2d7a16ebe service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:32:24.775752 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e66d338-26b1-4ea0-8781-37b2d7a16ebe service cinder] RBAC: Authorization granted {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:32:24.786275 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: [pid: 80245|app: 0|req: 1244/2479] 10.222.0.47 () {60 vars in 1365 bytes} [Thu Jul 21 08:32:24 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 111 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 08:32:34.661600 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG dbcounter [-] [80244] Writing DB stats keystone:SELECT=74 {{(pid=80244) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:32:34.781885 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG dbcounter [-] [80245] Writing DB stats keystone:SELECT=10 {{(pid=80245) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:32:43.185428 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44148c6d-b0a0-423d-b800-392d583ff433 service cinder] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:32:43.248660 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44148c6d-b0a0-423d-b800-392d583ff433 service cinder] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:32:43.254316 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04ca0761-eb01-43ae-8b5a-77802c966dbb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '42d15fc0515945acacbbffcf2356c7a5', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:32:43.255402 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04ca0761-eb01-43ae-8b5a-77802c966dbb admin admin] REQUEST_METHOD: `DELETE` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:32:43.255692 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04ca0761-eb01-43ae-8b5a-77802c966dbb admin admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:32:43.255985 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04ca0761-eb01-43ae-8b5a-77802c966dbb admin admin] PATH_INFO: `/v3/users/9669d6d508344f1da1fa98c6bbc75854` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:32:43.257359 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04ca0761-eb01-43ae-8b5a-77802c966dbb admin admin] RBAC: Authorizing `identity:delete_user(user_id=9669d6d508344f1da1fa98c6bbc75854)` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:32:43.259410 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04ca0761-eb01-43ae-8b5a-77802c966dbb admin admin] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:32:43.304371 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.notifications [None req-04ca0761-eb01-43ae-8b5a-77802c966dbb admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9669d6d508344f1da1fa98c6bbc75854'} {{(pid=80244) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 08:32:43.311658 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.notifications [None req-04ca0761-eb01-43ae-8b5a-77802c966dbb admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9669d6d508344f1da1fa98c6bbc75854'} {{(pid=80244) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 08:32:43.324142 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.notifications [None req-04ca0761-eb01-43ae-8b5a-77802c966dbb admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9669d6d508344f1da1fa98c6bbc75854'} {{(pid=80244) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 08:32:43.325047 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.notifications [None req-04ca0761-eb01-43ae-8b5a-77802c966dbb admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9669d6d508344f1da1fa98c6bbc75854'} {{(pid=80244) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 08:32:43.332010 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.notifications [None req-04ca0761-eb01-43ae-8b5a-77802c966dbb admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9669d6d508344f1da1fa98c6bbc75854'} {{(pid=80244) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 08:32:43.338019 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 1236/2480] 10.222.0.47 () {58 vars in 1223 bytes} [Thu Jul 21 08:32:43 2022] DELETE /identity/v3/users/9669d6d508344f1da1fa98c6bbc75854 => generated 0 bytes in 153 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 08:32:43.343173 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6585526-0bfa-4f4c-a09c-87e4e2338f2b None None] REQUEST_METHOD: `POST` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:32:43.343706 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6585526-0bfa-4f4c-a09c-87e4e2338f2b None None] SCRIPT_NAME: `/identity` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:32:43.343826 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6585526-0bfa-4f4c-a09c-87e4e2338f2b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:32:43.362492 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.auth.core [None req-e6585526-0bfa-4f4c-a09c-87e4e2338f2b None None] MFA Rules not processed for user `2dce40b8d74c41ec94cc36e6ff05d412`. Rule list: `[]` (Enabled: `True`). {{(pid=80245) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 08:32:43.363785 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.fernet_utils [None req-e6585526-0bfa-4f4c-a09c-87e4e2338f2b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80245) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 08:32:43.423766 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: [pid: 80245|app: 0|req: 1245/2481] 10.222.0.47 () {58 vars in 957 bytes} [Thu Jul 21 08:32:43 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 08:32:43.433979 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04ca0761-eb01-43ae-8b5a-77802c966dbb admin admin] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:32:43.435545 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.fernet_utils [None req-04ca0761-eb01-43ae-8b5a-77802c966dbb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80244) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 08:32:43.499224 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04ca0761-eb01-43ae-8b5a-77802c966dbb admin admin] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:32:43.502826 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69636cbf-8014-42c9-9a67-fc211314ec02 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20ca3c3bd3774691b3374cfbfec6e494', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4faef13aa50e4e36b6c265fb1398fa14', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:32:43.503512 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69636cbf-8014-42c9-9a67-fc211314ec02 service neutron] REQUEST_METHOD: `GET` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:32:43.504042 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69636cbf-8014-42c9-9a67-fc211314ec02 service neutron] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:32:43.504372 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69636cbf-8014-42c9-9a67-fc211314ec02 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:32:43.514889 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69636cbf-8014-42c9-9a67-fc211314ec02 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:32:43.516364 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69636cbf-8014-42c9-9a67-fc211314ec02 service neutron] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:32:43.526406 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 1237/2482] 10.222.0.47 () {60 vars in 1365 bytes} [Thu Jul 21 08:32:43 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 08:32:43.548196 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6585526-0bfa-4f4c-a09c-87e4e2338f2b None None] Authenticating user token {{(pid=80245) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:32:43.549942 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.fernet_utils [None req-e6585526-0bfa-4f4c-a09c-87e4e2338f2b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80245) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 08:32:43.568976 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6585526-0bfa-4f4c-a09c-87e4e2338f2b None None] Validating token access rules against request {{(pid=80245) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:32:43.574455 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bce0d544-576b-4dcf-924e-c5dc1c9a1046 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '42d15fc0515945acacbbffcf2356c7a5', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80245) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:32:43.575377 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bce0d544-576b-4dcf-924e-c5dc1c9a1046 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:32:43.575668 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bce0d544-576b-4dcf-924e-c5dc1c9a1046 admin admin] SCRIPT_NAME: `/identity` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:32:43.575955 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bce0d544-576b-4dcf-924e-c5dc1c9a1046 admin admin] PATH_INFO: `/v3/users/2c530c830bd8483282c78c595bbebf50` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:32:43.577270 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bce0d544-576b-4dcf-924e-c5dc1c9a1046 admin admin] RBAC: Authorizing `identity:delete_user(user_id=2c530c830bd8483282c78c595bbebf50)` {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:32:43.579542 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bce0d544-576b-4dcf-924e-c5dc1c9a1046 admin admin] RBAC: Authorization granted {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:32:43.638105 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.notifications [None req-bce0d544-576b-4dcf-924e-c5dc1c9a1046 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2c530c830bd8483282c78c595bbebf50'} {{(pid=80245) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 08:32:43.645155 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.notifications [None req-bce0d544-576b-4dcf-924e-c5dc1c9a1046 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2c530c830bd8483282c78c595bbebf50'} {{(pid=80245) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 08:32:43.659891 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.notifications [None req-bce0d544-576b-4dcf-924e-c5dc1c9a1046 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2c530c830bd8483282c78c595bbebf50'} {{(pid=80245) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 08:32:43.671045 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.notifications [None req-bce0d544-576b-4dcf-924e-c5dc1c9a1046 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2c530c830bd8483282c78c595bbebf50'} {{(pid=80245) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 08:32:43.671860 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.notifications [None req-bce0d544-576b-4dcf-924e-c5dc1c9a1046 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2c530c830bd8483282c78c595bbebf50'} {{(pid=80245) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 08:32:43.681433 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: [pid: 80245|app: 0|req: 1246/2483] 10.222.0.47 () {58 vars in 1223 bytes} [Thu Jul 21 08:32:43 2022] DELETE /identity/v3/users/2c530c830bd8483282c78c595bbebf50 => generated 0 bytes in 134 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 08:32:43.690835 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87c05037-3824-4822-97ea-d09a7e3ac7c9 None None] REQUEST_METHOD: `POST` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:32:43.691203 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87c05037-3824-4822-97ea-d09a7e3ac7c9 None None] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:32:43.691556 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87c05037-3824-4822-97ea-d09a7e3ac7c9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:32:43.716757 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.auth.core [None req-87c05037-3824-4822-97ea-d09a7e3ac7c9 None None] MFA Rules not processed for user `2dce40b8d74c41ec94cc36e6ff05d412`. Rule list: `[]` (Enabled: `True`). {{(pid=80244) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 08:32:43.716757 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.fernet_utils [None req-87c05037-3824-4822-97ea-d09a7e3ac7c9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80244) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 08:32:43.737115 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG dbcounter [-] [80244] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80244) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:32:43.830491 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bce0d544-576b-4dcf-924e-c5dc1c9a1046 admin admin] Authenticating user token {{(pid=80245) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:32:43.833651 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.fernet_utils [None req-bce0d544-576b-4dcf-924e-c5dc1c9a1046 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80245) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 08:32:43.865601 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bce0d544-576b-4dcf-924e-c5dc1c9a1046 admin admin] Validating token access rules against request {{(pid=80245) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:32:43.870603 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 1238/2484] 10.222.0.47 () {58 vars in 957 bytes} [Thu Jul 21 08:32:43 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 182 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 08:32:43.871515 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-528d31fa-4342-4f80-8f63-8037954683d5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '42d15fc0515945acacbbffcf2356c7a5', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80245) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:32:43.873215 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-528d31fa-4342-4f80-8f63-8037954683d5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:32:43.873762 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-528d31fa-4342-4f80-8f63-8037954683d5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:32:43.874256 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-528d31fa-4342-4f80-8f63-8037954683d5 admin admin] PATH_INFO: `/v3/projects/221cee8a42424d84b6e5629d8bcfd5e0` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:32:43.875929 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-528d31fa-4342-4f80-8f63-8037954683d5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=221cee8a42424d84b6e5629d8bcfd5e0)` {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:32:43.877929 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-528d31fa-4342-4f80-8f63-8037954683d5 admin admin] RBAC: Authorization granted {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:32:43.882637 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87c05037-3824-4822-97ea-d09a7e3ac7c9 None None] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:32:43.884284 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.fernet_utils [None req-87c05037-3824-4822-97ea-d09a7e3ac7c9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80244) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 08:32:43.951062 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.notifications [None req-528d31fa-4342-4f80-8f63-8037954683d5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '221cee8a42424d84b6e5629d8bcfd5e0', 'request_id': 'req-528d31fa-4342-4f80-8f63-8037954683d5'} {{(pid=80245) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 08:32:43.959577 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87c05037-3824-4822-97ea-d09a7e3ac7c9 None None] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:32:43.960458 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.notifications [None req-528d31fa-4342-4f80-8f63-8037954683d5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '221cee8a42424d84b6e5629d8bcfd5e0', 'request_id': 'req-528d31fa-4342-4f80-8f63-8037954683d5'} {{(pid=80245) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 08:32:43.966826 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.notifications [None req-528d31fa-4342-4f80-8f63-8037954683d5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '221cee8a42424d84b6e5629d8bcfd5e0', 'request_id': 'req-528d31fa-4342-4f80-8f63-8037954683d5'} {{(pid=80245) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 08:32:43.971749 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG dbcounter [-] [80245] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=80245) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:32:44.004024 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.notifications [None req-528d31fa-4342-4f80-8f63-8037954683d5 admin admin] The token cache is being invalidate because project 221cee8a42424d84b6e5629d8bcfd5e0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80245) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 21 08:32:44.004574 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.notifications [None req-528d31fa-4342-4f80-8f63-8037954683d5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80245) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 08:32:44.005409 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.notifications [None req-528d31fa-4342-4f80-8f63-8037954683d5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80245) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 08:32:44.009016 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: [pid: 80245|app: 0|req: 1247/2485] 10.222.0.47 () {58 vars in 1232 bytes} [Thu Jul 21 08:32:43 2022] DELETE /identity/v3/projects/221cee8a42424d84b6e5629d8bcfd5e0 => generated 0 bytes in 179 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 08:32:44.012707 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-528d31fa-4342-4f80-8f63-8037954683d5 admin admin] Authenticating user token {{(pid=80245) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:32:44.014389 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.fernet_utils [None req-528d31fa-4342-4f80-8f63-8037954683d5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80245) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 08:32:44.034799 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df6779e2-ec51-47c2-ade3-d8206e8d0eaf service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20ca3c3bd3774691b3374cfbfec6e494', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4faef13aa50e4e36b6c265fb1398fa14', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:32:44.035663 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df6779e2-ec51-47c2-ade3-d8206e8d0eaf service neutron] REQUEST_METHOD: `GET` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:32:44.036076 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df6779e2-ec51-47c2-ade3-d8206e8d0eaf service neutron] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:32:44.036359 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df6779e2-ec51-47c2-ade3-d8206e8d0eaf service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:32:44.038389 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.fernet_utils [None req-df6779e2-ec51-47c2-ade3-d8206e8d0eaf service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80244) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 08:32:44.109245 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-528d31fa-4342-4f80-8f63-8037954683d5 admin admin] Validating token access rules against request {{(pid=80245) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:32:44.138125 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c443b6b7-6201-47b3-b9d1-d293a9c2d83f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '42d15fc0515945acacbbffcf2356c7a5', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80245) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:32:44.139974 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c443b6b7-6201-47b3-b9d1-d293a9c2d83f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:32:44.140294 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c443b6b7-6201-47b3-b9d1-d293a9c2d83f admin admin] SCRIPT_NAME: `/identity` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:32:44.140765 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c443b6b7-6201-47b3-b9d1-d293a9c2d83f admin admin] PATH_INFO: `/v3/users/436a6f89d8654f57958ea8e07badfb93` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:32:44.143089 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c443b6b7-6201-47b3-b9d1-d293a9c2d83f admin admin] RBAC: Authorizing `identity:delete_user(user_id=436a6f89d8654f57958ea8e07badfb93)` {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:32:44.145397 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c443b6b7-6201-47b3-b9d1-d293a9c2d83f admin admin] RBAC: Authorization granted {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:32:44.179291 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df6779e2-ec51-47c2-ade3-d8206e8d0eaf service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:32:44.181226 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df6779e2-ec51-47c2-ade3-d8206e8d0eaf service neutron] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:32:44.210239 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 1239/2486] 10.222.0.47 () {60 vars in 1365 bytes} [Thu Jul 21 08:32:43 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 339 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 08:32:44.242853 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.notifications [None req-c443b6b7-6201-47b3-b9d1-d293a9c2d83f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '436a6f89d8654f57958ea8e07badfb93'} {{(pid=80245) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 08:32:44.255066 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.notifications [None req-c443b6b7-6201-47b3-b9d1-d293a9c2d83f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '436a6f89d8654f57958ea8e07badfb93'} {{(pid=80245) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 08:32:44.273195 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.notifications [None req-c443b6b7-6201-47b3-b9d1-d293a9c2d83f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '436a6f89d8654f57958ea8e07badfb93'} {{(pid=80245) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 08:32:44.281745 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.notifications [None req-c443b6b7-6201-47b3-b9d1-d293a9c2d83f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '436a6f89d8654f57958ea8e07badfb93'} {{(pid=80245) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 08:32:44.282668 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.notifications [None req-c443b6b7-6201-47b3-b9d1-d293a9c2d83f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '436a6f89d8654f57958ea8e07badfb93'} {{(pid=80245) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 08:32:44.297035 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: [pid: 80245|app: 0|req: 1248/2487] 10.222.0.47 () {58 vars in 1223 bytes} [Thu Jul 21 08:32:44 2022] DELETE /identity/v3/users/436a6f89d8654f57958ea8e07badfb93 => generated 0 bytes in 284 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 08:32:44.725325 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df6779e2-ec51-47c2-ade3-d8206e8d0eaf service neutron] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:32:44.726862 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.fernet_utils [None req-df6779e2-ec51-47c2-ade3-d8206e8d0eaf service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80244) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 08:32:44.750164 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df6779e2-ec51-47c2-ade3-d8206e8d0eaf service neutron] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:32:44.786911 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c443b6b7-6201-47b3-b9d1-d293a9c2d83f admin admin] Authenticating user token {{(pid=80245) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:32:44.788943 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.fernet_utils [None req-c443b6b7-6201-47b3-b9d1-d293a9c2d83f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80245) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 08:32:44.800780 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG dbcounter [-] [80244] Writing DB stats keystone:SELECT=100 {{(pid=80244) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:32:44.849187 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f811f6b7-19c3-4fba-b40a-f3cbe9962f15 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '42d15fc0515945acacbbffcf2356c7a5', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:32:44.850061 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f811f6b7-19c3-4fba-b40a-f3cbe9962f15 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:32:44.850345 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f811f6b7-19c3-4fba-b40a-f3cbe9962f15 admin admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:32:44.850610 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f811f6b7-19c3-4fba-b40a-f3cbe9962f15 admin admin] PATH_INFO: `/v3/projects/85beacd8d4d144faa3ad01231de4578d` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:32:44.851844 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f811f6b7-19c3-4fba-b40a-f3cbe9962f15 admin admin] RBAC: Authorizing `identity:delete_project(project_id=85beacd8d4d144faa3ad01231de4578d)` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:32:44.853740 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f811f6b7-19c3-4fba-b40a-f3cbe9962f15 admin admin] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:32:44.870758 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c443b6b7-6201-47b3-b9d1-d293a9c2d83f admin admin] Validating token access rules against request {{(pid=80245) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:32:44.875975 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44b85df1-6094-4811-834d-67255a1675d3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '42d15fc0515945acacbbffcf2356c7a5', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80245) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:32:44.877324 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44b85df1-6094-4811-834d-67255a1675d3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:32:44.877822 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44b85df1-6094-4811-834d-67255a1675d3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:32:44.881781 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44b85df1-6094-4811-834d-67255a1675d3 admin admin] PATH_INFO: `/v3/projects/b6ecbd2527fa430481e48c210e6a9f64` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:32:44.881781 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44b85df1-6094-4811-834d-67255a1675d3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b6ecbd2527fa430481e48c210e6a9f64)` {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:32:44.881781 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44b85df1-6094-4811-834d-67255a1675d3 admin admin] RBAC: Authorization granted {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:32:44.902677 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG dbcounter [-] [80245] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80245) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:32:44.912119 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.notifications [None req-f811f6b7-19c3-4fba-b40a-f3cbe9962f15 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '85beacd8d4d144faa3ad01231de4578d', 'request_id': 'req-f811f6b7-19c3-4fba-b40a-f3cbe9962f15'} {{(pid=80244) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 08:32:44.921877 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.notifications [None req-f811f6b7-19c3-4fba-b40a-f3cbe9962f15 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '85beacd8d4d144faa3ad01231de4578d', 'request_id': 'req-f811f6b7-19c3-4fba-b40a-f3cbe9962f15'} {{(pid=80244) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 08:32:44.930392 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.notifications [None req-f811f6b7-19c3-4fba-b40a-f3cbe9962f15 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '85beacd8d4d144faa3ad01231de4578d', 'request_id': 'req-f811f6b7-19c3-4fba-b40a-f3cbe9962f15'} {{(pid=80244) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 08:32:44.943573 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.notifications [None req-f811f6b7-19c3-4fba-b40a-f3cbe9962f15 admin admin] The token cache is being invalidate because project 85beacd8d4d144faa3ad01231de4578d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80244) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 21 08:32:44.943875 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.notifications [None req-f811f6b7-19c3-4fba-b40a-f3cbe9962f15 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80244) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 08:32:44.945029 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.notifications [None req-f811f6b7-19c3-4fba-b40a-f3cbe9962f15 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80244) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 08:32:44.946852 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 1240/2488] 10.222.0.47 () {58 vars in 1232 bytes} [Thu Jul 21 08:32:44 2022] DELETE /identity/v3/projects/85beacd8d4d144faa3ad01231de4578d => generated 0 bytes in 222 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 08:32:44.962523 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.notifications [None req-44b85df1-6094-4811-834d-67255a1675d3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b6ecbd2527fa430481e48c210e6a9f64', 'request_id': 'req-44b85df1-6094-4811-834d-67255a1675d3'} {{(pid=80245) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 08:32:44.973289 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.notifications [None req-44b85df1-6094-4811-834d-67255a1675d3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b6ecbd2527fa430481e48c210e6a9f64', 'request_id': 'req-44b85df1-6094-4811-834d-67255a1675d3'} {{(pid=80245) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 08:32:44.981499 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.notifications [None req-44b85df1-6094-4811-834d-67255a1675d3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b6ecbd2527fa430481e48c210e6a9f64', 'request_id': 'req-44b85df1-6094-4811-834d-67255a1675d3'} {{(pid=80245) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 08:32:45.000423 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.notifications [None req-44b85df1-6094-4811-834d-67255a1675d3 admin admin] The token cache is being invalidate because project b6ecbd2527fa430481e48c210e6a9f64 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80245) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 21 08:32:45.001076 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.notifications [None req-44b85df1-6094-4811-834d-67255a1675d3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80245) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 08:32:45.002017 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.notifications [None req-44b85df1-6094-4811-834d-67255a1675d3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80245) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 08:32:45.011119 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: [pid: 80245|app: 0|req: 1249/2489] 10.222.0.47 () {58 vars in 1232 bytes} [Thu Jul 21 08:32:44 2022] DELETE /identity/v3/projects/b6ecbd2527fa430481e48c210e6a9f64 => generated 0 bytes in 218 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 21 08:32:45.014050 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7c8de00-3b84-4874-a447-52fbeec14a5e None None] REQUEST_METHOD: `POST` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:32:45.014496 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7c8de00-3b84-4874-a447-52fbeec14a5e None None] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:32:45.014893 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7c8de00-3b84-4874-a447-52fbeec14a5e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:32:45.075168 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.auth.core [None req-d7c8de00-3b84-4874-a447-52fbeec14a5e None None] MFA Rules not processed for user `2dce40b8d74c41ec94cc36e6ff05d412`. Rule list: `[]` (Enabled: `True`). {{(pid=80244) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 08:32:45.076696 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.fernet_utils [None req-d7c8de00-3b84-4874-a447-52fbeec14a5e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80244) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 08:32:45.168934 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 1241/2490] 10.222.0.47 () {58 vars in 957 bytes} [Thu Jul 21 08:32:45 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 156 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 08:32:45.177166 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33d56bc2-dd24-4c7c-9d41-c42d0926ddce None None] REQUEST_METHOD: `POST` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:32:45.177480 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33d56bc2-dd24-4c7c-9d41-c42d0926ddce None None] SCRIPT_NAME: `/identity` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:32:45.177765 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33d56bc2-dd24-4c7c-9d41-c42d0926ddce None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:32:45.198329 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.auth.core [None req-33d56bc2-dd24-4c7c-9d41-c42d0926ddce None None] MFA Rules not processed for user `2dce40b8d74c41ec94cc36e6ff05d412`. Rule list: `[]` (Enabled: `True`). {{(pid=80245) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 08:32:45.199707 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.fernet_utils [None req-33d56bc2-dd24-4c7c-9d41-c42d0926ddce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80245) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 08:32:45.210623 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: [pid: 80245|app: 0|req: 1250/2491] 10.222.0.47 () {58 vars in 957 bytes} [Thu Jul 21 08:32:45 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 08:32:45.214692 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7c8de00-3b84-4874-a447-52fbeec14a5e None None] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:32:45.224879 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7c8de00-3b84-4874-a447-52fbeec14a5e None None] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:32:45.232678 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a113eca-735d-4025-8b96-59acfe1131d3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '42d15fc0515945acacbbffcf2356c7a5', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:32:45.233368 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a113eca-735d-4025-8b96-59acfe1131d3 admin admin] REQUEST_METHOD: `GET` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:32:45.233711 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a113eca-735d-4025-8b96-59acfe1131d3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:32:45.234061 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a113eca-735d-4025-8b96-59acfe1131d3 admin admin] PATH_INFO: `/v3/domains` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:32:45.235142 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a113eca-735d-4025-8b96-59acfe1131d3 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80244) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 08:32:45.235499 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a113eca-735d-4025-8b96-59acfe1131d3 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:32:45.238698 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a113eca-735d-4025-8b96-59acfe1131d3 admin admin] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:32:45.253144 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 1242/2492] 10.222.0.47 () {58 vars in 1152 bytes} [Thu Jul 21 08:32:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 21 08:32:45.258361 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33d56bc2-dd24-4c7c-9d41-c42d0926ddce None None] Authenticating user token {{(pid=80245) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:32:45.273000 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33d56bc2-dd24-4c7c-9d41-c42d0926ddce None None] Validating token access rules against request {{(pid=80245) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:32:45.284253 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cbec355-6cd1-4597-bdaa-b29169f95ef4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '42d15fc0515945acacbbffcf2356c7a5', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80245) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:32:45.285413 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cbec355-6cd1-4597-bdaa-b29169f95ef4 admin admin] REQUEST_METHOD: `POST` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:32:45.286044 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cbec355-6cd1-4597-bdaa-b29169f95ef4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:32:45.286406 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cbec355-6cd1-4597-bdaa-b29169f95ef4 admin admin] PATH_INFO: `/v3/projects` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:32:45.295098 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cbec355-6cd1-4597-bdaa-b29169f95ef4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:32:45.297088 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cbec355-6cd1-4597-bdaa-b29169f95ef4 admin admin] RBAC: Authorization granted {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:32:45.326568 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: [pid: 80245|app: 0|req: 1251/2493] 10.222.0.47 () {60 vars in 1152 bytes} [Thu Jul 21 08:32:45 2022] POST /identity/v3/projects => generated 373 bytes in 69 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 08:32:45.331692 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a113eca-735d-4025-8b96-59acfe1131d3 admin admin] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:32:45.343349 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a113eca-735d-4025-8b96-59acfe1131d3 admin admin] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:32:45.403019 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG dbcounter [-] [80244] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80244) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:32:45.422187 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b173218a-8c5b-463c-9aa1-b5ee8be9ee53 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '42d15fc0515945acacbbffcf2356c7a5', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:32:45.422953 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b173218a-8c5b-463c-9aa1-b5ee8be9ee53 admin admin] REQUEST_METHOD: `POST` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:32:45.423158 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b173218a-8c5b-463c-9aa1-b5ee8be9ee53 admin admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:32:45.423451 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b173218a-8c5b-463c-9aa1-b5ee8be9ee53 admin admin] PATH_INFO: `/v3/users` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:32:45.424188 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b173218a-8c5b-463c-9aa1-b5ee8be9ee53 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:32:45.426170 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b173218a-8c5b-463c-9aa1-b5ee8be9ee53 admin admin] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:32:45.448292 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 1243/2494] 10.222.0.47 () {60 vars in 1143 bytes} [Thu Jul 21 08:32:45 2022] POST /identity/v3/users => generated 287 bytes in 117 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 08:32:45.453344 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cbec355-6cd1-4597-bdaa-b29169f95ef4 admin admin] Authenticating user token {{(pid=80245) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:32:45.466733 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cbec355-6cd1-4597-bdaa-b29169f95ef4 admin admin] Validating token access rules against request {{(pid=80245) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:32:45.471826 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a08c65e2-a609-4948-bee7-d37f4623efdc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '42d15fc0515945acacbbffcf2356c7a5', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80245) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:32:45.472813 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a08c65e2-a609-4948-bee7-d37f4623efdc admin admin] REQUEST_METHOD: `GET` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:32:45.474391 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a08c65e2-a609-4948-bee7-d37f4623efdc admin admin] SCRIPT_NAME: `/identity` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:32:45.474391 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a08c65e2-a609-4948-bee7-d37f4623efdc admin admin] PATH_INFO: `/v3/roles` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:32:45.474391 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a08c65e2-a609-4948-bee7-d37f4623efdc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:32:45.475834 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a08c65e2-a609-4948-bee7-d37f4623efdc admin admin] RBAC: Authorization granted {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:32:45.487442 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: [pid: 80245|app: 0|req: 1252/2495] 10.222.0.47 () {58 vars in 1121 bytes} [Thu Jul 21 08:32:45 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 08:32:45.491694 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b173218a-8c5b-463c-9aa1-b5ee8be9ee53 admin admin] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:32:45.504060 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b173218a-8c5b-463c-9aa1-b5ee8be9ee53 admin admin] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:32:45.508833 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe85e0ba-0600-406d-bf30-6f03c8cde4f6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '42d15fc0515945acacbbffcf2356c7a5', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:32:45.509558 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe85e0ba-0600-406d-bf30-6f03c8cde4f6 admin admin] REQUEST_METHOD: `GET` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:32:45.509998 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe85e0ba-0600-406d-bf30-6f03c8cde4f6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:32:45.510279 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe85e0ba-0600-406d-bf30-6f03c8cde4f6 admin admin] PATH_INFO: `/v3/roles` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:32:45.511237 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe85e0ba-0600-406d-bf30-6f03c8cde4f6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:32:45.513445 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe85e0ba-0600-406d-bf30-6f03c8cde4f6 admin admin] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:32:45.527471 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 1244/2496] 10.222.0.47 () {58 vars in 1121 bytes} [Thu Jul 21 08:32:45 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 08:32:45.531807 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a08c65e2-a609-4948-bee7-d37f4623efdc admin admin] Authenticating user token {{(pid=80245) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:32:45.545266 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a08c65e2-a609-4948-bee7-d37f4623efdc admin admin] Validating token access rules against request {{(pid=80245) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:32:45.550825 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-246db486-417d-4646-87c8-b25c53a29a87 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '42d15fc0515945acacbbffcf2356c7a5', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80245) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:32:45.551881 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-246db486-417d-4646-87c8-b25c53a29a87 admin admin] REQUEST_METHOD: `PUT` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:32:45.552169 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-246db486-417d-4646-87c8-b25c53a29a87 admin admin] SCRIPT_NAME: `/identity` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:32:45.552441 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-246db486-417d-4646-87c8-b25c53a29a87 admin admin] PATH_INFO: `/v3/projects/ccb36757791d463e92faf5dbaa0ff13a/users/45688a2eafb840e9b2b4c56bbee69375/roles/3a074738221e4601b5205715ba87d240` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:32:45.573420 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-246db486-417d-4646-87c8-b25c53a29a87 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ccb36757791d463e92faf5dbaa0ff13a, user_id=45688a2eafb840e9b2b4c56bbee69375, role_id=3a074738221e4601b5205715ba87d240)` {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:32:45.577967 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-246db486-417d-4646-87c8-b25c53a29a87 admin admin] RBAC: Authorization granted {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:32:45.606765 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: [pid: 80245|app: 0|req: 1253/2497] 10.222.0.47 () {60 vars in 1482 bytes} [Thu Jul 21 08:32:45 2022] PUT /identity/v3/projects/ccb36757791d463e92faf5dbaa0ff13a/users/45688a2eafb840e9b2b4c56bbee69375/roles/3a074738221e4601b5205715ba87d240 => generated 0 bytes in 76 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 21 08:32:45.612337 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ba66310-7f24-4af0-97b4-344e699fa2d9 None None] REQUEST_METHOD: `POST` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:32:45.612808 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ba66310-7f24-4af0-97b4-344e699fa2d9 None None] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:32:45.613094 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ba66310-7f24-4af0-97b4-344e699fa2d9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:32:45.636614 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.auth.core [None req-9ba66310-7f24-4af0-97b4-344e699fa2d9 None None] MFA Rules not processed for user `2dce40b8d74c41ec94cc36e6ff05d412`. Rule list: `[]` (Enabled: `True`). {{(pid=80244) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 08:32:45.638073 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.fernet_utils [None req-9ba66310-7f24-4af0-97b4-344e699fa2d9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80244) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 08:32:45.735664 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 1245/2498] 10.222.0.47 () {58 vars in 957 bytes} [Thu Jul 21 08:32:45 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 08:32:45.752683 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-246db486-417d-4646-87c8-b25c53a29a87 admin admin] Authenticating user token {{(pid=80245) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:32:45.754921 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.fernet_utils [None req-246db486-417d-4646-87c8-b25c53a29a87 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80245) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 08:32:45.847074 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-246db486-417d-4646-87c8-b25c53a29a87 admin admin] Validating token access rules against request {{(pid=80245) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:32:45.851088 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5af7870a-e36e-4e78-983f-c545a10b876c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20ca3c3bd3774691b3374cfbfec6e494', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4faef13aa50e4e36b6c265fb1398fa14', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80245) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:32:45.851770 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5af7870a-e36e-4e78-983f-c545a10b876c service neutron] REQUEST_METHOD: `GET` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:32:45.852052 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5af7870a-e36e-4e78-983f-c545a10b876c service neutron] SCRIPT_NAME: `/identity` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:32:45.852335 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5af7870a-e36e-4e78-983f-c545a10b876c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:32:45.867885 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5af7870a-e36e-4e78-983f-c545a10b876c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:32:45.869524 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5af7870a-e36e-4e78-983f-c545a10b876c service neutron] RBAC: Authorization granted {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:32:45.892950 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: [pid: 80245|app: 0|req: 1254/2499] 10.222.0.47 () {60 vars in 1365 bytes} [Thu Jul 21 08:32:45 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 141 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 08:32:51.179733 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93564cc5-73a9-4cda-aae9-efa2c3f5494f None None] REQUEST_METHOD: `POST` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:32:51.179733 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93564cc5-73a9-4cda-aae9-efa2c3f5494f None None] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:32:51.179733 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93564cc5-73a9-4cda-aae9-efa2c3f5494f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:32:51.225210 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.auth.core [None req-93564cc5-73a9-4cda-aae9-efa2c3f5494f None None] MFA Rules not processed for user `45688a2eafb840e9b2b4c56bbee69375`. Rule list: `[]` (Enabled: `True`). {{(pid=80244) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 08:32:51.226552 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.fernet_utils [None req-93564cc5-73a9-4cda-aae9-efa2c3f5494f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80244) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 08:32:51.362706 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 1246/2500] 10.222.0.47 () {58 vars in 957 bytes} [Thu Jul 21 08:32:51 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 187 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 08:32:51.367921 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5af7870a-e36e-4e78-983f-c545a10b876c service neutron] Authenticating user token {{(pid=80245) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:32:51.388198 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5af7870a-e36e-4e78-983f-c545a10b876c service neutron] Validating token access rules against request {{(pid=80245) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:32:51.402376 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a5105ae-1cc9-4f37-a83a-423f1671864c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '42d15fc0515945acacbbffcf2356c7a5', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80245) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:32:51.403373 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a5105ae-1cc9-4f37-a83a-423f1671864c admin admin] REQUEST_METHOD: `POST` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:32:51.403812 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a5105ae-1cc9-4f37-a83a-423f1671864c admin admin] SCRIPT_NAME: `/identity` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:32:51.404243 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a5105ae-1cc9-4f37-a83a-423f1671864c admin admin] PATH_INFO: `/v3/projects` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:32:51.408671 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a5105ae-1cc9-4f37-a83a-423f1671864c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:32:51.429621 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a5105ae-1cc9-4f37-a83a-423f1671864c admin admin] RBAC: Authorization granted {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:32:51.514915 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: [pid: 80245|app: 0|req: 1255/2501] 10.222.0.47 () {60 vars in 1152 bytes} [Thu Jul 21 08:32:51 2022] POST /identity/v3/projects => generated 373 bytes in 146 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 08:32:51.516663 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93564cc5-73a9-4cda-aae9-efa2c3f5494f None None] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:32:51.529790 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93564cc5-73a9-4cda-aae9-efa2c3f5494f None None] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:32:51.606225 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG dbcounter [-] [80244] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80244) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:32:51.616128 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc2791c9-4004-45d5-9602-b95233442da9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '42d15fc0515945acacbbffcf2356c7a5', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:32:51.616846 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc2791c9-4004-45d5-9602-b95233442da9 admin admin] REQUEST_METHOD: `POST` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:32:51.617165 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc2791c9-4004-45d5-9602-b95233442da9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:32:51.617445 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc2791c9-4004-45d5-9602-b95233442da9 admin admin] PATH_INFO: `/v3/users` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:32:51.618149 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc2791c9-4004-45d5-9602-b95233442da9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:32:51.620789 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc2791c9-4004-45d5-9602-b95233442da9 admin admin] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:32:51.646774 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a5105ae-1cc9-4f37-a83a-423f1671864c admin admin] Authenticating user token {{(pid=80245) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:32:51.647203 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 1247/2502] 10.222.0.47 () {60 vars in 1143 bytes} [Thu Jul 21 08:32:51 2022] POST /identity/v3/users => generated 293 bytes in 129 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 08:32:51.658067 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a5105ae-1cc9-4f37-a83a-423f1671864c admin admin] Validating token access rules against request {{(pid=80245) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:32:51.666024 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06d69687-4650-4c9e-a588-f4a65e8f582d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '42d15fc0515945acacbbffcf2356c7a5', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80245) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:32:51.666749 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06d69687-4650-4c9e-a588-f4a65e8f582d admin admin] REQUEST_METHOD: `GET` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:32:51.667040 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06d69687-4650-4c9e-a588-f4a65e8f582d admin admin] SCRIPT_NAME: `/identity` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:32:51.667347 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06d69687-4650-4c9e-a588-f4a65e8f582d admin admin] PATH_INFO: `/v3/roles` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:32:51.668133 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06d69687-4650-4c9e-a588-f4a65e8f582d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:32:51.669720 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06d69687-4650-4c9e-a588-f4a65e8f582d admin admin] RBAC: Authorization granted {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:32:51.683813 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: [pid: 80245|app: 0|req: 1256/2503] 10.222.0.47 () {58 vars in 1121 bytes} [Thu Jul 21 08:32:51 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 08:32:51.688586 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc2791c9-4004-45d5-9602-b95233442da9 admin admin] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:32:51.700626 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc2791c9-4004-45d5-9602-b95233442da9 admin admin] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:32:51.705898 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80e32516-1707-4e27-a0b7-f3ae07e67327 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '42d15fc0515945acacbbffcf2356c7a5', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:32:51.706799 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80e32516-1707-4e27-a0b7-f3ae07e67327 admin admin] REQUEST_METHOD: `PUT` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:32:51.707152 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80e32516-1707-4e27-a0b7-f3ae07e67327 admin admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:32:51.707576 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80e32516-1707-4e27-a0b7-f3ae07e67327 admin admin] PATH_INFO: `/v3/projects/56b8aca439294e93b494a7d8c3177e91/users/4512e5eb630943a996a43ea931656696/roles/e29870f6e9e2411a9db0d5bf45f32ca5` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:32:51.726227 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80e32516-1707-4e27-a0b7-f3ae07e67327 admin admin] RBAC: Authorizing `identity:create_grant(project_id=56b8aca439294e93b494a7d8c3177e91, user_id=4512e5eb630943a996a43ea931656696, role_id=e29870f6e9e2411a9db0d5bf45f32ca5)` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:32:51.730366 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80e32516-1707-4e27-a0b7-f3ae07e67327 admin admin] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:32:51.746421 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 1248/2504] 10.222.0.47 () {60 vars in 1482 bytes} [Thu Jul 21 08:32:51 2022] PUT /identity/v3/projects/56b8aca439294e93b494a7d8c3177e91/users/4512e5eb630943a996a43ea931656696/roles/e29870f6e9e2411a9db0d5bf45f32ca5 => generated 0 bytes in 58 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 08:32:51.751035 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06d69687-4650-4c9e-a588-f4a65e8f582d admin admin] Authenticating user token {{(pid=80245) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:32:51.762354 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06d69687-4650-4c9e-a588-f4a65e8f582d admin admin] Validating token access rules against request {{(pid=80245) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:32:51.770902 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG dbcounter [-] [80245] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=80245) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:32:51.823587 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8245e47-5759-442a-b78b-fe021d4ee6e0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '42d15fc0515945acacbbffcf2356c7a5', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80245) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:32:51.824270 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8245e47-5759-442a-b78b-fe021d4ee6e0 admin admin] REQUEST_METHOD: `GET` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:32:51.824561 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8245e47-5759-442a-b78b-fe021d4ee6e0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:32:51.824834 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8245e47-5759-442a-b78b-fe021d4ee6e0 admin admin] PATH_INFO: `/v3/roles` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:32:51.825557 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8245e47-5759-442a-b78b-fe021d4ee6e0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:32:51.827157 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8245e47-5759-442a-b78b-fe021d4ee6e0 admin admin] RBAC: Authorization granted {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:32:51.837284 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: [pid: 80245|app: 0|req: 1257/2505] 10.222.0.47 () {58 vars in 1121 bytes} [Thu Jul 21 08:32:51 2022] GET /identity/v3/roles => generated 1420 bytes in 87 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 08:32:51.841501 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80e32516-1707-4e27-a0b7-f3ae07e67327 admin admin] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:32:51.852150 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80e32516-1707-4e27-a0b7-f3ae07e67327 admin admin] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:32:51.857395 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1303b3aa-9a57-4ae6-a784-13f9435a9b10 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '42d15fc0515945acacbbffcf2356c7a5', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:32:51.858292 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1303b3aa-9a57-4ae6-a784-13f9435a9b10 admin admin] REQUEST_METHOD: `PUT` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:32:51.858577 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1303b3aa-9a57-4ae6-a784-13f9435a9b10 admin admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:32:51.858849 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1303b3aa-9a57-4ae6-a784-13f9435a9b10 admin admin] PATH_INFO: `/v3/projects/56b8aca439294e93b494a7d8c3177e91/users/4512e5eb630943a996a43ea931656696/roles/e29870f6e9e2411a9db0d5bf45f32ca5` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:32:51.860849 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1303b3aa-9a57-4ae6-a784-13f9435a9b10 admin admin] RBAC: Authorizing `identity:create_grant(project_id=56b8aca439294e93b494a7d8c3177e91, user_id=4512e5eb630943a996a43ea931656696, role_id=e29870f6e9e2411a9db0d5bf45f32ca5)` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:32:51.863738 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1303b3aa-9a57-4ae6-a784-13f9435a9b10 admin admin] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:32:51.874953 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 1249/2506] 10.222.0.47 () {60 vars in 1482 bytes} [Thu Jul 21 08:32:51 2022] PUT /identity/v3/projects/56b8aca439294e93b494a7d8c3177e91/users/4512e5eb630943a996a43ea931656696/roles/e29870f6e9e2411a9db0d5bf45f32ca5 => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 08:32:55.154583 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8245e47-5759-442a-b78b-fe021d4ee6e0 admin admin] Authenticating user token {{(pid=80245) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:32:55.157320 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.fernet_utils [None req-c8245e47-5759-442a-b78b-fe021d4ee6e0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80245) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 08:32:55.201236 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8245e47-5759-442a-b78b-fe021d4ee6e0 admin admin] Validating token access rules against request {{(pid=80245) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:32:55.359803 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ef843ba-4e37-43e1-b64e-c12650dc280e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '42d15fc0515945acacbbffcf2356c7a5', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80245) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:32:55.360751 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ef843ba-4e37-43e1-b64e-c12650dc280e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:32:55.361038 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ef843ba-4e37-43e1-b64e-c12650dc280e admin admin] SCRIPT_NAME: `/identity` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:32:55.361317 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ef843ba-4e37-43e1-b64e-c12650dc280e admin admin] PATH_INFO: `/v3/users/baf6101307e54e6a9149f24db612f437` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:32:55.368203 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ef843ba-4e37-43e1-b64e-c12650dc280e admin admin] RBAC: Authorizing `identity:delete_user(user_id=baf6101307e54e6a9149f24db612f437)` {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:32:55.384732 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ef843ba-4e37-43e1-b64e-c12650dc280e admin admin] RBAC: Authorization granted {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:32:55.506500 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.notifications [None req-9ef843ba-4e37-43e1-b64e-c12650dc280e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'baf6101307e54e6a9149f24db612f437'} {{(pid=80245) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 08:32:55.526736 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.notifications [None req-9ef843ba-4e37-43e1-b64e-c12650dc280e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'baf6101307e54e6a9149f24db612f437'} {{(pid=80245) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 08:32:55.578724 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.notifications [None req-9ef843ba-4e37-43e1-b64e-c12650dc280e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'baf6101307e54e6a9149f24db612f437'} {{(pid=80245) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 08:32:55.602635 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.notifications [None req-9ef843ba-4e37-43e1-b64e-c12650dc280e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'baf6101307e54e6a9149f24db612f437'} {{(pid=80245) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 08:32:55.603340 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.notifications [None req-9ef843ba-4e37-43e1-b64e-c12650dc280e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'baf6101307e54e6a9149f24db612f437'} {{(pid=80245) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 08:32:55.611414 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: [pid: 80245|app: 0|req: 1258/2507] 10.222.0.47 () {58 vars in 1223 bytes} [Thu Jul 21 08:32:55 2022] DELETE /identity/v3/users/baf6101307e54e6a9149f24db612f437 => generated 0 bytes in 457 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 08:32:55.617360 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-246d169c-bb2a-414d-acb8-3b6e06b454d2 None None] REQUEST_METHOD: `POST` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:32:55.619290 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-246d169c-bb2a-414d-acb8-3b6e06b454d2 None None] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:32:55.619979 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-246d169c-bb2a-414d-acb8-3b6e06b454d2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:32:55.656861 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.auth.core [None req-246d169c-bb2a-414d-acb8-3b6e06b454d2 None None] MFA Rules not processed for user `2dce40b8d74c41ec94cc36e6ff05d412`. Rule list: `[]` (Enabled: `True`). {{(pid=80244) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 08:32:55.658752 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.fernet_utils [None req-246d169c-bb2a-414d-acb8-3b6e06b454d2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80244) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 08:32:55.754337 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 1250/2508] 10.222.0.47 () {58 vars in 957 bytes} [Thu Jul 21 08:32:55 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 140 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 08:32:55.764895 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ef843ba-4e37-43e1-b64e-c12650dc280e admin admin] Authenticating user token {{(pid=80245) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:32:55.766395 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.fernet_utils [None req-9ef843ba-4e37-43e1-b64e-c12650dc280e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80245) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 08:32:55.832101 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG dbcounter [-] [80245] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80245) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:32:55.894095 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ef843ba-4e37-43e1-b64e-c12650dc280e admin admin] Validating token access rules against request {{(pid=80245) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:32:55.898960 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0caec24-59bc-4bd5-8788-ef02ffe0650e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20ca3c3bd3774691b3374cfbfec6e494', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4faef13aa50e4e36b6c265fb1398fa14', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80245) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:32:55.899680 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0caec24-59bc-4bd5-8788-ef02ffe0650e service neutron] REQUEST_METHOD: `GET` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:32:55.899968 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0caec24-59bc-4bd5-8788-ef02ffe0650e service neutron] SCRIPT_NAME: `/identity` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:32:55.900251 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0caec24-59bc-4bd5-8788-ef02ffe0650e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:32:55.926354 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0caec24-59bc-4bd5-8788-ef02ffe0650e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:32:55.934610 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0caec24-59bc-4bd5-8788-ef02ffe0650e service neutron] RBAC: Authorization granted {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:32:55.959804 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: [pid: 80245|app: 0|req: 1259/2509] 10.222.0.47 () {60 vars in 1365 bytes} [Thu Jul 21 08:32:55 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 195 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 08:32:56.442503 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-246d169c-bb2a-414d-acb8-3b6e06b454d2 None None] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:32:56.444346 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.fernet_utils [None req-246d169c-bb2a-414d-acb8-3b6e06b454d2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80244) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 08:32:56.465542 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-246d169c-bb2a-414d-acb8-3b6e06b454d2 None None] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:32:56.475931 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f1ff61e-60ac-40f2-86a9-258c3edfc221 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '42d15fc0515945acacbbffcf2356c7a5', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:32:56.477412 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f1ff61e-60ac-40f2-86a9-258c3edfc221 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:32:56.478631 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f1ff61e-60ac-40f2-86a9-258c3edfc221 admin admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:32:56.479603 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f1ff61e-60ac-40f2-86a9-258c3edfc221 admin admin] PATH_INFO: `/v3/projects/afb8589160b94b4896e10d9096177e85` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:32:56.485369 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f1ff61e-60ac-40f2-86a9-258c3edfc221 admin admin] RBAC: Authorizing `identity:delete_project(project_id=afb8589160b94b4896e10d9096177e85)` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:32:56.487210 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f1ff61e-60ac-40f2-86a9-258c3edfc221 admin admin] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:32:56.608448 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.notifications [None req-9f1ff61e-60ac-40f2-86a9-258c3edfc221 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'afb8589160b94b4896e10d9096177e85', 'request_id': 'req-9f1ff61e-60ac-40f2-86a9-258c3edfc221'} {{(pid=80244) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 08:32:56.619067 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.notifications [None req-9f1ff61e-60ac-40f2-86a9-258c3edfc221 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'afb8589160b94b4896e10d9096177e85', 'request_id': 'req-9f1ff61e-60ac-40f2-86a9-258c3edfc221'} {{(pid=80244) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 08:32:56.628395 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.notifications [None req-9f1ff61e-60ac-40f2-86a9-258c3edfc221 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'afb8589160b94b4896e10d9096177e85', 'request_id': 'req-9f1ff61e-60ac-40f2-86a9-258c3edfc221'} {{(pid=80244) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 08:32:56.641025 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.notifications [None req-9f1ff61e-60ac-40f2-86a9-258c3edfc221 admin admin] The token cache is being invalidate because project afb8589160b94b4896e10d9096177e85 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80244) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 21 08:32:56.641666 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.notifications [None req-9f1ff61e-60ac-40f2-86a9-258c3edfc221 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80244) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 08:32:56.642609 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.notifications [None req-9f1ff61e-60ac-40f2-86a9-258c3edfc221 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80244) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 08:32:56.645298 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 1251/2510] 10.222.0.47 () {58 vars in 1232 bytes} [Thu Jul 21 08:32:56 2022] DELETE /identity/v3/projects/afb8589160b94b4896e10d9096177e85 => generated 0 bytes in 204 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 08:32:56.650161 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2376d88e-b3bd-4096-9f46-e0ced4a8c447 None None] REQUEST_METHOD: `POST` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:32:56.650582 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2376d88e-b3bd-4096-9f46-e0ced4a8c447 None None] SCRIPT_NAME: `/identity` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:32:56.650723 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2376d88e-b3bd-4096-9f46-e0ced4a8c447 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:32:56.674040 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.auth.core [None req-2376d88e-b3bd-4096-9f46-e0ced4a8c447 None None] MFA Rules not processed for user `2dce40b8d74c41ec94cc36e6ff05d412`. Rule list: `[]` (Enabled: `True`). {{(pid=80245) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 08:32:56.675922 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.fernet_utils [None req-2376d88e-b3bd-4096-9f46-e0ced4a8c447 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80245) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 08:32:56.813340 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: [pid: 80245|app: 0|req: 1260/2511] 10.222.0.47 () {58 vars in 957 bytes} [Thu Jul 21 08:32:56 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 08:32:56.822682 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80b51a34-0f47-49e3-b5cb-0959116ddf0f None None] REQUEST_METHOD: `POST` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:32:56.823561 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80b51a34-0f47-49e3-b5cb-0959116ddf0f None None] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:32:56.823847 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80b51a34-0f47-49e3-b5cb-0959116ddf0f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:32:56.872120 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.auth.core [None req-80b51a34-0f47-49e3-b5cb-0959116ddf0f None None] MFA Rules not processed for user `2dce40b8d74c41ec94cc36e6ff05d412`. Rule list: `[]` (Enabled: `True`). {{(pid=80244) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 08:32:56.873882 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.fernet_utils [None req-80b51a34-0f47-49e3-b5cb-0959116ddf0f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80244) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 08:32:56.896202 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 1252/2512] 10.222.0.47 () {58 vars in 957 bytes} [Thu Jul 21 08:32:56 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 75 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 08:32:56.900632 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2376d88e-b3bd-4096-9f46-e0ced4a8c447 None None] Authenticating user token {{(pid=80245) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:32:56.918535 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2376d88e-b3bd-4096-9f46-e0ced4a8c447 None None] Validating token access rules against request {{(pid=80245) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:32:56.933817 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7892d4c-b418-4df5-ba7c-4c3b8d3bbe18 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '42d15fc0515945acacbbffcf2356c7a5', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80245) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:32:56.935250 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7892d4c-b418-4df5-ba7c-4c3b8d3bbe18 admin admin] REQUEST_METHOD: `GET` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:32:56.935551 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7892d4c-b418-4df5-ba7c-4c3b8d3bbe18 admin admin] SCRIPT_NAME: `/identity` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:32:56.936746 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7892d4c-b418-4df5-ba7c-4c3b8d3bbe18 admin admin] PATH_INFO: `/v3/domains` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:32:56.937618 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7892d4c-b418-4df5-ba7c-4c3b8d3bbe18 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80245) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 08:32:56.937804 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7892d4c-b418-4df5-ba7c-4c3b8d3bbe18 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:32:56.943567 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7892d4c-b418-4df5-ba7c-4c3b8d3bbe18 admin admin] RBAC: Authorization granted {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:32:56.984362 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: [pid: 80245|app: 0|req: 1261/2513] 10.222.0.47 () {58 vars in 1152 bytes} [Thu Jul 21 08:32:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 84 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 08:32:56.988883 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80b51a34-0f47-49e3-b5cb-0959116ddf0f None None] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:32:57.012334 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80b51a34-0f47-49e3-b5cb-0959116ddf0f None None] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:32:57.017596 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf5369eb-3b73-4e59-8da0-6a1d9c31bc86 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '42d15fc0515945acacbbffcf2356c7a5', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:32:57.018403 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf5369eb-3b73-4e59-8da0-6a1d9c31bc86 admin admin] REQUEST_METHOD: `POST` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:32:57.018823 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf5369eb-3b73-4e59-8da0-6a1d9c31bc86 admin admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:32:57.019207 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf5369eb-3b73-4e59-8da0-6a1d9c31bc86 admin admin] PATH_INFO: `/v3/projects` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:32:57.020301 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf5369eb-3b73-4e59-8da0-6a1d9c31bc86 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:32:57.022297 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf5369eb-3b73-4e59-8da0-6a1d9c31bc86 admin admin] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:32:57.060945 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 1253/2514] 10.222.0.47 () {60 vars in 1152 bytes} [Thu Jul 21 08:32:56 2022] POST /identity/v3/projects => generated 385 bytes in 72 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 08:32:57.065192 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7892d4c-b418-4df5-ba7c-4c3b8d3bbe18 admin admin] Authenticating user token {{(pid=80245) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:32:57.100722 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7892d4c-b418-4df5-ba7c-4c3b8d3bbe18 admin admin] Validating token access rules against request {{(pid=80245) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:32:57.249894 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-031c74c3-c31d-4dde-a9d0-3dd27336f580 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '42d15fc0515945acacbbffcf2356c7a5', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80245) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:32:57.251087 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-031c74c3-c31d-4dde-a9d0-3dd27336f580 admin admin] REQUEST_METHOD: `POST` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:32:57.251611 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-031c74c3-c31d-4dde-a9d0-3dd27336f580 admin admin] SCRIPT_NAME: `/identity` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:32:57.252090 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-031c74c3-c31d-4dde-a9d0-3dd27336f580 admin admin] PATH_INFO: `/v3/users` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:32:57.253553 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-031c74c3-c31d-4dde-a9d0-3dd27336f580 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:32:57.256637 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-031c74c3-c31d-4dde-a9d0-3dd27336f580 admin admin] RBAC: Authorization granted {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:32:57.315659 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf5369eb-3b73-4e59-8da0-6a1d9c31bc86 admin admin] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:32:57.316503 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: [pid: 80245|app: 0|req: 1262/2515] 10.222.0.47 () {60 vars in 1143 bytes} [Thu Jul 21 08:32:57 2022] POST /identity/v3/users => generated 293 bytes in 252 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 08:32:57.321332 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG dbcounter [-] [80244] Writing DB stats keystone:SELECT=86,keystone:INSERT=7,keystone:DELETE=7 {{(pid=80244) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:32:57.337327 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf5369eb-3b73-4e59-8da0-6a1d9c31bc86 admin admin] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:32:57.352582 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf3b6161-e06b-49e4-b0d9-8e80554fb486 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '42d15fc0515945acacbbffcf2356c7a5', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:32:57.353280 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf3b6161-e06b-49e4-b0d9-8e80554fb486 admin admin] REQUEST_METHOD: `GET` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:32:57.353567 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf3b6161-e06b-49e4-b0d9-8e80554fb486 admin admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:32:57.354217 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf3b6161-e06b-49e4-b0d9-8e80554fb486 admin admin] PATH_INFO: `/v3/roles` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:32:57.355007 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf3b6161-e06b-49e4-b0d9-8e80554fb486 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:32:57.358882 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf3b6161-e06b-49e4-b0d9-8e80554fb486 admin admin] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:32:57.388040 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-031c74c3-c31d-4dde-a9d0-3dd27336f580 admin admin] Authenticating user token {{(pid=80245) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:32:57.389514 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 1254/2516] 10.222.0.47 () {58 vars in 1121 bytes} [Thu Jul 21 08:32:57 2022] GET /identity/v3/roles => generated 1420 bytes in 68 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 08:32:57.413643 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-031c74c3-c31d-4dde-a9d0-3dd27336f580 admin admin] Validating token access rules against request {{(pid=80245) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:32:57.427519 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cc1aca6-7a02-49ad-bc25-923d61bf5a1a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '42d15fc0515945acacbbffcf2356c7a5', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80245) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:32:57.428587 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cc1aca6-7a02-49ad-bc25-923d61bf5a1a admin admin] REQUEST_METHOD: `GET` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:32:57.428832 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cc1aca6-7a02-49ad-bc25-923d61bf5a1a admin admin] SCRIPT_NAME: `/identity` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:32:57.429398 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cc1aca6-7a02-49ad-bc25-923d61bf5a1a admin admin] PATH_INFO: `/v3/roles` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:32:57.430183 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cc1aca6-7a02-49ad-bc25-923d61bf5a1a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:32:57.438653 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cc1aca6-7a02-49ad-bc25-923d61bf5a1a admin admin] RBAC: Authorization granted {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:32:57.464487 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: [pid: 80245|app: 0|req: 1263/2517] 10.222.0.47 () {58 vars in 1121 bytes} [Thu Jul 21 08:32:57 2022] GET /identity/v3/roles => generated 1420 bytes in 77 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 08:32:57.469595 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf3b6161-e06b-49e4-b0d9-8e80554fb486 admin admin] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:32:57.505278 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf3b6161-e06b-49e4-b0d9-8e80554fb486 admin admin] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:32:57.514503 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7c28aab-2fdc-421f-accb-8b3609f3ec2a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '42d15fc0515945acacbbffcf2356c7a5', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:32:57.515840 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7c28aab-2fdc-421f-accb-8b3609f3ec2a admin admin] REQUEST_METHOD: `PUT` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:32:57.516187 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7c28aab-2fdc-421f-accb-8b3609f3ec2a admin admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:32:57.516474 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7c28aab-2fdc-421f-accb-8b3609f3ec2a admin admin] PATH_INFO: `/v3/projects/a8ccb394622d4b24930f8b6396671857/users/8249ebd6b2dc4811ad315d54030d927e/roles/3a074738221e4601b5205715ba87d240` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:32:57.538383 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7c28aab-2fdc-421f-accb-8b3609f3ec2a admin admin] RBAC: Authorizing `identity:create_grant(project_id=a8ccb394622d4b24930f8b6396671857, user_id=8249ebd6b2dc4811ad315d54030d927e, role_id=3a074738221e4601b5205715ba87d240)` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:32:57.548517 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7c28aab-2fdc-421f-accb-8b3609f3ec2a admin admin] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:32:57.608505 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 1255/2518] 10.222.0.47 () {60 vars in 1482 bytes} [Thu Jul 21 08:32:57 2022] PUT /identity/v3/projects/a8ccb394622d4b24930f8b6396671857/users/8249ebd6b2dc4811ad315d54030d927e/roles/3a074738221e4601b5205715ba87d240 => generated 0 bytes in 140 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 08:32:57.613972 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8e8c516-8e94-4982-89a8-7b1dc3e69700 None None] REQUEST_METHOD: `POST` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:32:57.614262 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8e8c516-8e94-4982-89a8-7b1dc3e69700 None None] SCRIPT_NAME: `/identity` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:32:57.614544 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8e8c516-8e94-4982-89a8-7b1dc3e69700 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:32:57.662054 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.auth.core [None req-b8e8c516-8e94-4982-89a8-7b1dc3e69700 None None] MFA Rules not processed for user `2dce40b8d74c41ec94cc36e6ff05d412`. Rule list: `[]` (Enabled: `True`). {{(pid=80245) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 08:32:57.665291 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.fernet_utils [None req-b8e8c516-8e94-4982-89a8-7b1dc3e69700 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80245) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 08:32:57.770911 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG dbcounter [-] [80245] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80245) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:32:57.826076 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: [pid: 80245|app: 0|req: 1264/2519] 10.222.0.47 () {58 vars in 957 bytes} [Thu Jul 21 08:32:57 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 210 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 08:32:57.853029 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7c28aab-2fdc-421f-accb-8b3609f3ec2a admin admin] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:32:57.855074 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.fernet_utils [None req-e7c28aab-2fdc-421f-accb-8b3609f3ec2a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80244) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 08:32:57.984803 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7c28aab-2fdc-421f-accb-8b3609f3ec2a admin admin] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:32:57.991959 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0ff0e79-3cbb-4e3b-b650-8e4775eabecf service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20ca3c3bd3774691b3374cfbfec6e494', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4faef13aa50e4e36b6c265fb1398fa14', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:32:57.992789 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0ff0e79-3cbb-4e3b-b650-8e4775eabecf service neutron] REQUEST_METHOD: `GET` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:32:57.993221 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0ff0e79-3cbb-4e3b-b650-8e4775eabecf service neutron] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:32:57.994821 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0ff0e79-3cbb-4e3b-b650-8e4775eabecf service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:32:58.008263 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0ff0e79-3cbb-4e3b-b650-8e4775eabecf service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:32:58.010966 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0ff0e79-3cbb-4e3b-b650-8e4775eabecf service neutron] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:32:58.028300 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 1256/2520] 10.222.0.47 () {60 vars in 1365 bytes} [Thu Jul 21 08:32:57 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 176 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 08:32:59.023241 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ee98e5d-72be-410f-8f9e-ed430825f374 None None] REQUEST_METHOD: `POST` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:32:59.023771 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ee98e5d-72be-410f-8f9e-ed430825f374 None None] SCRIPT_NAME: `/identity` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:32:59.023852 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ee98e5d-72be-410f-8f9e-ed430825f374 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:32:59.070914 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.auth.core [None req-2ee98e5d-72be-410f-8f9e-ed430825f374 None None] MFA Rules not processed for user `4512e5eb630943a996a43ea931656696`. Rule list: `[]` (Enabled: `True`). {{(pid=80245) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 08:32:59.073247 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.fernet_utils [None req-2ee98e5d-72be-410f-8f9e-ed430825f374 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80245) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 08:32:59.203435 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: [pid: 80245|app: 0|req: 1265/2521] 10.222.0.47 () {58 vars in 957 bytes} [Thu Jul 21 08:32:59 2022] POST /identity/v3/auth/tokens => generated 3307 bytes in 181 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 08:32:59.218206 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0ff0e79-3cbb-4e3b-b650-8e4775eabecf service neutron] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:32:59.221087 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.fernet_utils [None req-c0ff0e79-3cbb-4e3b-b650-8e4775eabecf service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80244) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 08:32:59.343420 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0ff0e79-3cbb-4e3b-b650-8e4775eabecf service neutron] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:32:59.350662 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6496769d-a4ca-4375-a03f-afcbd302d9a4 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2b5b99a36442148fe743269054e991', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4faef13aa50e4e36b6c265fb1398fa14', 'project_domain_id': 'default', 'roles': ['member', 'service', 'ResellerAdmin', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:32:59.352371 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6496769d-a4ca-4375-a03f-afcbd302d9a4 service nova] REQUEST_METHOD: `GET` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:32:59.352865 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6496769d-a4ca-4375-a03f-afcbd302d9a4 service nova] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:32:59.354310 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6496769d-a4ca-4375-a03f-afcbd302d9a4 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:32:59.364732 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.fernet_utils [None req-6496769d-a4ca-4375-a03f-afcbd302d9a4 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80244) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 08:32:59.465287 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6496769d-a4ca-4375-a03f-afcbd302d9a4 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:32:59.475122 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6496769d-a4ca-4375-a03f-afcbd302d9a4 service nova] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:32:59.490079 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 1257/2522] 10.222.0.47 () {60 vars in 1365 bytes} [Thu Jul 21 08:32:59 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 272 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 08:33:00.135569 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-930ca6ba-2cb2-480a-a62f-497ce644c080 None None] REQUEST_METHOD: `POST` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:33:00.136801 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-930ca6ba-2cb2-480a-a62f-497ce644c080 None None] SCRIPT_NAME: `/identity` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:33:00.137448 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-930ca6ba-2cb2-480a-a62f-497ce644c080 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:33:00.159757 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.auth.core [None req-930ca6ba-2cb2-480a-a62f-497ce644c080 None None] MFA Rules not processed for user `3528104f3e914d62a10e80e13d3c97de`. Rule list: `[]` (Enabled: `True`). {{(pid=80245) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 08:33:00.169376 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.fernet_utils [None req-930ca6ba-2cb2-480a-a62f-497ce644c080 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80245) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 08:33:00.355844 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: [pid: 80245|app: 0|req: 1266/2523] 10.222.0.47 () {58 vars in 957 bytes} [Thu Jul 21 08:33:00 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 223 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 08:33:01.332311 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d8d4bb3-dce1-470a-8d48-bf37108d2204 None None] REQUEST_METHOD: `POST` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:33:01.333257 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d8d4bb3-dce1-470a-8d48-bf37108d2204 None None] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:33:01.333759 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d8d4bb3-dce1-470a-8d48-bf37108d2204 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:33:01.347433 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG dbcounter [-] [80244] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80244) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:33:01.361788 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.auth.core [None req-6d8d4bb3-dce1-470a-8d48-bf37108d2204 None None] MFA Rules not processed for user `45688a2eafb840e9b2b4c56bbee69375`. Rule list: `[]` (Enabled: `True`). {{(pid=80244) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 08:33:01.363422 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.fernet_utils [None req-6d8d4bb3-dce1-470a-8d48-bf37108d2204 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80244) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 08:33:01.375655 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 1258/2524] 10.222.0.47 () {58 vars in 957 bytes} [Thu Jul 21 08:33:01 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 44 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 08:33:04.380678 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d98127b8-f96f-48e0-b18d-64e84c786596 None None] REQUEST_METHOD: `POST` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:33:04.381770 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d98127b8-f96f-48e0-b18d-64e84c786596 None None] SCRIPT_NAME: `/identity` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:33:04.383580 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d98127b8-f96f-48e0-b18d-64e84c786596 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:33:04.410628 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.auth.core [None req-d98127b8-f96f-48e0-b18d-64e84c786596 None None] MFA Rules not processed for user `2ef7f1e91e554a1f88eb6a63e60a475f`. Rule list: `[]` (Enabled: `True`). {{(pid=80245) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 08:33:04.412204 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.fernet_utils [None req-d98127b8-f96f-48e0-b18d-64e84c786596 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80245) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 08:33:04.523772 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: [pid: 80245|app: 0|req: 1267/2525] 10.222.0.47 () {58 vars in 1009 bytes} [Thu Jul 21 08:33:04 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 145 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 08:33:04.570487 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d8d4bb3-dce1-470a-8d48-bf37108d2204 None None] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:33:04.573240 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.fernet_utils [None req-6d8d4bb3-dce1-470a-8d48-bf37108d2204 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80244) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 08:33:04.709092 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d8d4bb3-dce1-470a-8d48-bf37108d2204 None None] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:33:04.727738 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d05775a-8496-4f5c-a5ee-2de0c3b5fb32 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '302351411fc84248a6b166b25d21609f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4faef13aa50e4e36b6c265fb1398fa14', 'project_domain_id': 'default', 'roles': ['service', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:33:04.731531 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d05775a-8496-4f5c-a5ee-2de0c3b5fb32 service swift] REQUEST_METHOD: `GET` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:33:04.736638 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d05775a-8496-4f5c-a5ee-2de0c3b5fb32 service swift] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:33:04.737308 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d05775a-8496-4f5c-a5ee-2de0c3b5fb32 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:33:04.751623 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d05775a-8496-4f5c-a5ee-2de0c3b5fb32 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:33:04.754044 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d05775a-8496-4f5c-a5ee-2de0c3b5fb32 service swift] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:33:04.787467 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 1259/2526] 10.222.0.47 () {60 vars in 1384 bytes} [Thu Jul 21 08:33:04 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 218 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 21 08:33:05.832474 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4fb6d18-bd07-448a-a25b-91d39b4d314b None None] REQUEST_METHOD: `POST` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:33:05.833011 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4fb6d18-bd07-448a-a25b-91d39b4d314b None None] SCRIPT_NAME: `/identity` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:33:05.833127 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4fb6d18-bd07-448a-a25b-91d39b4d314b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:33:05.876299 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG dbcounter [-] [80245] Writing DB stats keystone:SELECT=100 {{(pid=80245) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:33:05.887108 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.auth.core [None req-f4fb6d18-bd07-448a-a25b-91d39b4d314b None None] MFA Rules not processed for user `8249ebd6b2dc4811ad315d54030d927e`. Rule list: `[]` (Enabled: `True`). {{(pid=80245) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 08:33:05.888522 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.fernet_utils [None req-f4fb6d18-bd07-448a-a25b-91d39b4d314b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80245) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 08:33:06.001288 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d05775a-8496-4f5c-a5ee-2de0c3b5fb32 service swift] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:33:06.002427 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: [pid: 80245|app: 0|req: 1268/2527] 10.222.0.47 () {58 vars in 957 bytes} [Thu Jul 21 08:33:05 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 08:33:06.021494 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d05775a-8496-4f5c-a5ee-2de0c3b5fb32 service swift] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:33:06.030566 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62883230-58e8-45f7-b48f-d499f50146d5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '42d15fc0515945acacbbffcf2356c7a5', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:33:06.031969 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62883230-58e8-45f7-b48f-d499f50146d5 admin admin] REQUEST_METHOD: `POST` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:33:06.032220 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62883230-58e8-45f7-b48f-d499f50146d5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:33:06.032691 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62883230-58e8-45f7-b48f-d499f50146d5 admin admin] PATH_INFO: `/v3/projects` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:33:06.033799 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62883230-58e8-45f7-b48f-d499f50146d5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:33:06.036331 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62883230-58e8-45f7-b48f-d499f50146d5 admin admin] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:33:06.070971 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 1260/2528] 10.222.0.47 () {60 vars in 1152 bytes} [Thu Jul 21 08:33:05 2022] POST /identity/v3/projects => generated 385 bytes in 72 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 08:33:06.075696 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4fb6d18-bd07-448a-a25b-91d39b4d314b None None] Authenticating user token {{(pid=80245) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:33:06.087778 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4fb6d18-bd07-448a-a25b-91d39b4d314b None None] Validating token access rules against request {{(pid=80245) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:33:06.161176 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fd175c9-cbea-4db6-bca2-f3da8c12b4ae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '42d15fc0515945acacbbffcf2356c7a5', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80245) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:33:06.162104 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fd175c9-cbea-4db6-bca2-f3da8c12b4ae admin admin] REQUEST_METHOD: `POST` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:33:06.162402 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fd175c9-cbea-4db6-bca2-f3da8c12b4ae admin admin] SCRIPT_NAME: `/identity` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:33:06.162701 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fd175c9-cbea-4db6-bca2-f3da8c12b4ae admin admin] PATH_INFO: `/v3/users` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:33:06.163517 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fd175c9-cbea-4db6-bca2-f3da8c12b4ae admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:33:06.165923 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fd175c9-cbea-4db6-bca2-f3da8c12b4ae admin admin] RBAC: Authorization granted {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:33:06.206935 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62883230-58e8-45f7-b48f-d499f50146d5 admin admin] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:33:06.212704 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: [pid: 80245|app: 0|req: 1269/2529] 10.222.0.47 () {60 vars in 1143 bytes} [Thu Jul 21 08:33:06 2022] POST /identity/v3/users => generated 299 bytes in 137 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 21 08:33:06.243447 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62883230-58e8-45f7-b48f-d499f50146d5 admin admin] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:33:06.273844 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f306f0ff-64a1-4525-9503-ebc31aec827b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '42d15fc0515945acacbbffcf2356c7a5', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:33:06.275177 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f306f0ff-64a1-4525-9503-ebc31aec827b admin admin] REQUEST_METHOD: `GET` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:33:06.276345 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f306f0ff-64a1-4525-9503-ebc31aec827b admin admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:33:06.276881 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f306f0ff-64a1-4525-9503-ebc31aec827b admin admin] PATH_INFO: `/v3/roles` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:33:06.278119 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f306f0ff-64a1-4525-9503-ebc31aec827b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:33:06.280837 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f306f0ff-64a1-4525-9503-ebc31aec827b admin admin] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:33:06.297500 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 1261/2530] 10.222.0.47 () {58 vars in 1121 bytes} [Thu Jul 21 08:33:06 2022] GET /identity/v3/roles => generated 1420 bytes in 91 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 08:33:06.307751 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fd175c9-cbea-4db6-bca2-f3da8c12b4ae admin admin] Authenticating user token {{(pid=80245) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:33:06.336238 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fd175c9-cbea-4db6-bca2-f3da8c12b4ae admin admin] Validating token access rules against request {{(pid=80245) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:33:06.349346 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60b8caaa-9efe-461a-90c2-a752828a5e9c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '42d15fc0515945acacbbffcf2356c7a5', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80245) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:33:06.350581 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60b8caaa-9efe-461a-90c2-a752828a5e9c admin admin] REQUEST_METHOD: `PUT` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:33:06.350936 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60b8caaa-9efe-461a-90c2-a752828a5e9c admin admin] SCRIPT_NAME: `/identity` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:33:06.351288 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60b8caaa-9efe-461a-90c2-a752828a5e9c admin admin] PATH_INFO: `/v3/projects/da18c71220bf49458a8826a45a82d2e1/users/edcf4542e2cf43bfb62a41c82b3a3384/roles/e29870f6e9e2411a9db0d5bf45f32ca5` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:33:06.373202 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60b8caaa-9efe-461a-90c2-a752828a5e9c admin admin] RBAC: Authorizing `identity:create_grant(project_id=da18c71220bf49458a8826a45a82d2e1, user_id=edcf4542e2cf43bfb62a41c82b3a3384, role_id=e29870f6e9e2411a9db0d5bf45f32ca5)` {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:33:06.376235 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60b8caaa-9efe-461a-90c2-a752828a5e9c admin admin] RBAC: Authorization granted {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:33:06.426705 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: [pid: 80245|app: 0|req: 1270/2531] 10.222.0.47 () {60 vars in 1482 bytes} [Thu Jul 21 08:33:06 2022] PUT /identity/v3/projects/da18c71220bf49458a8826a45a82d2e1/users/edcf4542e2cf43bfb62a41c82b3a3384/roles/e29870f6e9e2411a9db0d5bf45f32ca5 => generated 0 bytes in 119 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 08:33:06.433121 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f306f0ff-64a1-4525-9503-ebc31aec827b admin admin] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:33:06.446416 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f306f0ff-64a1-4525-9503-ebc31aec827b admin admin] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:33:06.561251 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1df212f-8678-4fad-882a-e3a6b39031d1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '42d15fc0515945acacbbffcf2356c7a5', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:33:06.561940 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1df212f-8678-4fad-882a-e3a6b39031d1 admin admin] REQUEST_METHOD: `GET` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:33:06.562229 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1df212f-8678-4fad-882a-e3a6b39031d1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:33:06.562506 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1df212f-8678-4fad-882a-e3a6b39031d1 admin admin] PATH_INFO: `/v3/roles` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:33:06.563299 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1df212f-8678-4fad-882a-e3a6b39031d1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:33:06.565021 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1df212f-8678-4fad-882a-e3a6b39031d1 admin admin] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:33:06.582227 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 1262/2532] 10.222.0.47 () {58 vars in 1121 bytes} [Thu Jul 21 08:33:06 2022] GET /identity/v3/roles => generated 1420 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 08:33:06.586357 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60b8caaa-9efe-461a-90c2-a752828a5e9c admin admin] Authenticating user token {{(pid=80245) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:33:06.601351 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60b8caaa-9efe-461a-90c2-a752828a5e9c admin admin] Validating token access rules against request {{(pid=80245) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:33:06.608606 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff592948-5b9a-4d41-86a4-8679b7f1d843 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '42d15fc0515945acacbbffcf2356c7a5', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80245) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:33:06.609709 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff592948-5b9a-4d41-86a4-8679b7f1d843 admin admin] REQUEST_METHOD: `PUT` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:33:06.610312 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff592948-5b9a-4d41-86a4-8679b7f1d843 admin admin] SCRIPT_NAME: `/identity` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:33:06.610426 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff592948-5b9a-4d41-86a4-8679b7f1d843 admin admin] PATH_INFO: `/v3/projects/da18c71220bf49458a8826a45a82d2e1/users/edcf4542e2cf43bfb62a41c82b3a3384/roles/e29870f6e9e2411a9db0d5bf45f32ca5` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:33:06.612566 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff592948-5b9a-4d41-86a4-8679b7f1d843 admin admin] RBAC: Authorizing `identity:create_grant(project_id=da18c71220bf49458a8826a45a82d2e1, user_id=edcf4542e2cf43bfb62a41c82b3a3384, role_id=e29870f6e9e2411a9db0d5bf45f32ca5)` {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:33:06.615984 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff592948-5b9a-4d41-86a4-8679b7f1d843 admin admin] RBAC: Authorization granted {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:33:06.665296 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: [pid: 80245|app: 0|req: 1271/2533] 10.222.0.47 () {60 vars in 1482 bytes} [Thu Jul 21 08:33:06 2022] PUT /identity/v3/projects/da18c71220bf49458a8826a45a82d2e1/users/edcf4542e2cf43bfb62a41c82b3a3384/roles/e29870f6e9e2411a9db0d5bf45f32ca5 => generated 0 bytes in 79 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 08:33:09.267991 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a23d0e69-0a26-4e10-b7b3-f3520c2844e6 None None] REQUEST_METHOD: `POST` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:33:09.268583 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a23d0e69-0a26-4e10-b7b3-f3520c2844e6 None None] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:33:09.269041 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a23d0e69-0a26-4e10-b7b3-f3520c2844e6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:33:09.321260 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.auth.core [None req-a23d0e69-0a26-4e10-b7b3-f3520c2844e6 None None] MFA Rules not processed for user `1f914243baf846f5b43a929332bcef6e`. Rule list: `[]` (Enabled: `True`). {{(pid=80244) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 08:33:09.325127 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.fernet_utils [None req-a23d0e69-0a26-4e10-b7b3-f3520c2844e6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80244) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 08:33:09.500220 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG dbcounter [-] [80244] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80244) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:33:09.541915 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 1263/2534] 10.222.0.47 () {58 vars in 957 bytes} [Thu Jul 21 08:33:09 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 274 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 08:33:14.965097 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b542ffb-58a9-44cc-93d9-48927d62c774 None None] REQUEST_METHOD: `POST` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:33:14.965695 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b542ffb-58a9-44cc-93d9-48927d62c774 None None] SCRIPT_NAME: `/identity` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:33:14.965917 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b542ffb-58a9-44cc-93d9-48927d62c774 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:33:15.003413 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.auth.core [None req-3b542ffb-58a9-44cc-93d9-48927d62c774 None None] MFA Rules not processed for user `edcf4542e2cf43bfb62a41c82b3a3384`. Rule list: `[]` (Enabled: `True`). {{(pid=80245) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 08:33:15.005435 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.fernet_utils [None req-3b542ffb-58a9-44cc-93d9-48927d62c774 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80245) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 08:33:15.112604 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG dbcounter [-] [80245] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80245) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:33:15.194859 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: [pid: 80245|app: 0|req: 1272/2535] 10.222.0.47 () {58 vars in 957 bytes} [Thu Jul 21 08:33:14 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 227 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 08:33:17.456266 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a23d0e69-0a26-4e10-b7b3-f3520c2844e6 None None] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:33:17.458517 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.fernet_utils [None req-a23d0e69-0a26-4e10-b7b3-f3520c2844e6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80244) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 08:33:17.658211 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a23d0e69-0a26-4e10-b7b3-f3520c2844e6 None None] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:33:17.669214 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-852d67d1-1f17-4ff1-aedc-92a1b2da70a5 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3379b32543474d0a8e897484e85c7d04', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4faef13aa50e4e36b6c265fb1398fa14', 'project_domain_id': 'default', 'roles': ['service', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:33:17.670090 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-852d67d1-1f17-4ff1-aedc-92a1b2da70a5 service placement] REQUEST_METHOD: `GET` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:33:17.670380 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-852d67d1-1f17-4ff1-aedc-92a1b2da70a5 service placement] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:33:17.670843 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-852d67d1-1f17-4ff1-aedc-92a1b2da70a5 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:33:17.673320 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.fernet_utils [None req-852d67d1-1f17-4ff1-aedc-92a1b2da70a5 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80244) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 08:33:17.689698 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-852d67d1-1f17-4ff1-aedc-92a1b2da70a5 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:33:17.691717 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-852d67d1-1f17-4ff1-aedc-92a1b2da70a5 service placement] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:33:17.712090 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 1264/2536] 10.222.0.47 () {60 vars in 1365 bytes} [Thu Jul 21 08:33:17 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 256 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 08:33:18.045729 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b542ffb-58a9-44cc-93d9-48927d62c774 None None] Authenticating user token {{(pid=80245) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:33:18.047826 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.fernet_utils [None req-3b542ffb-58a9-44cc-93d9-48927d62c774 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80245) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 08:33:18.152416 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b542ffb-58a9-44cc-93d9-48927d62c774 None None] Validating token access rules against request {{(pid=80245) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:33:18.156308 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1427cdea-525d-4cd6-9c72-cd63f506e92d service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9b0524aed8c74abd8949b860d6b93936', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4faef13aa50e4e36b6c265fb1398fa14', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80245) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:33:18.156978 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1427cdea-525d-4cd6-9c72-cd63f506e92d service glance] REQUEST_METHOD: `GET` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:33:18.157256 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1427cdea-525d-4cd6-9c72-cd63f506e92d service glance] SCRIPT_NAME: `/identity` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:33:18.157533 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1427cdea-525d-4cd6-9c72-cd63f506e92d service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:33:18.256874 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1427cdea-525d-4cd6-9c72-cd63f506e92d service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:33:18.263221 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1427cdea-525d-4cd6-9c72-cd63f506e92d service glance] RBAC: Authorization granted {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:33:18.286633 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: [pid: 80245|app: 0|req: 1273/2537] 10.222.0.47 () {60 vars in 1365 bytes} [Thu Jul 21 08:33:18 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 241 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 08:33:18.646231 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7447d30-b0c6-4536-a02b-21b8961ecefa None None] REQUEST_METHOD: `POST` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:33:18.646767 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7447d30-b0c6-4536-a02b-21b8961ecefa None None] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:33:18.646767 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7447d30-b0c6-4536-a02b-21b8961ecefa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:33:18.685559 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.auth.core [None req-d7447d30-b0c6-4536-a02b-21b8961ecefa None None] MFA Rules not processed for user `2ef7f1e91e554a1f88eb6a63e60a475f`. Rule list: `[]` (Enabled: `True`). {{(pid=80244) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 08:33:18.686827 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.fernet_utils [None req-d7447d30-b0c6-4536-a02b-21b8961ecefa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80244) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 08:33:18.854124 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 1265/2538] 10.222.0.47 () {58 vars in 1009 bytes} [Thu Jul 21 08:33:18 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 209 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 08:33:18.886085 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1427cdea-525d-4cd6-9c72-cd63f506e92d service glance] Authenticating user token {{(pid=80245) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:33:19.097886 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1427cdea-525d-4cd6-9c72-cd63f506e92d service glance] Validating token access rules against request {{(pid=80245) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:33:19.112403 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7aa1d8d-3247-434b-b84f-3ee5e56a8c03 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '302351411fc84248a6b166b25d21609f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4faef13aa50e4e36b6c265fb1398fa14', 'project_domain_id': 'default', 'roles': ['service', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80245) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:33:19.113142 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7aa1d8d-3247-434b-b84f-3ee5e56a8c03 service swift] REQUEST_METHOD: `GET` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:33:19.113432 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7aa1d8d-3247-434b-b84f-3ee5e56a8c03 service swift] SCRIPT_NAME: `/identity` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:33:19.113713 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7aa1d8d-3247-434b-b84f-3ee5e56a8c03 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:33:19.126143 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7aa1d8d-3247-434b-b84f-3ee5e56a8c03 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:33:19.128172 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7aa1d8d-3247-434b-b84f-3ee5e56a8c03 service swift] RBAC: Authorization granted {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:33:19.174169 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: [pid: 80245|app: 0|req: 1274/2539] 10.222.0.47 () {60 vars in 1384 bytes} [Thu Jul 21 08:33:18 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 283 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 21 08:33:28.836033 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG dbcounter [-] [80244] Writing DB stats keystone:SELECT=58 {{(pid=80244) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:33:29.141981 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG dbcounter [-] [80245] Writing DB stats keystone:SELECT=90 {{(pid=80245) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:33:32.347423 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b32da25-bb3d-4d31-bb95-411c691f8e7e None None] REQUEST_METHOD: `POST` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:33:32.348379 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b32da25-bb3d-4d31-bb95-411c691f8e7e None None] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:33:32.348815 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b32da25-bb3d-4d31-bb95-411c691f8e7e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:33:32.371474 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.auth.core [None req-4b32da25-bb3d-4d31-bb95-411c691f8e7e None None] MFA Rules not processed for user `8249ebd6b2dc4811ad315d54030d927e`. Rule list: `[]` (Enabled: `True`). {{(pid=80244) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 08:33:32.372969 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.fernet_utils [None req-4b32da25-bb3d-4d31-bb95-411c691f8e7e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80244) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 08:33:32.383483 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 1266/2540] 10.222.0.47 () {58 vars in 957 bytes} [Thu Jul 21 08:33:32 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 08:33:32.552717 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d210472-c861-44b0-aa32-dcd4b60f9910 None None] REQUEST_METHOD: `POST` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:33:32.553308 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d210472-c861-44b0-aa32-dcd4b60f9910 None None] SCRIPT_NAME: `/identity` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:33:32.553308 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d210472-c861-44b0-aa32-dcd4b60f9910 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:33:32.573490 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.auth.core [None req-2d210472-c861-44b0-aa32-dcd4b60f9910 None None] MFA Rules not processed for user `2ef7f1e91e554a1f88eb6a63e60a475f`. Rule list: `[]` (Enabled: `True`). {{(pid=80245) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 08:33:32.574809 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.fernet_utils [None req-2d210472-c861-44b0-aa32-dcd4b60f9910 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80245) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 08:33:32.589821 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: [pid: 80245|app: 0|req: 1275/2541] 10.222.0.47 () {58 vars in 1009 bytes} [Thu Jul 21 08:33:32 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 08:33:32.597748 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b32da25-bb3d-4d31-bb95-411c691f8e7e None None] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:33:32.611040 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b32da25-bb3d-4d31-bb95-411c691f8e7e None None] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:33:32.616388 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdb0d9eb-12cf-4915-8b62-2253554ae355 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '302351411fc84248a6b166b25d21609f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4faef13aa50e4e36b6c265fb1398fa14', 'project_domain_id': 'default', 'roles': ['service', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:33:32.617047 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdb0d9eb-12cf-4915-8b62-2253554ae355 service swift] REQUEST_METHOD: `GET` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:33:32.617328 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdb0d9eb-12cf-4915-8b62-2253554ae355 service swift] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:33:32.617598 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdb0d9eb-12cf-4915-8b62-2253554ae355 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:33:32.627827 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdb0d9eb-12cf-4915-8b62-2253554ae355 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:33:32.629700 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdb0d9eb-12cf-4915-8b62-2253554ae355 service swift] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:33:32.639024 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 1267/2542] 10.222.0.47 () {60 vars in 1384 bytes} [Thu Jul 21 08:33:32 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 21 08:33:33.504552 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46731c76-2711-4f19-9af9-9a674994c410 None None] REQUEST_METHOD: `POST` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:33:33.505058 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46731c76-2711-4f19-9af9-9a674994c410 None None] SCRIPT_NAME: `/identity` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:33:33.505340 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46731c76-2711-4f19-9af9-9a674994c410 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:33:33.526077 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.auth.core [None req-46731c76-2711-4f19-9af9-9a674994c410 None None] MFA Rules not processed for user `2ef7f1e91e554a1f88eb6a63e60a475f`. Rule list: `[]` (Enabled: `True`). {{(pid=80245) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 08:33:33.527501 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.fernet_utils [None req-46731c76-2711-4f19-9af9-9a674994c410 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80245) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 08:33:33.542667 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: [pid: 80245|app: 0|req: 1276/2543] 10.222.0.47 () {58 vars in 967 bytes} [Thu Jul 21 08:33:33 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 08:33:33.566092 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdb0d9eb-12cf-4915-8b62-2253554ae355 service swift] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:33:33.582508 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdb0d9eb-12cf-4915-8b62-2253554ae355 service swift] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:33:33.588902 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3111f3e8-0678-40d9-b402-6cb4887e670c service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '302351411fc84248a6b166b25d21609f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4faef13aa50e4e36b6c265fb1398fa14', 'project_domain_id': 'default', 'roles': ['service', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:33:33.589948 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3111f3e8-0678-40d9-b402-6cb4887e670c service swift] REQUEST_METHOD: `GET` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:33:33.590300 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3111f3e8-0678-40d9-b402-6cb4887e670c service swift] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:33:33.590685 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3111f3e8-0678-40d9-b402-6cb4887e670c service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:33:33.601571 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3111f3e8-0678-40d9-b402-6cb4887e670c service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:33:33.603249 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3111f3e8-0678-40d9-b402-6cb4887e670c service swift] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:33:33.613215 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 1268/2544] 10.222.0.47 () {60 vars in 1384 bytes} [Thu Jul 21 08:33:33 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 48 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 21 08:33:34.773536 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-701b2707-8bec-47e5-8ab7-0a27ac4f2e13 None None] REQUEST_METHOD: `POST` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:33:34.774565 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-701b2707-8bec-47e5-8ab7-0a27ac4f2e13 None None] SCRIPT_NAME: `/identity` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:33:34.774999 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-701b2707-8bec-47e5-8ab7-0a27ac4f2e13 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:33:34.809349 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.auth.core [None req-701b2707-8bec-47e5-8ab7-0a27ac4f2e13 None None] MFA Rules not processed for user `2ef7f1e91e554a1f88eb6a63e60a475f`. Rule list: `[]` (Enabled: `True`). {{(pid=80245) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 08:33:34.811909 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.fernet_utils [None req-701b2707-8bec-47e5-8ab7-0a27ac4f2e13 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80245) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 08:33:34.826073 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: [pid: 80245|app: 0|req: 1277/2545] 10.222.0.47 () {58 vars in 1009 bytes} [Thu Jul 21 08:33:34 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 53 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 21 08:33:34.842372 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3111f3e8-0678-40d9-b402-6cb4887e670c service swift] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:33:34.858107 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3111f3e8-0678-40d9-b402-6cb4887e670c service swift] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:33:34.864003 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96155758-3bdc-4acf-bf40-d1dadf8827f9 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '302351411fc84248a6b166b25d21609f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4faef13aa50e4e36b6c265fb1398fa14', 'project_domain_id': 'default', 'roles': ['service', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:33:34.864895 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96155758-3bdc-4acf-bf40-d1dadf8827f9 service swift] REQUEST_METHOD: `GET` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:33:34.865179 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96155758-3bdc-4acf-bf40-d1dadf8827f9 service swift] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:33:34.865456 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96155758-3bdc-4acf-bf40-d1dadf8827f9 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:33:34.886761 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96155758-3bdc-4acf-bf40-d1dadf8827f9 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:33:34.888426 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96155758-3bdc-4acf-bf40-d1dadf8827f9 service swift] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:33:34.898330 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 1269/2546] 10.222.0.47 () {60 vars in 1384 bytes} [Thu Jul 21 08:33:34 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 57 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 21 08:33:37.134237 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca2321d9-8fe0-49ae-8f29-12ab2b4bd95e None None] REQUEST_METHOD: `GET` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:33:37.135264 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca2321d9-8fe0-49ae-8f29-12ab2b4bd95e None None] SCRIPT_NAME: `/identity` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:33:37.135717 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca2321d9-8fe0-49ae-8f29-12ab2b4bd95e None None] PATH_INFO: `/` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:33:37.137363 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: [pid: 80245|app: 0|req: 1278/2547] 10.222.0.47 () {54 vars in 921 bytes} [Thu Jul 21 08:33:37 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 08:33:37.223861 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f82a4f40-6df8-4811-82f8-78d10d5c1b02 None None] REQUEST_METHOD: `POST` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:33:37.224452 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f82a4f40-6df8-4811-82f8-78d10d5c1b02 None None] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:33:37.224452 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f82a4f40-6df8-4811-82f8-78d10d5c1b02 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:33:37.268300 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.auth.core [None req-f82a4f40-6df8-4811-82f8-78d10d5c1b02 None None] MFA Rules not processed for user `3379b32543474d0a8e897484e85c7d04`. Rule list: `[]` (Enabled: `True`). {{(pid=80244) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 08:33:37.274764 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.fernet_utils [None req-f82a4f40-6df8-4811-82f8-78d10d5c1b02 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80244) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 08:33:37.301036 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 1270/2548] 10.222.0.47 () {58 vars in 1018 bytes} [Thu Jul 21 08:33:37 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 08:33:37.330532 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca2321d9-8fe0-49ae-8f29-12ab2b4bd95e None None] Authenticating user token {{(pid=80245) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:33:37.332705 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.fernet_utils [None req-ca2321d9-8fe0-49ae-8f29-12ab2b4bd95e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80245) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 08:33:37.337661 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f82a4f40-6df8-4811-82f8-78d10d5c1b02 None None] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:33:37.340233 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.fernet_utils [None req-f82a4f40-6df8-4811-82f8-78d10d5c1b02 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80244) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 08:33:37.368420 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca2321d9-8fe0-49ae-8f29-12ab2b4bd95e None None] Validating token access rules against request {{(pid=80245) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:33:37.376674 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af326676-9891-4ca9-bdc6-b55a1c0f8d0e service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3379b32543474d0a8e897484e85c7d04', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4faef13aa50e4e36b6c265fb1398fa14', 'project_domain_id': 'default', 'roles': ['service', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80245) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:33:37.377354 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af326676-9891-4ca9-bdc6-b55a1c0f8d0e service placement] REQUEST_METHOD: `GET` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:33:37.377640 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af326676-9891-4ca9-bdc6-b55a1c0f8d0e service placement] SCRIPT_NAME: `/identity` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:33:37.377961 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af326676-9891-4ca9-bdc6-b55a1c0f8d0e service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:33:37.403395 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af326676-9891-4ca9-bdc6-b55a1c0f8d0e service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:33:37.405452 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af326676-9891-4ca9-bdc6-b55a1c0f8d0e service placement] RBAC: Authorization granted {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:33:37.431704 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: [pid: 80245|app: 0|req: 1279/2549] 10.222.0.47 () {60 vars in 1365 bytes} [Thu Jul 21 08:33:37 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 102 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 08:33:37.606992 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f82a4f40-6df8-4811-82f8-78d10d5c1b02 None None] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:33:37.638478 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-858536f5-4f34-46dd-a0cf-ed5f872f4053 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '42d15fc0515945acacbbffcf2356c7a5', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:33:37.644843 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-858536f5-4f34-46dd-a0cf-ed5f872f4053 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:33:37.645390 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-858536f5-4f34-46dd-a0cf-ed5f872f4053 admin admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:33:37.646516 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-858536f5-4f34-46dd-a0cf-ed5f872f4053 admin admin] PATH_INFO: `/v3/users/aa156584880d4449bd89a7bcdcfe403f` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:33:37.653000 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-858536f5-4f34-46dd-a0cf-ed5f872f4053 admin admin] RBAC: Authorizing `identity:delete_user(user_id=aa156584880d4449bd89a7bcdcfe403f)` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:33:37.665733 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-858536f5-4f34-46dd-a0cf-ed5f872f4053 admin admin] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:33:37.824908 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.notifications [None req-858536f5-4f34-46dd-a0cf-ed5f872f4053 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'aa156584880d4449bd89a7bcdcfe403f'} {{(pid=80244) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 08:33:37.842484 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.notifications [None req-858536f5-4f34-46dd-a0cf-ed5f872f4053 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'aa156584880d4449bd89a7bcdcfe403f'} {{(pid=80244) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 08:33:37.900382 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.notifications [None req-858536f5-4f34-46dd-a0cf-ed5f872f4053 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'aa156584880d4449bd89a7bcdcfe403f'} {{(pid=80244) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 08:33:37.901684 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.notifications [None req-858536f5-4f34-46dd-a0cf-ed5f872f4053 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'aa156584880d4449bd89a7bcdcfe403f'} {{(pid=80244) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 08:33:37.927932 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.notifications [None req-858536f5-4f34-46dd-a0cf-ed5f872f4053 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'aa156584880d4449bd89a7bcdcfe403f'} {{(pid=80244) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 08:33:37.942489 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 1271/2550] 10.222.0.47 () {58 vars in 1223 bytes} [Thu Jul 21 08:33:37 2022] DELETE /identity/v3/users/aa156584880d4449bd89a7bcdcfe403f => generated 0 bytes in 604 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 08:33:38.736883 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af326676-9891-4ca9-bdc6-b55a1c0f8d0e service placement] Authenticating user token {{(pid=80245) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:33:38.751437 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af326676-9891-4ca9-bdc6-b55a1c0f8d0e service placement] Validating token access rules against request {{(pid=80245) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:33:38.988081 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb5132e2-74c9-47c9-b8c4-1d61af22b592 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '42d15fc0515945acacbbffcf2356c7a5', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80245) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:33:38.988982 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb5132e2-74c9-47c9-b8c4-1d61af22b592 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:33:38.989260 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb5132e2-74c9-47c9-b8c4-1d61af22b592 admin admin] SCRIPT_NAME: `/identity` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:33:38.989534 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb5132e2-74c9-47c9-b8c4-1d61af22b592 admin admin] PATH_INFO: `/v3/projects/6c163cbb79544869bb823308163a4142` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:33:39.004091 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb5132e2-74c9-47c9-b8c4-1d61af22b592 admin admin] RBAC: Authorizing `identity:delete_project(project_id=6c163cbb79544869bb823308163a4142)` {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:33:39.026750 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb5132e2-74c9-47c9-b8c4-1d61af22b592 admin admin] RBAC: Authorization granted {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:33:39.297078 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.notifications [None req-cb5132e2-74c9-47c9-b8c4-1d61af22b592 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6c163cbb79544869bb823308163a4142', 'request_id': 'req-cb5132e2-74c9-47c9-b8c4-1d61af22b592'} {{(pid=80245) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 08:33:39.317473 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.notifications [None req-cb5132e2-74c9-47c9-b8c4-1d61af22b592 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6c163cbb79544869bb823308163a4142', 'request_id': 'req-cb5132e2-74c9-47c9-b8c4-1d61af22b592'} {{(pid=80245) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 08:33:39.335306 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.notifications [None req-cb5132e2-74c9-47c9-b8c4-1d61af22b592 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6c163cbb79544869bb823308163a4142', 'request_id': 'req-cb5132e2-74c9-47c9-b8c4-1d61af22b592'} {{(pid=80245) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 08:33:39.364693 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.notifications [None req-cb5132e2-74c9-47c9-b8c4-1d61af22b592 admin admin] The token cache is being invalidate because project 6c163cbb79544869bb823308163a4142 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80245) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 21 08:33:39.365321 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.notifications [None req-cb5132e2-74c9-47c9-b8c4-1d61af22b592 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80245) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 08:33:39.368040 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.notifications [None req-cb5132e2-74c9-47c9-b8c4-1d61af22b592 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80245) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 08:33:39.373322 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: [pid: 80245|app: 0|req: 1280/2551] 10.222.0.47 () {58 vars in 1232 bytes} [Thu Jul 21 08:33:38 2022] DELETE /identity/v3/projects/6c163cbb79544869bb823308163a4142 => generated 0 bytes in 637 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 08:33:39.378355 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a94255fd-9685-4b0e-891f-d59e83bce0fd None None] REQUEST_METHOD: `POST` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:33:39.378355 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a94255fd-9685-4b0e-891f-d59e83bce0fd None None] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:33:39.379467 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a94255fd-9685-4b0e-891f-d59e83bce0fd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:33:39.436343 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.auth.core [None req-a94255fd-9685-4b0e-891f-d59e83bce0fd None None] MFA Rules not processed for user `2dce40b8d74c41ec94cc36e6ff05d412`. Rule list: `[]` (Enabled: `True`). {{(pid=80244) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 08:33:39.440827 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.fernet_utils [None req-a94255fd-9685-4b0e-891f-d59e83bce0fd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80244) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 08:33:39.537938 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG dbcounter [-] [80244] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80244) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:33:39.641680 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 1272/2552] 10.222.0.47 () {58 vars in 957 bytes} [Thu Jul 21 08:33:39 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 265 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 21 08:33:39.738297 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-753bc9ff-dc8c-4e48-8f57-73b9b101fbb0 None None] REQUEST_METHOD: `POST` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:33:39.738837 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-753bc9ff-dc8c-4e48-8f57-73b9b101fbb0 None None] SCRIPT_NAME: `/identity` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:33:39.739308 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-753bc9ff-dc8c-4e48-8f57-73b9b101fbb0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:33:39.799354 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.auth.core [None req-753bc9ff-dc8c-4e48-8f57-73b9b101fbb0 None None] MFA Rules not processed for user `2dce40b8d74c41ec94cc36e6ff05d412`. Rule list: `[]` (Enabled: `True`). {{(pid=80245) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 08:33:39.805403 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.fernet_utils [None req-753bc9ff-dc8c-4e48-8f57-73b9b101fbb0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80245) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 08:33:39.829967 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: [pid: 80245|app: 0|req: 1281/2553] 10.222.0.47 () {58 vars in 957 bytes} [Thu Jul 21 08:33:39 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 08:33:39.832966 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a94255fd-9685-4b0e-891f-d59e83bce0fd None None] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:33:39.848046 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a94255fd-9685-4b0e-891f-d59e83bce0fd None None] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:33:39.867153 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7daf6322-fc58-47f4-bc69-1cb70a764405 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '42d15fc0515945acacbbffcf2356c7a5', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:33:39.869522 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7daf6322-fc58-47f4-bc69-1cb70a764405 admin admin] REQUEST_METHOD: `GET` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:33:39.873416 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7daf6322-fc58-47f4-bc69-1cb70a764405 admin admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:33:39.874103 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7daf6322-fc58-47f4-bc69-1cb70a764405 admin admin] PATH_INFO: `/v3/domains` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:33:39.875138 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7daf6322-fc58-47f4-bc69-1cb70a764405 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80244) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 08:33:39.875710 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7daf6322-fc58-47f4-bc69-1cb70a764405 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:33:39.877636 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7daf6322-fc58-47f4-bc69-1cb70a764405 admin admin] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:33:39.895808 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 1273/2554] 10.222.0.47 () {58 vars in 1152 bytes} [Thu Jul 21 08:33:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 64 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 08:33:39.899673 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-753bc9ff-dc8c-4e48-8f57-73b9b101fbb0 None None] Authenticating user token {{(pid=80245) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:33:39.918470 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-753bc9ff-dc8c-4e48-8f57-73b9b101fbb0 None None] Validating token access rules against request {{(pid=80245) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:33:39.945341 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc61406f-5f0b-47ca-9212-83cfc576137a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '42d15fc0515945acacbbffcf2356c7a5', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80245) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:33:39.948259 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc61406f-5f0b-47ca-9212-83cfc576137a admin admin] REQUEST_METHOD: `POST` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:33:39.950804 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc61406f-5f0b-47ca-9212-83cfc576137a admin admin] SCRIPT_NAME: `/identity` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:33:39.951181 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc61406f-5f0b-47ca-9212-83cfc576137a admin admin] PATH_INFO: `/v3/projects` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:33:39.961249 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc61406f-5f0b-47ca-9212-83cfc576137a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:33:39.969399 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc61406f-5f0b-47ca-9212-83cfc576137a admin admin] RBAC: Authorization granted {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:33:40.099362 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: [pid: 80245|app: 0|req: 1282/2555] 10.222.0.47 () {60 vars in 1152 bytes} [Thu Jul 21 08:33:39 2022] POST /identity/v3/projects => generated 375 bytes in 200 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 08:33:40.104693 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7daf6322-fc58-47f4-bc69-1cb70a764405 admin admin] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:33:40.121678 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7daf6322-fc58-47f4-bc69-1cb70a764405 admin admin] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:33:40.328487 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc7b8772-30a5-49a4-ad1d-2d72dac46c8d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '42d15fc0515945acacbbffcf2356c7a5', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:33:40.329301 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc7b8772-30a5-49a4-ad1d-2d72dac46c8d admin admin] REQUEST_METHOD: `POST` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:33:40.329577 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc7b8772-30a5-49a4-ad1d-2d72dac46c8d admin admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:33:40.329853 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc7b8772-30a5-49a4-ad1d-2d72dac46c8d admin admin] PATH_INFO: `/v3/users` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:33:40.330567 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc7b8772-30a5-49a4-ad1d-2d72dac46c8d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:33:40.333201 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc7b8772-30a5-49a4-ad1d-2d72dac46c8d admin admin] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:33:40.429167 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 1274/2556] 10.222.0.47 () {60 vars in 1143 bytes} [Thu Jul 21 08:33:40 2022] POST /identity/v3/users => generated 288 bytes in 325 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 08:33:40.432118 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc61406f-5f0b-47ca-9212-83cfc576137a admin admin] Authenticating user token {{(pid=80245) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:33:40.457171 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc61406f-5f0b-47ca-9212-83cfc576137a admin admin] Validating token access rules against request {{(pid=80245) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:33:40.464687 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-436a8a9a-3ec0-4e73-bdc7-1a1dea68edd3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '42d15fc0515945acacbbffcf2356c7a5', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80245) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:33:40.465591 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-436a8a9a-3ec0-4e73-bdc7-1a1dea68edd3 admin admin] REQUEST_METHOD: `GET` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:33:40.466080 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-436a8a9a-3ec0-4e73-bdc7-1a1dea68edd3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:33:40.466395 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-436a8a9a-3ec0-4e73-bdc7-1a1dea68edd3 admin admin] PATH_INFO: `/v3/roles` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:33:40.467282 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-436a8a9a-3ec0-4e73-bdc7-1a1dea68edd3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:33:40.469705 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-436a8a9a-3ec0-4e73-bdc7-1a1dea68edd3 admin admin] RBAC: Authorization granted {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:33:40.494401 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: [pid: 80245|app: 0|req: 1283/2557] 10.222.0.47 () {58 vars in 1121 bytes} [Thu Jul 21 08:33:40 2022] GET /identity/v3/roles => generated 1420 bytes in 64 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 08:33:40.496003 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc7b8772-30a5-49a4-ad1d-2d72dac46c8d admin admin] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:33:40.513132 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc7b8772-30a5-49a4-ad1d-2d72dac46c8d admin admin] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:33:40.527121 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d002310-08e6-46f9-b2cb-9cda0929c47b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '42d15fc0515945acacbbffcf2356c7a5', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:33:40.528323 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d002310-08e6-46f9-b2cb-9cda0929c47b admin admin] REQUEST_METHOD: `GET` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:33:40.528608 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d002310-08e6-46f9-b2cb-9cda0929c47b admin admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:33:40.528896 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d002310-08e6-46f9-b2cb-9cda0929c47b admin admin] PATH_INFO: `/v3/roles` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:33:40.529748 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d002310-08e6-46f9-b2cb-9cda0929c47b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:33:40.531544 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d002310-08e6-46f9-b2cb-9cda0929c47b admin admin] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:33:40.562680 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 1275/2558] 10.222.0.47 () {58 vars in 1121 bytes} [Thu Jul 21 08:33:40 2022] GET /identity/v3/roles => generated 1420 bytes in 67 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 08:33:40.566332 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-436a8a9a-3ec0-4e73-bdc7-1a1dea68edd3 admin admin] Authenticating user token {{(pid=80245) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:33:40.586250 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-436a8a9a-3ec0-4e73-bdc7-1a1dea68edd3 admin admin] Validating token access rules against request {{(pid=80245) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:33:40.596392 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-785c77f8-8b8f-418d-b1d5-67b155796d92 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '42d15fc0515945acacbbffcf2356c7a5', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80245) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:33:40.597888 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-785c77f8-8b8f-418d-b1d5-67b155796d92 admin admin] REQUEST_METHOD: `PUT` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:33:40.598333 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-785c77f8-8b8f-418d-b1d5-67b155796d92 admin admin] SCRIPT_NAME: `/identity` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:33:40.598787 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-785c77f8-8b8f-418d-b1d5-67b155796d92 admin admin] PATH_INFO: `/v3/projects/05b859e8bf4847eea99f02efae52cdd1/users/1c6e41c41d76488fb71c1b8a5358b637/roles/3a074738221e4601b5205715ba87d240` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:33:40.629007 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-785c77f8-8b8f-418d-b1d5-67b155796d92 admin admin] RBAC: Authorizing `identity:create_grant(project_id=05b859e8bf4847eea99f02efae52cdd1, user_id=1c6e41c41d76488fb71c1b8a5358b637, role_id=3a074738221e4601b5205715ba87d240)` {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:33:40.646772 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-785c77f8-8b8f-418d-b1d5-67b155796d92 admin admin] RBAC: Authorization granted {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:33:40.650134 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG dbcounter [-] [80245] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80245) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:33:40.689268 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: [pid: 80245|app: 0|req: 1284/2559] 10.222.0.47 () {60 vars in 1482 bytes} [Thu Jul 21 08:33:40 2022] PUT /identity/v3/projects/05b859e8bf4847eea99f02efae52cdd1/users/1c6e41c41d76488fb71c1b8a5358b637/roles/3a074738221e4601b5205715ba87d240 => generated 0 bytes in 115 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 08:33:40.691015 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7935a42a-80b2-4113-9aed-d0836f379926 None None] REQUEST_METHOD: `POST` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:33:40.691563 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7935a42a-80b2-4113-9aed-d0836f379926 None None] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:33:40.691988 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7935a42a-80b2-4113-9aed-d0836f379926 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:33:40.766068 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.auth.core [None req-7935a42a-80b2-4113-9aed-d0836f379926 None None] MFA Rules not processed for user `1c6e41c41d76488fb71c1b8a5358b637`. Rule list: `[]` (Enabled: `True`). {{(pid=80244) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 08:33:40.768311 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.fernet_utils [None req-7935a42a-80b2-4113-9aed-d0836f379926 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80244) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 08:33:40.982908 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 1276/2560] 10.222.0.47 () {58 vars in 957 bytes} [Thu Jul 21 08:33:40 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 293 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 08:33:40.992238 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-785c77f8-8b8f-418d-b1d5-67b155796d92 admin admin] Authenticating user token {{(pid=80245) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:33:40.996000 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.fernet_utils [None req-785c77f8-8b8f-418d-b1d5-67b155796d92 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80245) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 08:33:41.123017 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-785c77f8-8b8f-418d-b1d5-67b155796d92 admin admin] Validating token access rules against request {{(pid=80245) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:33:41.144272 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c1d299b-0272-43e1-a08e-13b7f9afa0af service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4f40410b7674687ab5362ac982248ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4faef13aa50e4e36b6c265fb1398fa14', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80245) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:33:41.144972 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c1d299b-0272-43e1-a08e-13b7f9afa0af service cinder] REQUEST_METHOD: `GET` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:33:41.145256 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c1d299b-0272-43e1-a08e-13b7f9afa0af service cinder] SCRIPT_NAME: `/identity` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:33:41.145536 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c1d299b-0272-43e1-a08e-13b7f9afa0af service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:33:41.174122 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c1d299b-0272-43e1-a08e-13b7f9afa0af service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:33:41.176107 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c1d299b-0272-43e1-a08e-13b7f9afa0af service cinder] RBAC: Authorization granted {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:33:41.205709 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: [pid: 80245|app: 0|req: 1285/2561] 10.222.0.47 () {60 vars in 1365 bytes} [Thu Jul 21 08:33:40 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 214 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 08:33:42.920913 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-180decad-9287-4fdc-907e-13c33582e7df None None] REQUEST_METHOD: `POST` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:33:42.921434 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-180decad-9287-4fdc-907e-13c33582e7df None None] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:33:42.921524 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-180decad-9287-4fdc-907e-13c33582e7df None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:33:42.988152 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.auth.core [None req-180decad-9287-4fdc-907e-13c33582e7df None None] MFA Rules not processed for user `1c6e41c41d76488fb71c1b8a5358b637`. Rule list: `[]` (Enabled: `True`). {{(pid=80244) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 08:33:42.989516 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.fernet_utils [None req-180decad-9287-4fdc-907e-13c33582e7df None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80244) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 08:33:43.013459 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 1277/2562] 10.222.0.47 () {58 vars in 957 bytes} [Thu Jul 21 08:33:42 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 21 08:33:51.185070 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG dbcounter [-] [80245] Writing DB stats keystone:INSERT=1,keystone:SELECT=28 {{(pid=80245) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:33:52.952347 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG dbcounter [-] [80244] Writing DB stats keystone:SELECT=90,keystone:INSERT=3 {{(pid=80244) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:33:56.309901 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df3dccc0-bfcb-4bdc-8954-8b4ead16acf3 None None] REQUEST_METHOD: `GET` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:33:56.310863 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df3dccc0-bfcb-4bdc-8954-8b4ead16acf3 None None] SCRIPT_NAME: `/identity` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:33:56.311305 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df3dccc0-bfcb-4bdc-8954-8b4ead16acf3 None None] PATH_INFO: `/` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:33:56.313232 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: [pid: 80245|app: 0|req: 1286/2563] 10.222.0.47 () {54 vars in 921 bytes} [Thu Jul 21 08:33:56 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 08:33:56.318957 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c18f933-4c95-4124-b58a-2aeb1af052d6 None None] REQUEST_METHOD: `POST` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:33:56.319476 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c18f933-4c95-4124-b58a-2aeb1af052d6 None None] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:33:56.319591 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c18f933-4c95-4124-b58a-2aeb1af052d6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:33:56.338378 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.auth.core [None req-3c18f933-4c95-4124-b58a-2aeb1af052d6 None None] MFA Rules not processed for user `ac2b5b99a36442148fe743269054e991`. Rule list: `[]` (Enabled: `True`). {{(pid=80244) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 08:33:56.340196 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.fernet_utils [None req-3c18f933-4c95-4124-b58a-2aeb1af052d6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80244) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 08:33:56.431257 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 1278/2564] 10.222.0.47 () {58 vars in 1018 bytes} [Thu Jul 21 08:33:56 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 08:33:56.443896 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df3dccc0-bfcb-4bdc-8954-8b4ead16acf3 None None] Authenticating user token {{(pid=80245) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:33:56.445973 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.fernet_utils [None req-df3dccc0-bfcb-4bdc-8954-8b4ead16acf3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80245) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 08:33:56.531972 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df3dccc0-bfcb-4bdc-8954-8b4ead16acf3 None None] Validating token access rules against request {{(pid=80245) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:33:56.541297 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3824478a-7428-4031-a0dc-569b4668c2cc service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9b0524aed8c74abd8949b860d6b93936', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4faef13aa50e4e36b6c265fb1398fa14', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80245) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:33:56.542274 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3824478a-7428-4031-a0dc-569b4668c2cc service glance] REQUEST_METHOD: `GET` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:33:56.542704 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3824478a-7428-4031-a0dc-569b4668c2cc service glance] SCRIPT_NAME: `/identity` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:33:56.543090 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3824478a-7428-4031-a0dc-569b4668c2cc service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:33:56.554507 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3824478a-7428-4031-a0dc-569b4668c2cc service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:33:56.556600 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3824478a-7428-4031-a0dc-569b4668c2cc service glance] RBAC: Authorization granted {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:33:56.568043 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: [pid: 80245|app: 0|req: 1287/2565] 10.222.0.47 () {60 vars in 1365 bytes} [Thu Jul 21 08:33:56 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 125 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 08:33:56.634976 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f55d4fd-d5b0-451b-bb9c-bb2a181e2916 None None] REQUEST_METHOD: `GET` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:33:56.635260 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f55d4fd-d5b0-451b-bb9c-bb2a181e2916 None None] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:33:56.635572 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f55d4fd-d5b0-451b-bb9c-bb2a181e2916 None None] PATH_INFO: `/` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:33:56.636939 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 1279/2566] 10.222.0.47 () {54 vars in 921 bytes} [Thu Jul 21 08:33:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 08:33:56.641827 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8093074d-8e61-4296-a479-83306bf4b251 None None] REQUEST_METHOD: `POST` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:33:56.642121 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8093074d-8e61-4296-a479-83306bf4b251 None None] SCRIPT_NAME: `/identity` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:33:56.642407 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8093074d-8e61-4296-a479-83306bf4b251 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:33:56.661665 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.auth.core [None req-8093074d-8e61-4296-a479-83306bf4b251 None None] MFA Rules not processed for user `20ca3c3bd3774691b3374cfbfec6e494`. Rule list: `[]` (Enabled: `True`). {{(pid=80245) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 08:33:56.662882 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.fernet_utils [None req-8093074d-8e61-4296-a479-83306bf4b251 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80245) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 08:33:56.714063 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: [pid: 80245|app: 0|req: 1288/2567] 10.222.0.47 () {58 vars in 1018 bytes} [Thu Jul 21 08:33:56 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 73 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 08:33:56.727198 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f55d4fd-d5b0-451b-bb9c-bb2a181e2916 None None] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:33:56.728824 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.fernet_utils [None req-6f55d4fd-d5b0-451b-bb9c-bb2a181e2916 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80244) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 08:33:56.741851 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f55d4fd-d5b0-451b-bb9c-bb2a181e2916 None None] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:33:56.745380 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0a1e2ab-2b91-4173-8db9-c62eef41aef5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20ca3c3bd3774691b3374cfbfec6e494', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4faef13aa50e4e36b6c265fb1398fa14', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:33:56.746011 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0a1e2ab-2b91-4173-8db9-c62eef41aef5 service neutron] REQUEST_METHOD: `GET` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:33:56.746291 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0a1e2ab-2b91-4173-8db9-c62eef41aef5 service neutron] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:33:56.746566 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0a1e2ab-2b91-4173-8db9-c62eef41aef5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:33:56.753838 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0a1e2ab-2b91-4173-8db9-c62eef41aef5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:33:56.755244 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0a1e2ab-2b91-4173-8db9-c62eef41aef5 service neutron] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:33:56.762964 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 1280/2568] 10.222.0.47 () {60 vars in 1365 bytes} [Thu Jul 21 08:33:56 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 08:33:56.849261 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2a75eae-b357-4125-9ed8-d09d09f60551 None None] REQUEST_METHOD: `GET` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:33:56.849443 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2a75eae-b357-4125-9ed8-d09d09f60551 None None] SCRIPT_NAME: `/identity` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:33:56.849796 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2a75eae-b357-4125-9ed8-d09d09f60551 None None] PATH_INFO: `/` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:33:56.851286 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: [pid: 80245|app: 0|req: 1289/2569] 10.222.0.47 () {54 vars in 921 bytes} [Thu Jul 21 08:33:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 08:33:56.857404 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-622ae5b7-aa80-4967-8809-3b865052a32c None None] REQUEST_METHOD: `POST` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:33:56.857698 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-622ae5b7-aa80-4967-8809-3b865052a32c None None] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:33:56.858007 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-622ae5b7-aa80-4967-8809-3b865052a32c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:33:56.876257 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.auth.core [None req-622ae5b7-aa80-4967-8809-3b865052a32c None None] MFA Rules not processed for user `ac2b5b99a36442148fe743269054e991`. Rule list: `[]` (Enabled: `True`). {{(pid=80244) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 08:33:56.877585 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.fernet_utils [None req-622ae5b7-aa80-4967-8809-3b865052a32c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80244) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 08:33:56.887661 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 1281/2570] 10.222.0.47 () {58 vars in 1018 bytes} [Thu Jul 21 08:33:56 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 08:33:56.898848 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2a75eae-b357-4125-9ed8-d09d09f60551 None None] Authenticating user token {{(pid=80245) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:33:56.900709 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.fernet_utils [None req-c2a75eae-b357-4125-9ed8-d09d09f60551 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80245) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 08:33:56.917930 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2a75eae-b357-4125-9ed8-d09d09f60551 None None] Validating token access rules against request {{(pid=80245) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:33:56.924141 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1288f202-aa7b-4f21-8460-c469c8895d53 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20ca3c3bd3774691b3374cfbfec6e494', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4faef13aa50e4e36b6c265fb1398fa14', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80245) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:33:56.925143 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1288f202-aa7b-4f21-8460-c469c8895d53 service neutron] REQUEST_METHOD: `GET` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:33:56.925607 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1288f202-aa7b-4f21-8460-c469c8895d53 service neutron] SCRIPT_NAME: `/identity` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:33:56.926128 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1288f202-aa7b-4f21-8460-c469c8895d53 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:33:56.939715 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1288f202-aa7b-4f21-8460-c469c8895d53 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:33:56.941568 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1288f202-aa7b-4f21-8460-c469c8895d53 service neutron] RBAC: Authorization granted {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:33:56.954817 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: [pid: 80245|app: 0|req: 1290/2571] 10.222.0.47 () {60 vars in 1365 bytes} [Thu Jul 21 08:33:56 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 57 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 21 08:34:05.264837 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cb887ed-8cb6-4193-8b68-2d8b5181b5ef None None] REQUEST_METHOD: `POST` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:34:05.265343 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cb887ed-8cb6-4193-8b68-2d8b5181b5ef None None] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:34:05.265455 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cb887ed-8cb6-4193-8b68-2d8b5181b5ef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:34:05.301329 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.auth.core [None req-8cb887ed-8cb6-4193-8b68-2d8b5181b5ef None None] MFA Rules not processed for user `2ef7f1e91e554a1f88eb6a63e60a475f`. Rule list: `[]` (Enabled: `True`). {{(pid=80244) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 08:34:05.302547 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.fernet_utils [None req-8cb887ed-8cb6-4193-8b68-2d8b5181b5ef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80244) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 08:34:05.385183 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 1282/2572] 10.222.0.47 () {58 vars in 1009 bytes} [Thu Jul 21 08:34:05 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 08:34:05.412118 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1288f202-aa7b-4f21-8460-c469c8895d53 service neutron] Authenticating user token {{(pid=80245) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:34:05.413880 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.fernet_utils [None req-1288f202-aa7b-4f21-8460-c469c8895d53 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80245) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 08:34:05.493943 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1288f202-aa7b-4f21-8460-c469c8895d53 service neutron] Validating token access rules against request {{(pid=80245) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:34:05.499394 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7896bfdc-fcd8-40db-94a1-87719db4cae5 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '302351411fc84248a6b166b25d21609f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4faef13aa50e4e36b6c265fb1398fa14', 'project_domain_id': 'default', 'roles': ['service', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80245) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:34:05.500101 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7896bfdc-fcd8-40db-94a1-87719db4cae5 service swift] REQUEST_METHOD: `GET` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:34:05.500394 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7896bfdc-fcd8-40db-94a1-87719db4cae5 service swift] SCRIPT_NAME: `/identity` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:34:05.500677 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7896bfdc-fcd8-40db-94a1-87719db4cae5 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:34:05.510297 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7896bfdc-fcd8-40db-94a1-87719db4cae5 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:34:05.511940 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7896bfdc-fcd8-40db-94a1-87719db4cae5 service swift] RBAC: Authorization granted {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:34:05.520744 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: [pid: 80245|app: 0|req: 1291/2573] 10.222.0.47 () {60 vars in 1384 bytes} [Thu Jul 21 08:34:05 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 109 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 21 08:34:15.370587 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG dbcounter [-] [80244] Writing DB stats keystone:SELECT=65 {{(pid=80244) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:34:15.517137 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG dbcounter [-] [80245] Writing DB stats keystone:SELECT=87 {{(pid=80245) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:34:22.740994 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cb887ed-8cb6-4193-8b68-2d8b5181b5ef None None] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:34:22.753751 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cb887ed-8cb6-4193-8b68-2d8b5181b5ef None None] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:34:22.757260 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4df537e8-914e-489c-b2c2-9debc0374062 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20ca3c3bd3774691b3374cfbfec6e494', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4faef13aa50e4e36b6c265fb1398fa14', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:34:22.758576 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4df537e8-914e-489c-b2c2-9debc0374062 service neutron] REQUEST_METHOD: `GET` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:34:22.759021 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4df537e8-914e-489c-b2c2-9debc0374062 service neutron] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:34:22.759494 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4df537e8-914e-489c-b2c2-9debc0374062 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:34:22.761640 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.fernet_utils [None req-4df537e8-914e-489c-b2c2-9debc0374062 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80244) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 08:34:22.826264 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4df537e8-914e-489c-b2c2-9debc0374062 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:34:22.827870 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4df537e8-914e-489c-b2c2-9debc0374062 service neutron] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:34:22.838392 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 1283/2574] 10.222.0.47 () {60 vars in 1365 bytes} [Thu Jul 21 08:34:22 2022] GET /identity/v3/auth/tokens => generated 3319 bytes in 98 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 08:34:25.575883 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7896bfdc-fcd8-40db-94a1-87719db4cae5 service swift] Authenticating user token {{(pid=80245) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:34:25.588707 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7896bfdc-fcd8-40db-94a1-87719db4cae5 service swift] Validating token access rules against request {{(pid=80245) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:34:25.593077 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b69d17e1-d913-46b8-b39e-0c19e58dbcc3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20ca3c3bd3774691b3374cfbfec6e494', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4faef13aa50e4e36b6c265fb1398fa14', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80245) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:34:25.593741 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b69d17e1-d913-46b8-b39e-0c19e58dbcc3 service neutron] REQUEST_METHOD: `GET` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:34:25.594027 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b69d17e1-d913-46b8-b39e-0c19e58dbcc3 service neutron] SCRIPT_NAME: `/identity` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:34:25.594320 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b69d17e1-d913-46b8-b39e-0c19e58dbcc3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:34:25.596243 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.fernet_utils [None req-b69d17e1-d913-46b8-b39e-0c19e58dbcc3 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80245) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 08:34:25.695859 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b69d17e1-d913-46b8-b39e-0c19e58dbcc3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:34:25.698574 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b69d17e1-d913-46b8-b39e-0c19e58dbcc3 service neutron] RBAC: Authorization granted {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:34:25.716351 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: [pid: 80245|app: 0|req: 1292/2575] 10.222.0.47 () {60 vars in 1365 bytes} [Thu Jul 21 08:34:25 2022] GET /identity/v3/auth/tokens => generated 3307 bytes in 141 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 08:34:28.246474 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01890494-68ca-4e94-af06-6dc4e58c9dfa None None] REQUEST_METHOD: `POST` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:34:28.247011 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01890494-68ca-4e94-af06-6dc4e58c9dfa None None] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:34:28.247011 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01890494-68ca-4e94-af06-6dc4e58c9dfa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:34:28.266631 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.auth.core [None req-01890494-68ca-4e94-af06-6dc4e58c9dfa None None] MFA Rules not processed for user `2ef7f1e91e554a1f88eb6a63e60a475f`. Rule list: `[]` (Enabled: `True`). {{(pid=80244) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 08:34:28.267970 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.fernet_utils [None req-01890494-68ca-4e94-af06-6dc4e58c9dfa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80244) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 08:34:28.277208 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 1284/2576] 10.222.0.47 () {58 vars in 1009 bytes} [Thu Jul 21 08:34:28 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 08:34:28.291403 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b69d17e1-d913-46b8-b39e-0c19e58dbcc3 service neutron] Authenticating user token {{(pid=80245) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:34:28.303722 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b69d17e1-d913-46b8-b39e-0c19e58dbcc3 service neutron] Validating token access rules against request {{(pid=80245) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:34:28.309934 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc6cb19a-b387-4d11-aadc-56919b84a1ed service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '302351411fc84248a6b166b25d21609f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4faef13aa50e4e36b6c265fb1398fa14', 'project_domain_id': 'default', 'roles': ['service', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80245) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:34:28.310957 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc6cb19a-b387-4d11-aadc-56919b84a1ed service swift] REQUEST_METHOD: `GET` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:34:28.311493 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc6cb19a-b387-4d11-aadc-56919b84a1ed service swift] SCRIPT_NAME: `/identity` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:34:28.311880 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc6cb19a-b387-4d11-aadc-56919b84a1ed service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:34:28.321238 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc6cb19a-b387-4d11-aadc-56919b84a1ed service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:34:28.323175 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc6cb19a-b387-4d11-aadc-56919b84a1ed service swift] RBAC: Authorization granted {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:34:28.333138 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: [pid: 80245|app: 0|req: 1293/2577] 10.222.0.47 () {60 vars in 1384 bytes} [Thu Jul 21 08:34:28 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Jul 21 08:34:29.222213 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd9c511a-e85f-4966-9274-c41371679c92 None None] REQUEST_METHOD: `POST` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:34:29.222917 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd9c511a-e85f-4966-9274-c41371679c92 None None] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:34:29.223597 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd9c511a-e85f-4966-9274-c41371679c92 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:34:29.252592 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.auth.core [None req-dd9c511a-e85f-4966-9274-c41371679c92 None None] MFA Rules not processed for user `2ef7f1e91e554a1f88eb6a63e60a475f`. Rule list: `[]` (Enabled: `True`). {{(pid=80244) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 08:34:29.254729 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.fernet_utils [None req-dd9c511a-e85f-4966-9274-c41371679c92 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80244) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 08:34:29.268708 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 1285/2578] 10.222.0.47 () {58 vars in 967 bytes} [Thu Jul 21 08:34:29 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 48 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 08:34:29.286625 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc6cb19a-b387-4d11-aadc-56919b84a1ed service swift] Authenticating user token {{(pid=80245) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:34:29.303251 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc6cb19a-b387-4d11-aadc-56919b84a1ed service swift] Validating token access rules against request {{(pid=80245) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:34:29.309287 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b68a52c5-cbf8-4a94-922c-c456fbb56af6 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '302351411fc84248a6b166b25d21609f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4faef13aa50e4e36b6c265fb1398fa14', 'project_domain_id': 'default', 'roles': ['service', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80245) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:34:29.310211 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b68a52c5-cbf8-4a94-922c-c456fbb56af6 service swift] REQUEST_METHOD: `GET` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:34:29.310701 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b68a52c5-cbf8-4a94-922c-c456fbb56af6 service swift] SCRIPT_NAME: `/identity` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:34:29.311166 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b68a52c5-cbf8-4a94-922c-c456fbb56af6 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:34:29.324114 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b68a52c5-cbf8-4a94-922c-c456fbb56af6 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:34:29.326172 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b68a52c5-cbf8-4a94-922c-c456fbb56af6 service swift] RBAC: Authorization granted {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:34:29.336458 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: [pid: 80245|app: 0|req: 1294/2579] 10.222.0.47 () {60 vars in 1384 bytes} [Thu Jul 21 08:34:29 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 50 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 21 08:34:29.606499 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a31dfca-d44a-43e9-826b-514e154a9700 None None] REQUEST_METHOD: `POST` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:34:29.606835 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a31dfca-d44a-43e9-826b-514e154a9700 None None] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:34:29.607076 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a31dfca-d44a-43e9-826b-514e154a9700 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:34:29.625226 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.auth.core [None req-3a31dfca-d44a-43e9-826b-514e154a9700 None None] MFA Rules not processed for user `2ef7f1e91e554a1f88eb6a63e60a475f`. Rule list: `[]` (Enabled: `True`). {{(pid=80244) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 08:34:29.626441 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.fernet_utils [None req-3a31dfca-d44a-43e9-826b-514e154a9700 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80244) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 08:34:29.634169 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 1286/2580] 10.222.0.47 () {58 vars in 1009 bytes} [Thu Jul 21 08:34:29 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 08:34:29.646274 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b68a52c5-cbf8-4a94-922c-c456fbb56af6 service swift] Authenticating user token {{(pid=80245) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:34:29.657333 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b68a52c5-cbf8-4a94-922c-c456fbb56af6 service swift] Validating token access rules against request {{(pid=80245) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:34:29.664495 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6187bd66-ba76-4cc5-be73-e655d8ded0d9 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '302351411fc84248a6b166b25d21609f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4faef13aa50e4e36b6c265fb1398fa14', 'project_domain_id': 'default', 'roles': ['service', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80245) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:34:29.665148 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6187bd66-ba76-4cc5-be73-e655d8ded0d9 service swift] REQUEST_METHOD: `GET` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:34:29.665430 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6187bd66-ba76-4cc5-be73-e655d8ded0d9 service swift] SCRIPT_NAME: `/identity` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:34:29.665701 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6187bd66-ba76-4cc5-be73-e655d8ded0d9 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:34:29.674551 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6187bd66-ba76-4cc5-be73-e655d8ded0d9 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:34:29.676167 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6187bd66-ba76-4cc5-be73-e655d8ded0d9 service swift] RBAC: Authorization granted {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:34:29.684952 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: [pid: 80245|app: 0|req: 1295/2581] 10.222.0.47 () {60 vars in 1384 bytes} [Thu Jul 21 08:34:29 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 21 08:34:30.049877 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e848687-d0b8-4e47-af9d-bab5d32ae9da None None] REQUEST_METHOD: `POST` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:34:30.050366 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e848687-d0b8-4e47-af9d-bab5d32ae9da None None] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:34:30.050714 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e848687-d0b8-4e47-af9d-bab5d32ae9da None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:34:30.071271 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.auth.core [None req-0e848687-d0b8-4e47-af9d-bab5d32ae9da None None] MFA Rules not processed for user `2ef7f1e91e554a1f88eb6a63e60a475f`. Rule list: `[]` (Enabled: `True`). {{(pid=80244) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 08:34:30.072507 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.fernet_utils [None req-0e848687-d0b8-4e47-af9d-bab5d32ae9da None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80244) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 08:34:30.079972 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 1287/2582] 10.222.0.47 () {58 vars in 967 bytes} [Thu Jul 21 08:34:30 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 08:34:30.103116 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6187bd66-ba76-4cc5-be73-e655d8ded0d9 service swift] Authenticating user token {{(pid=80245) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:34:30.114572 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6187bd66-ba76-4cc5-be73-e655d8ded0d9 service swift] Validating token access rules against request {{(pid=80245) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:34:30.122449 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f371060-15a7-4d05-ac75-8180a484915b service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '302351411fc84248a6b166b25d21609f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4faef13aa50e4e36b6c265fb1398fa14', 'project_domain_id': 'default', 'roles': ['service', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80245) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:34:30.123219 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f371060-15a7-4d05-ac75-8180a484915b service swift] REQUEST_METHOD: `GET` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:34:30.123774 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f371060-15a7-4d05-ac75-8180a484915b service swift] SCRIPT_NAME: `/identity` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:34:30.124230 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f371060-15a7-4d05-ac75-8180a484915b service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:34:30.135611 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f371060-15a7-4d05-ac75-8180a484915b service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:34:30.137803 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f371060-15a7-4d05-ac75-8180a484915b service swift] RBAC: Authorization granted {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:34:30.150116 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: [pid: 80245|app: 0|req: 1296/2583] 10.222.0.47 () {60 vars in 1384 bytes} [Thu Jul 21 08:34:30 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 47 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Jul 21 08:34:34.516055 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e848687-d0b8-4e47-af9d-bab5d32ae9da None None] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:34:34.582862 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e848687-d0b8-4e47-af9d-bab5d32ae9da None None] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:34:34.587343 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2d8c950-55de-4d8f-93ef-a2372aa6e6da admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '42d15fc0515945acacbbffcf2356c7a5', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:34:34.588218 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2d8c950-55de-4d8f-93ef-a2372aa6e6da admin admin] REQUEST_METHOD: `DELETE` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:34:34.588499 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2d8c950-55de-4d8f-93ef-a2372aa6e6da admin admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:34:34.588769 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2d8c950-55de-4d8f-93ef-a2372aa6e6da admin admin] PATH_INFO: `/v3/users/1c6e41c41d76488fb71c1b8a5358b637` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:34:34.589975 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2d8c950-55de-4d8f-93ef-a2372aa6e6da admin admin] RBAC: Authorizing `identity:delete_user(user_id=1c6e41c41d76488fb71c1b8a5358b637)` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:34:34.592705 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2d8c950-55de-4d8f-93ef-a2372aa6e6da admin admin] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:34:34.640104 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.notifications [None req-b2d8c950-55de-4d8f-93ef-a2372aa6e6da admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1c6e41c41d76488fb71c1b8a5358b637'} {{(pid=80244) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 08:34:34.648234 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.notifications [None req-b2d8c950-55de-4d8f-93ef-a2372aa6e6da admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1c6e41c41d76488fb71c1b8a5358b637'} {{(pid=80244) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 08:34:34.659625 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG dbcounter [-] [80244] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80244) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:34:34.660981 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.notifications [None req-b2d8c950-55de-4d8f-93ef-a2372aa6e6da admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1c6e41c41d76488fb71c1b8a5358b637'} {{(pid=80244) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 08:34:34.661817 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.notifications [None req-b2d8c950-55de-4d8f-93ef-a2372aa6e6da admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1c6e41c41d76488fb71c1b8a5358b637'} {{(pid=80244) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 08:34:34.671172 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.notifications [None req-b2d8c950-55de-4d8f-93ef-a2372aa6e6da admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1c6e41c41d76488fb71c1b8a5358b637'} {{(pid=80244) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 08:34:34.677263 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 1288/2584] 10.222.0.47 () {58 vars in 1223 bytes} [Thu Jul 21 08:34:34 2022] DELETE /identity/v3/users/1c6e41c41d76488fb71c1b8a5358b637 => generated 0 bytes in 162 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 08:34:34.682390 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc38d7ba-d6d7-467e-b8cb-5afd4c165a45 None None] REQUEST_METHOD: `POST` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:34:34.682846 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc38d7ba-d6d7-467e-b8cb-5afd4c165a45 None None] SCRIPT_NAME: `/identity` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:34:34.682938 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc38d7ba-d6d7-467e-b8cb-5afd4c165a45 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:34:34.701170 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.auth.core [None req-dc38d7ba-d6d7-467e-b8cb-5afd4c165a45 None None] MFA Rules not processed for user `2dce40b8d74c41ec94cc36e6ff05d412`. Rule list: `[]` (Enabled: `True`). {{(pid=80245) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 08:34:34.702374 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.fernet_utils [None req-dc38d7ba-d6d7-467e-b8cb-5afd4c165a45 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80245) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 08:34:34.759847 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: [pid: 80245|app: 0|req: 1297/2585] 10.222.0.47 () {58 vars in 957 bytes} [Thu Jul 21 08:34:34 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 78 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 08:34:34.769890 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2d8c950-55de-4d8f-93ef-a2372aa6e6da admin admin] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:34:34.771359 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.fernet_utils [None req-b2d8c950-55de-4d8f-93ef-a2372aa6e6da admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80244) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 08:34:34.828589 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2d8c950-55de-4d8f-93ef-a2372aa6e6da admin admin] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:34:34.832268 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76fb2dce-c056-484d-bcb3-3eb031c3b6dc service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20ca3c3bd3774691b3374cfbfec6e494', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4faef13aa50e4e36b6c265fb1398fa14', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:34:34.832915 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76fb2dce-c056-484d-bcb3-3eb031c3b6dc service neutron] REQUEST_METHOD: `GET` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:34:34.833204 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76fb2dce-c056-484d-bcb3-3eb031c3b6dc service neutron] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:34:34.833483 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76fb2dce-c056-484d-bcb3-3eb031c3b6dc service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:34:34.843590 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76fb2dce-c056-484d-bcb3-3eb031c3b6dc service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:34:34.845037 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76fb2dce-c056-484d-bcb3-3eb031c3b6dc service neutron] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:34:34.856113 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 1289/2586] 10.222.0.47 () {60 vars in 1365 bytes} [Thu Jul 21 08:34:34 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 08:34:35.157305 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc38d7ba-d6d7-467e-b8cb-5afd4c165a45 None None] Authenticating user token {{(pid=80245) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:34:35.158889 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.fernet_utils [None req-dc38d7ba-d6d7-467e-b8cb-5afd4c165a45 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80245) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 08:34:35.172297 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc38d7ba-d6d7-467e-b8cb-5afd4c165a45 None None] Validating token access rules against request {{(pid=80245) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:34:35.179464 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6ef586e-2bfe-42f4-8a57-261da947d8b6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '42d15fc0515945acacbbffcf2356c7a5', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80245) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:34:35.180896 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6ef586e-2bfe-42f4-8a57-261da947d8b6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:34:35.181538 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6ef586e-2bfe-42f4-8a57-261da947d8b6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:34:35.182028 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6ef586e-2bfe-42f4-8a57-261da947d8b6 admin admin] PATH_INFO: `/v3/projects/05b859e8bf4847eea99f02efae52cdd1` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:34:35.183779 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6ef586e-2bfe-42f4-8a57-261da947d8b6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=05b859e8bf4847eea99f02efae52cdd1)` {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:34:35.186022 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6ef586e-2bfe-42f4-8a57-261da947d8b6 admin admin] RBAC: Authorization granted {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:34:35.234979 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG dbcounter [-] [80245] Writing DB stats keystone:SELECT=97,keystone:DELETE=3 {{(pid=80245) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:34:35.248380 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.notifications [None req-a6ef586e-2bfe-42f4-8a57-261da947d8b6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '05b859e8bf4847eea99f02efae52cdd1', 'request_id': 'req-a6ef586e-2bfe-42f4-8a57-261da947d8b6'} {{(pid=80245) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 08:34:35.255316 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.notifications [None req-a6ef586e-2bfe-42f4-8a57-261da947d8b6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '05b859e8bf4847eea99f02efae52cdd1', 'request_id': 'req-a6ef586e-2bfe-42f4-8a57-261da947d8b6'} {{(pid=80245) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 08:34:35.264726 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.notifications [None req-a6ef586e-2bfe-42f4-8a57-261da947d8b6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '05b859e8bf4847eea99f02efae52cdd1', 'request_id': 'req-a6ef586e-2bfe-42f4-8a57-261da947d8b6'} {{(pid=80245) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 08:34:35.286133 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.notifications [None req-a6ef586e-2bfe-42f4-8a57-261da947d8b6 admin admin] The token cache is being invalidate because project 05b859e8bf4847eea99f02efae52cdd1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80245) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 21 08:34:35.286529 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.notifications [None req-a6ef586e-2bfe-42f4-8a57-261da947d8b6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80245) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 08:34:35.287403 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.notifications [None req-a6ef586e-2bfe-42f4-8a57-261da947d8b6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80245) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 08:34:35.289774 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: [pid: 80245|app: 0|req: 1298/2587] 10.222.0.47 () {58 vars in 1232 bytes} [Thu Jul 21 08:34:35 2022] DELETE /identity/v3/projects/05b859e8bf4847eea99f02efae52cdd1 => generated 0 bytes in 132 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 08:34:36.915630 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62d25566-7e90-45f5-8e29-4d0911136b17 None None] REQUEST_METHOD: `POST` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:34:36.916087 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62d25566-7e90-45f5-8e29-4d0911136b17 None None] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:34:36.916197 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62d25566-7e90-45f5-8e29-4d0911136b17 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:34:36.934318 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.auth.core [None req-62d25566-7e90-45f5-8e29-4d0911136b17 None None] MFA Rules not processed for user `2ef7f1e91e554a1f88eb6a63e60a475f`. Rule list: `[]` (Enabled: `True`). {{(pid=80244) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 08:34:36.935552 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.fernet_utils [None req-62d25566-7e90-45f5-8e29-4d0911136b17 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80244) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 08:34:36.996917 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 1290/2588] 10.222.0.47 () {58 vars in 967 bytes} [Thu Jul 21 08:34:36 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 08:34:37.011576 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6ef586e-2bfe-42f4-8a57-261da947d8b6 admin admin] Authenticating user token {{(pid=80245) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:34:37.013528 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.fernet_utils [None req-a6ef586e-2bfe-42f4-8a57-261da947d8b6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80245) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 08:34:37.088891 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6ef586e-2bfe-42f4-8a57-261da947d8b6 admin admin] Validating token access rules against request {{(pid=80245) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:34:37.094258 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1a1fd55-c01c-4dba-adec-53674609e138 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '302351411fc84248a6b166b25d21609f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4faef13aa50e4e36b6c265fb1398fa14', 'project_domain_id': 'default', 'roles': ['service', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80245) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:34:37.094917 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1a1fd55-c01c-4dba-adec-53674609e138 service swift] REQUEST_METHOD: `GET` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:34:37.095199 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1a1fd55-c01c-4dba-adec-53674609e138 service swift] SCRIPT_NAME: `/identity` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:34:37.095506 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1a1fd55-c01c-4dba-adec-53674609e138 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:34:37.104176 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1a1fd55-c01c-4dba-adec-53674609e138 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:34:37.105751 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1a1fd55-c01c-4dba-adec-53674609e138 service swift] RBAC: Authorization granted {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:34:37.114793 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: [pid: 80245|app: 0|req: 1299/2589] 10.222.0.47 () {60 vars in 1384 bytes} [Thu Jul 21 08:34:37 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 104 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 21 08:34:46.990211 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG dbcounter [-] [80244] Writing DB stats keystone:SELECT=57 {{(pid=80244) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:34:47.111531 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG dbcounter [-] [80245] Writing DB stats keystone:SELECT=43,keystone:DELETE=4,keystone:INSERT=1 {{(pid=80245) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:34:53.138855 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62d25566-7e90-45f5-8e29-4d0911136b17 None None] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:34:53.140589 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.fernet_utils [None req-62d25566-7e90-45f5-8e29-4d0911136b17 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80244) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 08:34:53.207484 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62d25566-7e90-45f5-8e29-4d0911136b17 None None] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:34:53.212661 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3458277-1c68-4c09-84a5-b9a35f1468dc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '42d15fc0515945acacbbffcf2356c7a5', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:34:53.213540 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3458277-1c68-4c09-84a5-b9a35f1468dc admin admin] REQUEST_METHOD: `DELETE` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:34:53.213829 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3458277-1c68-4c09-84a5-b9a35f1468dc admin admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:34:53.214107 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3458277-1c68-4c09-84a5-b9a35f1468dc admin admin] PATH_INFO: `/v3/users/1f914243baf846f5b43a929332bcef6e` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:34:53.215344 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3458277-1c68-4c09-84a5-b9a35f1468dc admin admin] RBAC: Authorizing `identity:delete_user(user_id=1f914243baf846f5b43a929332bcef6e)` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:34:53.217379 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3458277-1c68-4c09-84a5-b9a35f1468dc admin admin] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:34:53.264042 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.notifications [None req-b3458277-1c68-4c09-84a5-b9a35f1468dc admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1f914243baf846f5b43a929332bcef6e'} {{(pid=80244) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 08:34:53.271530 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.notifications [None req-b3458277-1c68-4c09-84a5-b9a35f1468dc admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1f914243baf846f5b43a929332bcef6e'} {{(pid=80244) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 08:34:53.283857 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.notifications [None req-b3458277-1c68-4c09-84a5-b9a35f1468dc admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1f914243baf846f5b43a929332bcef6e'} {{(pid=80244) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 08:34:53.284501 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.notifications [None req-b3458277-1c68-4c09-84a5-b9a35f1468dc admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1f914243baf846f5b43a929332bcef6e'} {{(pid=80244) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 08:34:53.291341 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.notifications [None req-b3458277-1c68-4c09-84a5-b9a35f1468dc admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1f914243baf846f5b43a929332bcef6e'} {{(pid=80244) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 08:34:53.297223 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 1291/2590] 10.222.0.47 () {58 vars in 1223 bytes} [Thu Jul 21 08:34:53 2022] DELETE /identity/v3/users/1f914243baf846f5b43a929332bcef6e => generated 0 bytes in 159 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 08:34:53.561025 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1a1fd55-c01c-4dba-adec-53674609e138 service swift] Authenticating user token {{(pid=80245) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:34:53.583805 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1a1fd55-c01c-4dba-adec-53674609e138 service swift] Validating token access rules against request {{(pid=80245) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:34:53.650014 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c1c50c0-6363-4828-b8a1-231b34e3be0b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '42d15fc0515945acacbbffcf2356c7a5', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80245) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:34:53.650784 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c1c50c0-6363-4828-b8a1-231b34e3be0b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:34:53.651065 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c1c50c0-6363-4828-b8a1-231b34e3be0b admin admin] SCRIPT_NAME: `/identity` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:34:53.651381 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c1c50c0-6363-4828-b8a1-231b34e3be0b admin admin] PATH_INFO: `/v3/projects/419b6c6707544364b82f3f6fb5030032` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:34:53.652531 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c1c50c0-6363-4828-b8a1-231b34e3be0b admin admin] RBAC: Authorizing `identity:delete_project(project_id=419b6c6707544364b82f3f6fb5030032)` {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:34:53.654420 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c1c50c0-6363-4828-b8a1-231b34e3be0b admin admin] RBAC: Authorization granted {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:34:53.699860 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.notifications [None req-3c1c50c0-6363-4828-b8a1-231b34e3be0b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '419b6c6707544364b82f3f6fb5030032', 'request_id': 'req-3c1c50c0-6363-4828-b8a1-231b34e3be0b'} {{(pid=80245) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 08:34:53.705938 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.notifications [None req-3c1c50c0-6363-4828-b8a1-231b34e3be0b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '419b6c6707544364b82f3f6fb5030032', 'request_id': 'req-3c1c50c0-6363-4828-b8a1-231b34e3be0b'} {{(pid=80245) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 08:34:53.711414 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.notifications [None req-3c1c50c0-6363-4828-b8a1-231b34e3be0b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '419b6c6707544364b82f3f6fb5030032', 'request_id': 'req-3c1c50c0-6363-4828-b8a1-231b34e3be0b'} {{(pid=80245) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 08:34:53.724178 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.notifications [None req-3c1c50c0-6363-4828-b8a1-231b34e3be0b admin admin] The token cache is being invalidate because project 419b6c6707544364b82f3f6fb5030032 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80245) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 21 08:34:53.724469 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.notifications [None req-3c1c50c0-6363-4828-b8a1-231b34e3be0b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80245) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 08:34:53.725084 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.notifications [None req-3c1c50c0-6363-4828-b8a1-231b34e3be0b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80245) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 08:34:53.726648 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: [pid: 80245|app: 0|req: 1300/2591] 10.222.0.47 () {58 vars in 1232 bytes} [Thu Jul 21 08:34:53 2022] DELETE /identity/v3/projects/419b6c6707544364b82f3f6fb5030032 => generated 0 bytes in 166 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 08:34:53.731805 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-557f6943-a979-46b6-9085-e34bc210e4d7 None None] REQUEST_METHOD: `POST` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:34:53.732100 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-557f6943-a979-46b6-9085-e34bc210e4d7 None None] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:34:53.732381 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-557f6943-a979-46b6-9085-e34bc210e4d7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:34:53.750660 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.auth.core [None req-557f6943-a979-46b6-9085-e34bc210e4d7 None None] MFA Rules not processed for user `2dce40b8d74c41ec94cc36e6ff05d412`. Rule list: `[]` (Enabled: `True`). {{(pid=80244) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 08:34:53.751902 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.fernet_utils [None req-557f6943-a979-46b6-9085-e34bc210e4d7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80244) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 08:34:53.810512 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 1292/2592] 10.222.0.47 () {58 vars in 957 bytes} [Thu Jul 21 08:34:53 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 08:34:53.818144 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6daf8fb-0320-4ce4-933c-1fc12a445148 None None] REQUEST_METHOD: `POST` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:34:53.818450 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6daf8fb-0320-4ce4-933c-1fc12a445148 None None] SCRIPT_NAME: `/identity` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:34:53.819366 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6daf8fb-0320-4ce4-933c-1fc12a445148 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:34:53.837849 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.auth.core [None req-e6daf8fb-0320-4ce4-933c-1fc12a445148 None None] MFA Rules not processed for user `2dce40b8d74c41ec94cc36e6ff05d412`. Rule list: `[]` (Enabled: `True`). {{(pid=80245) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 08:34:53.839056 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.fernet_utils [None req-e6daf8fb-0320-4ce4-933c-1fc12a445148 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80245) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 08:34:53.846967 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: [pid: 80245|app: 0|req: 1301/2593] 10.222.0.47 () {58 vars in 957 bytes} [Thu Jul 21 08:34:53 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 08:34:53.851217 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-557f6943-a979-46b6-9085-e34bc210e4d7 None None] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:34:53.862028 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-557f6943-a979-46b6-9085-e34bc210e4d7 None None] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:34:53.867384 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1135bb4f-7e5b-4700-8aae-3058aded5821 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '42d15fc0515945acacbbffcf2356c7a5', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:34:53.868044 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1135bb4f-7e5b-4700-8aae-3058aded5821 admin admin] REQUEST_METHOD: `GET` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:34:53.868330 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1135bb4f-7e5b-4700-8aae-3058aded5821 admin admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:34:53.868609 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1135bb4f-7e5b-4700-8aae-3058aded5821 admin admin] PATH_INFO: `/v3/domains` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:34:53.869342 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1135bb4f-7e5b-4700-8aae-3058aded5821 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80244) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 08:34:53.869653 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1135bb4f-7e5b-4700-8aae-3058aded5821 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:34:53.871261 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1135bb4f-7e5b-4700-8aae-3058aded5821 admin admin] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:34:53.884750 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 1293/2594] 10.222.0.47 () {58 vars in 1152 bytes} [Thu Jul 21 08:34:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 08:34:53.888886 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6daf8fb-0320-4ce4-933c-1fc12a445148 None None] Authenticating user token {{(pid=80245) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:34:53.898121 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6daf8fb-0320-4ce4-933c-1fc12a445148 None None] Validating token access rules against request {{(pid=80245) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:34:53.904208 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b39e8108-b7b8-4c2c-aeba-398108bbac51 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '42d15fc0515945acacbbffcf2356c7a5', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80245) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:34:53.904873 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b39e8108-b7b8-4c2c-aeba-398108bbac51 admin admin] REQUEST_METHOD: `POST` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:34:53.905152 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b39e8108-b7b8-4c2c-aeba-398108bbac51 admin admin] SCRIPT_NAME: `/identity` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:34:53.905427 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b39e8108-b7b8-4c2c-aeba-398108bbac51 admin admin] PATH_INFO: `/v3/projects` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:34:53.906130 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b39e8108-b7b8-4c2c-aeba-398108bbac51 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:34:53.908059 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b39e8108-b7b8-4c2c-aeba-398108bbac51 admin admin] RBAC: Authorization granted {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:34:53.938262 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: [pid: 80245|app: 0|req: 1302/2595] 10.222.0.47 () {60 vars in 1152 bytes} [Thu Jul 21 08:34:53 2022] POST /identity/v3/projects => generated 391 bytes in 50 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 08:34:53.942227 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1135bb4f-7e5b-4700-8aae-3058aded5821 admin admin] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:34:53.955481 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1135bb4f-7e5b-4700-8aae-3058aded5821 admin admin] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:34:53.959133 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG dbcounter [-] [80244] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80244) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:34:54.014846 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfb0bfd6-10b9-4067-88d4-60051e608c0a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '42d15fc0515945acacbbffcf2356c7a5', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:34:54.015559 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfb0bfd6-10b9-4067-88d4-60051e608c0a admin admin] REQUEST_METHOD: `POST` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:34:54.015848 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfb0bfd6-10b9-4067-88d4-60051e608c0a admin admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:34:54.016126 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfb0bfd6-10b9-4067-88d4-60051e608c0a admin admin] PATH_INFO: `/v3/users` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:34:54.016829 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfb0bfd6-10b9-4067-88d4-60051e608c0a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:34:54.018757 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfb0bfd6-10b9-4067-88d4-60051e608c0a admin admin] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:34:54.038389 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 1294/2596] 10.222.0.47 () {60 vars in 1143 bytes} [Thu Jul 21 08:34:53 2022] POST /identity/v3/users => generated 296 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 08:34:54.042663 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b39e8108-b7b8-4c2c-aeba-398108bbac51 admin admin] Authenticating user token {{(pid=80245) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:34:54.052679 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b39e8108-b7b8-4c2c-aeba-398108bbac51 admin admin] Validating token access rules against request {{(pid=80245) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:34:54.057724 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b57f718-7fa6-4056-8c63-d9707cca8ea0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '42d15fc0515945acacbbffcf2356c7a5', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80245) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:34:54.058384 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b57f718-7fa6-4056-8c63-d9707cca8ea0 admin admin] REQUEST_METHOD: `GET` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:34:54.058664 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b57f718-7fa6-4056-8c63-d9707cca8ea0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:34:54.058941 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b57f718-7fa6-4056-8c63-d9707cca8ea0 admin admin] PATH_INFO: `/v3/roles` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:34:54.059701 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b57f718-7fa6-4056-8c63-d9707cca8ea0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:34:54.061413 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b57f718-7fa6-4056-8c63-d9707cca8ea0 admin admin] RBAC: Authorization granted {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:34:54.071959 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: [pid: 80245|app: 0|req: 1303/2597] 10.222.0.47 () {58 vars in 1121 bytes} [Thu Jul 21 08:34:54 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 08:34:54.076032 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfb0bfd6-10b9-4067-88d4-60051e608c0a admin admin] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:34:54.086035 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfb0bfd6-10b9-4067-88d4-60051e608c0a admin admin] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:34:54.090694 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62b8b0c5-8546-464a-bee9-41f938dc1c5f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '42d15fc0515945acacbbffcf2356c7a5', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:34:54.091393 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62b8b0c5-8546-464a-bee9-41f938dc1c5f admin admin] REQUEST_METHOD: `GET` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:34:54.091683 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62b8b0c5-8546-464a-bee9-41f938dc1c5f admin admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:34:54.091963 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62b8b0c5-8546-464a-bee9-41f938dc1c5f admin admin] PATH_INFO: `/v3/roles` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:34:54.092689 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62b8b0c5-8546-464a-bee9-41f938dc1c5f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:34:54.094311 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62b8b0c5-8546-464a-bee9-41f938dc1c5f admin admin] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:34:54.104870 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 1295/2598] 10.222.0.47 () {58 vars in 1121 bytes} [Thu Jul 21 08:34:54 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 08:34:54.108941 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b57f718-7fa6-4056-8c63-d9707cca8ea0 admin admin] Authenticating user token {{(pid=80245) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:34:54.119780 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b57f718-7fa6-4056-8c63-d9707cca8ea0 admin admin] Validating token access rules against request {{(pid=80245) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:34:54.124656 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bc64fa9-d09e-43db-8f5e-f3c0361a24f2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '42d15fc0515945acacbbffcf2356c7a5', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80245) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:34:54.125610 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bc64fa9-d09e-43db-8f5e-f3c0361a24f2 admin admin] REQUEST_METHOD: `PUT` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:34:54.125899 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bc64fa9-d09e-43db-8f5e-f3c0361a24f2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:34:54.126177 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bc64fa9-d09e-43db-8f5e-f3c0361a24f2 admin admin] PATH_INFO: `/v3/projects/586beb1749f4473ca3731389237fd38a/users/b955bbd533e5444ca88fb0b2f2143121/roles/3a074738221e4601b5205715ba87d240` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:34:54.141041 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bc64fa9-d09e-43db-8f5e-f3c0361a24f2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=586beb1749f4473ca3731389237fd38a, user_id=b955bbd533e5444ca88fb0b2f2143121, role_id=3a074738221e4601b5205715ba87d240)` {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:34:54.143733 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bc64fa9-d09e-43db-8f5e-f3c0361a24f2 admin admin] RBAC: Authorization granted {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:34:54.157539 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: [pid: 80245|app: 0|req: 1304/2599] 10.222.0.47 () {60 vars in 1482 bytes} [Thu Jul 21 08:34:54 2022] PUT /identity/v3/projects/586beb1749f4473ca3731389237fd38a/users/b955bbd533e5444ca88fb0b2f2143121/roles/3a074738221e4601b5205715ba87d240 => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 08:34:54.168654 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9121f9b-2b2d-4e44-afdb-a3e6bc672d5d None None] REQUEST_METHOD: `POST` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:34:54.169119 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9121f9b-2b2d-4e44-afdb-a3e6bc672d5d None None] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:34:54.169229 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9121f9b-2b2d-4e44-afdb-a3e6bc672d5d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:34:54.198742 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.auth.core [None req-f9121f9b-2b2d-4e44-afdb-a3e6bc672d5d None None] MFA Rules not processed for user `b955bbd533e5444ca88fb0b2f2143121`. Rule list: `[]` (Enabled: `True`). {{(pid=80244) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 08:34:54.200028 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.fernet_utils [None req-f9121f9b-2b2d-4e44-afdb-a3e6bc672d5d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80244) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 08:34:54.264907 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 1296/2600] 10.222.0.47 () {58 vars in 957 bytes} [Thu Jul 21 08:34:54 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 08:34:54.275821 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bc64fa9-d09e-43db-8f5e-f3c0361a24f2 admin admin] Authenticating user token {{(pid=80245) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:34:54.277537 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.fernet_utils [None req-1bc64fa9-d09e-43db-8f5e-f3c0361a24f2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80245) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 08:34:54.309013 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG dbcounter [-] [80245] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80245) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:34:54.337931 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bc64fa9-d09e-43db-8f5e-f3c0361a24f2 admin admin] Validating token access rules against request {{(pid=80245) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:34:54.341511 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a10c8b76-9d3e-4e20-b722-b447ca617b20 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4f40410b7674687ab5362ac982248ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4faef13aa50e4e36b6c265fb1398fa14', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80245) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:34:54.342157 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a10c8b76-9d3e-4e20-b722-b447ca617b20 service cinder] REQUEST_METHOD: `GET` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:34:54.342445 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a10c8b76-9d3e-4e20-b722-b447ca617b20 service cinder] SCRIPT_NAME: `/identity` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:34:54.342723 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a10c8b76-9d3e-4e20-b722-b447ca617b20 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:34:54.351605 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a10c8b76-9d3e-4e20-b722-b447ca617b20 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:34:54.353040 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a10c8b76-9d3e-4e20-b722-b447ca617b20 service cinder] RBAC: Authorization granted {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:34:54.361863 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: [pid: 80245|app: 0|req: 1305/2601] 10.222.0.47 () {60 vars in 1365 bytes} [Thu Jul 21 08:34:54 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 08:34:55.914529 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a438978-5f22-4b44-82a1-d29605322e4d None None] REQUEST_METHOD: `POST` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:34:55.915049 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a438978-5f22-4b44-82a1-d29605322e4d None None] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:34:55.915202 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a438978-5f22-4b44-82a1-d29605322e4d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:34:55.932862 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.auth.core [None req-5a438978-5f22-4b44-82a1-d29605322e4d None None] MFA Rules not processed for user `3528104f3e914d62a10e80e13d3c97de`. Rule list: `[]` (Enabled: `True`). {{(pid=80244) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 08:34:55.934055 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.fernet_utils [None req-5a438978-5f22-4b44-82a1-d29605322e4d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80244) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 08:34:55.989334 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 1297/2602] 10.222.0.47 () {58 vars in 957 bytes} [Thu Jul 21 08:34:55 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 77 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 21 08:35:04.357936 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG dbcounter [-] [80245] Writing DB stats keystone:SELECT=14 {{(pid=80245) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:35:05.982536 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG dbcounter [-] [80244] Writing DB stats keystone:SELECT=88,keystone:INSERT=3 {{(pid=80244) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:35:12.535538 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a10c8b76-9d3e-4e20-b722-b447ca617b20 service cinder] Authenticating user token {{(pid=80245) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:35:12.537582 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.fernet_utils [None req-a10c8b76-9d3e-4e20-b722-b447ca617b20 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80245) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 08:35:12.596074 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a10c8b76-9d3e-4e20-b722-b447ca617b20 service cinder] Validating token access rules against request {{(pid=80245) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:35:12.600329 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cd1572f-e7bd-4b0a-bf63-6ee15f35482f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20ca3c3bd3774691b3374cfbfec6e494', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4faef13aa50e4e36b6c265fb1398fa14', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80245) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:35:12.601009 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cd1572f-e7bd-4b0a-bf63-6ee15f35482f service neutron] REQUEST_METHOD: `GET` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:35:12.601291 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cd1572f-e7bd-4b0a-bf63-6ee15f35482f service neutron] SCRIPT_NAME: `/identity` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:35:12.601566 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cd1572f-e7bd-4b0a-bf63-6ee15f35482f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:35:12.603381 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.fernet_utils [None req-9cd1572f-e7bd-4b0a-bf63-6ee15f35482f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80245) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 08:35:12.638531 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a438978-5f22-4b44-82a1-d29605322e4d None None] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:35:12.679358 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cd1572f-e7bd-4b0a-bf63-6ee15f35482f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:35:12.680819 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cd1572f-e7bd-4b0a-bf63-6ee15f35482f service neutron] RBAC: Authorization granted {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:35:12.692063 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: [pid: 80245|app: 0|req: 1306/2603] 10.222.0.47 () {60 vars in 1365 bytes} [Thu Jul 21 08:35:12 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 157 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 08:35:12.705569 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a438978-5f22-4b44-82a1-d29605322e4d None None] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:35:12.711128 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ac94bfc-b43e-4bf5-b436-2fdbf74fda95 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '42d15fc0515945acacbbffcf2356c7a5', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:35:12.712042 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ac94bfc-b43e-4bf5-b436-2fdbf74fda95 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:35:12.712328 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ac94bfc-b43e-4bf5-b436-2fdbf74fda95 admin admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:35:12.712606 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ac94bfc-b43e-4bf5-b436-2fdbf74fda95 admin admin] PATH_INFO: `/v3/users/b955bbd533e5444ca88fb0b2f2143121` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:35:12.713802 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ac94bfc-b43e-4bf5-b436-2fdbf74fda95 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b955bbd533e5444ca88fb0b2f2143121)` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:35:12.715764 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ac94bfc-b43e-4bf5-b436-2fdbf74fda95 admin admin] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:35:12.767301 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.notifications [None req-1ac94bfc-b43e-4bf5-b436-2fdbf74fda95 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b955bbd533e5444ca88fb0b2f2143121'} {{(pid=80244) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 08:35:12.776614 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.notifications [None req-1ac94bfc-b43e-4bf5-b436-2fdbf74fda95 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b955bbd533e5444ca88fb0b2f2143121'} {{(pid=80244) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 08:35:12.790116 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.notifications [None req-1ac94bfc-b43e-4bf5-b436-2fdbf74fda95 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b955bbd533e5444ca88fb0b2f2143121'} {{(pid=80244) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 08:35:12.790797 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.notifications [None req-1ac94bfc-b43e-4bf5-b436-2fdbf74fda95 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b955bbd533e5444ca88fb0b2f2143121'} {{(pid=80244) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 08:35:12.797832 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.notifications [None req-1ac94bfc-b43e-4bf5-b436-2fdbf74fda95 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b955bbd533e5444ca88fb0b2f2143121'} {{(pid=80244) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 08:35:12.804344 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 1298/2604] 10.222.0.47 () {58 vars in 1223 bytes} [Thu Jul 21 08:35:12 2022] DELETE /identity/v3/users/b955bbd533e5444ca88fb0b2f2143121 => generated 0 bytes in 166 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 08:35:12.808627 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-572946b2-0cef-4b22-9f85-f8e34afd1cac None None] REQUEST_METHOD: `POST` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:35:12.808939 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-572946b2-0cef-4b22-9f85-f8e34afd1cac None None] SCRIPT_NAME: `/identity` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:35:12.809209 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-572946b2-0cef-4b22-9f85-f8e34afd1cac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:35:12.826756 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.auth.core [None req-572946b2-0cef-4b22-9f85-f8e34afd1cac None None] MFA Rules not processed for user `2dce40b8d74c41ec94cc36e6ff05d412`. Rule list: `[]` (Enabled: `True`). {{(pid=80245) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 08:35:12.827995 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.fernet_utils [None req-572946b2-0cef-4b22-9f85-f8e34afd1cac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80245) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 08:35:12.894205 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: [pid: 80245|app: 0|req: 1307/2605] 10.222.0.47 () {58 vars in 957 bytes} [Thu Jul 21 08:35:12 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 08:35:12.904430 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ac94bfc-b43e-4bf5-b436-2fdbf74fda95 admin admin] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:35:12.905885 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.fernet_utils [None req-1ac94bfc-b43e-4bf5-b436-2fdbf74fda95 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80244) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 08:35:12.964736 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ac94bfc-b43e-4bf5-b436-2fdbf74fda95 admin admin] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:35:12.968966 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4b9402a-b106-458a-9d6a-4b63b3b48dd1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20ca3c3bd3774691b3374cfbfec6e494', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4faef13aa50e4e36b6c265fb1398fa14', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:35:12.969623 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4b9402a-b106-458a-9d6a-4b63b3b48dd1 service neutron] REQUEST_METHOD: `GET` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:35:12.969907 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4b9402a-b106-458a-9d6a-4b63b3b48dd1 service neutron] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:35:12.970179 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4b9402a-b106-458a-9d6a-4b63b3b48dd1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:35:12.979543 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4b9402a-b106-458a-9d6a-4b63b3b48dd1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:35:12.980976 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4b9402a-b106-458a-9d6a-4b63b3b48dd1 service neutron] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:35:12.990786 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 1299/2606] 10.222.0.47 () {60 vars in 1365 bytes} [Thu Jul 21 08:35:12 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 08:35:13.258509 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-572946b2-0cef-4b22-9f85-f8e34afd1cac None None] Authenticating user token {{(pid=80245) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:35:13.259918 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.fernet_utils [None req-572946b2-0cef-4b22-9f85-f8e34afd1cac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80245) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 08:35:13.272801 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-572946b2-0cef-4b22-9f85-f8e34afd1cac None None] Validating token access rules against request {{(pid=80245) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:35:13.277448 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7ca8791-5ecf-4739-9189-a7fff2251151 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '42d15fc0515945acacbbffcf2356c7a5', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80245) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:35:13.278320 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7ca8791-5ecf-4739-9189-a7fff2251151 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:35:13.278744 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7ca8791-5ecf-4739-9189-a7fff2251151 admin admin] SCRIPT_NAME: `/identity` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:35:13.279027 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7ca8791-5ecf-4739-9189-a7fff2251151 admin admin] PATH_INFO: `/v3/projects/586beb1749f4473ca3731389237fd38a` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:35:13.280248 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7ca8791-5ecf-4739-9189-a7fff2251151 admin admin] RBAC: Authorizing `identity:delete_project(project_id=586beb1749f4473ca3731389237fd38a)` {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:35:13.282177 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7ca8791-5ecf-4739-9189-a7fff2251151 admin admin] RBAC: Authorization granted {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:35:13.323301 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG dbcounter [-] [80245] Writing DB stats keystone:SELECT=97,keystone:DELETE=3 {{(pid=80245) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:35:13.333486 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.notifications [None req-d7ca8791-5ecf-4739-9189-a7fff2251151 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '586beb1749f4473ca3731389237fd38a', 'request_id': 'req-d7ca8791-5ecf-4739-9189-a7fff2251151'} {{(pid=80245) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 08:35:13.339548 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.notifications [None req-d7ca8791-5ecf-4739-9189-a7fff2251151 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '586beb1749f4473ca3731389237fd38a', 'request_id': 'req-d7ca8791-5ecf-4739-9189-a7fff2251151'} {{(pid=80245) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 08:35:13.345183 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.notifications [None req-d7ca8791-5ecf-4739-9189-a7fff2251151 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '586beb1749f4473ca3731389237fd38a', 'request_id': 'req-d7ca8791-5ecf-4739-9189-a7fff2251151'} {{(pid=80245) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 08:35:13.357082 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.notifications [None req-d7ca8791-5ecf-4739-9189-a7fff2251151 admin admin] The token cache is being invalidate because project 586beb1749f4473ca3731389237fd38a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80245) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 21 08:35:13.357372 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.notifications [None req-d7ca8791-5ecf-4739-9189-a7fff2251151 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80245) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 08:35:13.357974 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.notifications [None req-d7ca8791-5ecf-4739-9189-a7fff2251151 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80245) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 08:35:13.359637 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: [pid: 80245|app: 0|req: 1308/2607] 10.222.0.47 () {58 vars in 1232 bytes} [Thu Jul 21 08:35:13 2022] DELETE /identity/v3/projects/586beb1749f4473ca3731389237fd38a => generated 0 bytes in 102 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 21 08:35:16.252637 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4b9402a-b106-458a-9d6a-4b63b3b48dd1 service neutron] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:35:16.254534 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.fernet_utils [None req-e4b9402a-b106-458a-9d6a-4b63b3b48dd1 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80244) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 08:35:16.266959 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4b9402a-b106-458a-9d6a-4b63b3b48dd1 service neutron] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:35:16.284869 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG dbcounter [-] [80244] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80244) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:35:16.317319 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5d644f9-42d1-4811-923e-ff1243294568 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20ca3c3bd3774691b3374cfbfec6e494', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4faef13aa50e4e36b6c265fb1398fa14', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:35:16.318527 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5d644f9-42d1-4811-923e-ff1243294568 service neutron] REQUEST_METHOD: `GET` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:35:16.318836 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5d644f9-42d1-4811-923e-ff1243294568 service neutron] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:35:16.319095 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5d644f9-42d1-4811-923e-ff1243294568 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:35:16.320945 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.fernet_utils [None req-b5d644f9-42d1-4811-923e-ff1243294568 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80244) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 08:35:16.390311 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5d644f9-42d1-4811-923e-ff1243294568 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:35:16.391800 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5d644f9-42d1-4811-923e-ff1243294568 service neutron] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:35:16.402752 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 1300/2608] 10.222.0.47 () {60 vars in 1365 bytes} [Thu Jul 21 08:35:16 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 151 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 08:35:23.354027 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG dbcounter [-] [80245] Writing DB stats keystone:SELECT=11,keystone:DELETE=4,keystone:INSERT=1 {{(pid=80245) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:35:26.397964 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG dbcounter [-] [80244] Writing DB stats keystone:SELECT=44 {{(pid=80244) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:35:27.641377 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7ca8791-5ecf-4739-9189-a7fff2251151 admin admin] Authenticating user token {{(pid=80245) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:35:27.643016 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.fernet_utils [None req-d7ca8791-5ecf-4739-9189-a7fff2251151 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80245) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 08:35:27.659678 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7ca8791-5ecf-4739-9189-a7fff2251151 admin admin] Validating token access rules against request {{(pid=80245) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:35:27.665074 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-138c8531-98ce-4f4a-9afd-4b63efd888b7 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2b5b99a36442148fe743269054e991', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4faef13aa50e4e36b6c265fb1398fa14', 'project_domain_id': 'default', 'roles': ['member', 'service', 'ResellerAdmin', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80245) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:35:27.665830 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-138c8531-98ce-4f4a-9afd-4b63efd888b7 service nova] REQUEST_METHOD: `GET` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:35:27.666123 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-138c8531-98ce-4f4a-9afd-4b63efd888b7 service nova] SCRIPT_NAME: `/identity` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:35:27.666390 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-138c8531-98ce-4f4a-9afd-4b63efd888b7 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:35:27.668183 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.fernet_utils [None req-138c8531-98ce-4f4a-9afd-4b63efd888b7 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80245) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 08:35:27.728554 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-138c8531-98ce-4f4a-9afd-4b63efd888b7 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:35:27.730157 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-138c8531-98ce-4f4a-9afd-4b63efd888b7 service nova] RBAC: Authorization granted {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:35:27.739987 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: [pid: 80245|app: 0|req: 1309/2609] 10.222.0.47 () {60 vars in 1365 bytes} [Thu Jul 21 08:35:27 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 08:35:36.139276 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-357b018c-7aed-469c-aff6-2b79a50f9dbe None None] REQUEST_METHOD: `POST` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:35:36.139806 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-357b018c-7aed-469c-aff6-2b79a50f9dbe None None] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:35:36.139906 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-357b018c-7aed-469c-aff6-2b79a50f9dbe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:35:36.157296 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.auth.core [None req-357b018c-7aed-469c-aff6-2b79a50f9dbe None None] MFA Rules not processed for user `ad56309db7044cb2a86651d4149cd090`. Rule list: `[]` (Enabled: `True`). {{(pid=80244) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 08:35:36.158487 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.fernet_utils [None req-357b018c-7aed-469c-aff6-2b79a50f9dbe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80244) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 08:35:36.165315 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 1301/2610] 10.222.0.47 () {58 vars in 957 bytes} [Thu Jul 21 08:35:36 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 27 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 08:35:36.476620 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-138c8531-98ce-4f4a-9afd-4b63efd888b7 service nova] Authenticating user token {{(pid=80245) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:35:36.478166 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.fernet_utils [None req-138c8531-98ce-4f4a-9afd-4b63efd888b7 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80245) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 08:35:36.490537 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-138c8531-98ce-4f4a-9afd-4b63efd888b7 service nova] Validating token access rules against request {{(pid=80245) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:35:36.494369 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b29705f2-41e0-4de0-ad17-b5ed170c07c4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20ca3c3bd3774691b3374cfbfec6e494', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4faef13aa50e4e36b6c265fb1398fa14', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80245) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:35:36.495026 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b29705f2-41e0-4de0-ad17-b5ed170c07c4 service neutron] REQUEST_METHOD: `GET` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:35:36.495342 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b29705f2-41e0-4de0-ad17-b5ed170c07c4 service neutron] SCRIPT_NAME: `/identity` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:35:36.495618 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b29705f2-41e0-4de0-ad17-b5ed170c07c4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:35:36.497402 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.fernet_utils [None req-b29705f2-41e0-4de0-ad17-b5ed170c07c4 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80245) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 08:35:36.507645 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b29705f2-41e0-4de0-ad17-b5ed170c07c4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:35:36.509077 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b29705f2-41e0-4de0-ad17-b5ed170c07c4 service neutron] RBAC: Authorization granted {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:35:36.518923 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: [pid: 80245|app: 0|req: 1310/2611] 10.222.0.47 () {60 vars in 1365 bytes} [Thu Jul 21 08:35:36 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 08:35:36.836535 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-357b018c-7aed-469c-aff6-2b79a50f9dbe None None] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:35:36.838686 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.fernet_utils [None req-357b018c-7aed-469c-aff6-2b79a50f9dbe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80244) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 08:35:36.921724 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-357b018c-7aed-469c-aff6-2b79a50f9dbe None None] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:35:36.927593 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ece45475-78aa-4dca-bc53-0119744030cb service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3379b32543474d0a8e897484e85c7d04', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4faef13aa50e4e36b6c265fb1398fa14', 'project_domain_id': 'default', 'roles': ['service', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:35:36.928258 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ece45475-78aa-4dca-bc53-0119744030cb service placement] REQUEST_METHOD: `GET` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:35:36.928543 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ece45475-78aa-4dca-bc53-0119744030cb service placement] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:35:36.928822 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ece45475-78aa-4dca-bc53-0119744030cb service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:35:36.930661 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.fernet_utils [None req-ece45475-78aa-4dca-bc53-0119744030cb service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80244) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 08:35:36.944864 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ece45475-78aa-4dca-bc53-0119744030cb service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:35:36.946484 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ece45475-78aa-4dca-bc53-0119744030cb service placement] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:35:36.957098 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 1302/2612] 10.222.0.47 () {60 vars in 1365 bytes} [Thu Jul 21 08:35:36 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 121 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 08:35:39.522176 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b29705f2-41e0-4de0-ad17-b5ed170c07c4 service neutron] Authenticating user token {{(pid=80245) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:35:39.523623 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.fernet_utils [None req-b29705f2-41e0-4de0-ad17-b5ed170c07c4 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80245) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 08:35:39.534402 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b29705f2-41e0-4de0-ad17-b5ed170c07c4 service neutron] Validating token access rules against request {{(pid=80245) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:35:39.538053 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b60221bb-20d2-47ed-bc45-2dc7c1c2e4f3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20ca3c3bd3774691b3374cfbfec6e494', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4faef13aa50e4e36b6c265fb1398fa14', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80245) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:35:39.538685 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b60221bb-20d2-47ed-bc45-2dc7c1c2e4f3 service neutron] REQUEST_METHOD: `GET` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:35:39.538966 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b60221bb-20d2-47ed-bc45-2dc7c1c2e4f3 service neutron] SCRIPT_NAME: `/identity` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:35:39.539239 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b60221bb-20d2-47ed-bc45-2dc7c1c2e4f3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:35:39.541672 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.fernet_utils [None req-b60221bb-20d2-47ed-bc45-2dc7c1c2e4f3 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80245) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 08:35:39.551310 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b60221bb-20d2-47ed-bc45-2dc7c1c2e4f3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:35:39.552821 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b60221bb-20d2-47ed-bc45-2dc7c1c2e4f3 service neutron] RBAC: Authorization granted {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:35:39.561532 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: [pid: 80245|app: 0|req: 1311/2613] 10.222.0.47 () {60 vars in 1365 bytes} [Thu Jul 21 08:35:39 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 08:35:46.952876 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG dbcounter [-] [80244] Writing DB stats keystone:SELECT=35 {{(pid=80244) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:35:47.290658 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ece45475-78aa-4dca-bc53-0119744030cb service placement] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:35:47.303434 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ece45475-78aa-4dca-bc53-0119744030cb service placement] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:35:47.309298 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7302b058-171b-4ad3-be93-59bff53450d9 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2b5b99a36442148fe743269054e991', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4faef13aa50e4e36b6c265fb1398fa14', 'project_domain_id': 'default', 'roles': ['member', 'service', 'ResellerAdmin', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:35:47.309983 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7302b058-171b-4ad3-be93-59bff53450d9 service nova] REQUEST_METHOD: `GET` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:35:47.310267 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7302b058-171b-4ad3-be93-59bff53450d9 service nova] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:35:47.310545 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7302b058-171b-4ad3-be93-59bff53450d9 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:35:47.312405 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.fernet_utils [None req-7302b058-171b-4ad3-be93-59bff53450d9 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80244) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 08:35:47.325665 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7302b058-171b-4ad3-be93-59bff53450d9 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:35:47.327344 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7302b058-171b-4ad3-be93-59bff53450d9 service nova] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:35:47.339082 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 1303/2614] 10.222.0.47 () {60 vars in 1365 bytes} [Thu Jul 21 08:35:47 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 49 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 08:35:49.557497 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG dbcounter [-] [80245] Writing DB stats keystone:SELECT=40 {{(pid=80245) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:35:50.257605 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b60221bb-20d2-47ed-bc45-2dc7c1c2e4f3 service neutron] Authenticating user token {{(pid=80245) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:35:50.258993 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.fernet_utils [None req-b60221bb-20d2-47ed-bc45-2dc7c1c2e4f3 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80245) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 08:35:50.315567 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b60221bb-20d2-47ed-bc45-2dc7c1c2e4f3 service neutron] Validating token access rules against request {{(pid=80245) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:35:50.319142 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-624d76d2-a220-4ddb-a96c-a263cb8954b9 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4f40410b7674687ab5362ac982248ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4faef13aa50e4e36b6c265fb1398fa14', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80245) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:35:50.319898 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-624d76d2-a220-4ddb-a96c-a263cb8954b9 service cinder] REQUEST_METHOD: `GET` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:35:50.320188 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-624d76d2-a220-4ddb-a96c-a263cb8954b9 service cinder] SCRIPT_NAME: `/identity` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:35:50.320472 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-624d76d2-a220-4ddb-a96c-a263cb8954b9 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:35:50.322767 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.fernet_utils [None req-624d76d2-a220-4ddb-a96c-a263cb8954b9 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80245) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 08:35:50.379923 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-624d76d2-a220-4ddb-a96c-a263cb8954b9 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:35:50.381365 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-624d76d2-a220-4ddb-a96c-a263cb8954b9 service cinder] RBAC: Authorization granted {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:35:50.391835 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: [pid: 80245|app: 0|req: 1312/2615] 10.222.0.47 () {60 vars in 1365 bytes} [Thu Jul 21 08:35:50 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 135 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 08:35:56.568808 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7302b058-171b-4ad3-be93-59bff53450d9 service nova] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:35:56.570743 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.fernet_utils [None req-7302b058-171b-4ad3-be93-59bff53450d9 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80244) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 08:35:56.586221 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7302b058-171b-4ad3-be93-59bff53450d9 service nova] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:35:56.592513 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e4f8782-2854-4462-84e9-249a20c4d4b1 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2b5b99a36442148fe743269054e991', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4faef13aa50e4e36b6c265fb1398fa14', 'project_domain_id': 'default', 'roles': ['member', 'service', 'ResellerAdmin', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:35:56.593186 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e4f8782-2854-4462-84e9-249a20c4d4b1 service nova] REQUEST_METHOD: `GET` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:35:56.593470 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e4f8782-2854-4462-84e9-249a20c4d4b1 service nova] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:35:56.593735 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e4f8782-2854-4462-84e9-249a20c4d4b1 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:35:56.595627 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.fernet_utils [None req-8e4f8782-2854-4462-84e9-249a20c4d4b1 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80244) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 08:35:56.609849 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e4f8782-2854-4462-84e9-249a20c4d4b1 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:35:56.612295 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e4f8782-2854-4462-84e9-249a20c4d4b1 service nova] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:35:56.624949 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 1304/2616] 10.222.0.47 () {60 vars in 1365 bytes} [Thu Jul 21 08:35:56 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 57 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 08:36:00.387466 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG dbcounter [-] [80245] Writing DB stats keystone:SELECT=48 {{(pid=80245) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:36:06.619884 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG dbcounter [-] [80244] Writing DB stats keystone:SELECT=12 {{(pid=80244) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:36:11.579850 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-624d76d2-a220-4ddb-a96c-a263cb8954b9 service cinder] Authenticating user token {{(pid=80245) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:36:11.588624 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-624d76d2-a220-4ddb-a96c-a263cb8954b9 service cinder] Validating token access rules against request {{(pid=80245) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:36:11.592520 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f4dfa5c-bccd-4e94-ac37-6b373f075998 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20ca3c3bd3774691b3374cfbfec6e494', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4faef13aa50e4e36b6c265fb1398fa14', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80245) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:36:11.593174 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f4dfa5c-bccd-4e94-ac37-6b373f075998 service neutron] REQUEST_METHOD: `GET` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:36:11.593460 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f4dfa5c-bccd-4e94-ac37-6b373f075998 service neutron] SCRIPT_NAME: `/identity` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:36:11.593732 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f4dfa5c-bccd-4e94-ac37-6b373f075998 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:36:11.595561 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.fernet_utils [None req-1f4dfa5c-bccd-4e94-ac37-6b373f075998 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80245) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 08:36:11.637533 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f35f97ba-300d-4589-b2ce-334e22cf08e4 None None] REQUEST_METHOD: `POST` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:36:11.638016 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f35f97ba-300d-4589-b2ce-334e22cf08e4 None None] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:36:11.638125 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f35f97ba-300d-4589-b2ce-334e22cf08e4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:36:11.656722 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.auth.core [None req-f35f97ba-300d-4589-b2ce-334e22cf08e4 None None] MFA Rules not processed for user `2ef7f1e91e554a1f88eb6a63e60a475f`. Rule list: `[]` (Enabled: `True`). {{(pid=80244) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 08:36:11.658000 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.fernet_utils [None req-f35f97ba-300d-4589-b2ce-334e22cf08e4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80244) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 08:36:11.666365 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f4dfa5c-bccd-4e94-ac37-6b373f075998 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:36:11.667920 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f4dfa5c-bccd-4e94-ac37-6b373f075998 service neutron] RBAC: Authorization granted {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:36:11.677644 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: [pid: 80245|app: 0|req: 1313/2617] 10.222.0.47 () {60 vars in 1365 bytes} [Thu Jul 21 08:36:11 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 100 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 08:36:11.836536 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 1305/2618] 10.222.0.47 () {58 vars in 967 bytes} [Thu Jul 21 08:36:11 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 200 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 21 08:36:11.888554 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f4dfa5c-bccd-4e94-ac37-6b373f075998 service neutron] Authenticating user token {{(pid=80245) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:36:11.889976 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.fernet_utils [None req-1f4dfa5c-bccd-4e94-ac37-6b373f075998 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80245) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 08:36:11.958592 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f4dfa5c-bccd-4e94-ac37-6b373f075998 service neutron] Validating token access rules against request {{(pid=80245) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:36:11.963981 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7b4355b-e410-4fa3-8eba-bf77b868ec3f service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '302351411fc84248a6b166b25d21609f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4faef13aa50e4e36b6c265fb1398fa14', 'project_domain_id': 'default', 'roles': ['service', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80245) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:36:11.964649 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7b4355b-e410-4fa3-8eba-bf77b868ec3f service swift] REQUEST_METHOD: `GET` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:36:11.964938 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7b4355b-e410-4fa3-8eba-bf77b868ec3f service swift] SCRIPT_NAME: `/identity` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:36:11.965223 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7b4355b-e410-4fa3-8eba-bf77b868ec3f service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:36:11.974834 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7b4355b-e410-4fa3-8eba-bf77b868ec3f service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:36:11.976142 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7b4355b-e410-4fa3-8eba-bf77b868ec3f service swift] RBAC: Authorization granted {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:36:11.984904 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: [pid: 80245|app: 0|req: 1314/2619] 10.222.0.47 () {60 vars in 1384 bytes} [Thu Jul 21 08:36:11 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 21 08:36:15.230034 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f35f97ba-300d-4589-b2ce-334e22cf08e4 None None] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:36:15.231742 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.fernet_utils [None req-f35f97ba-300d-4589-b2ce-334e22cf08e4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80244) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 08:36:15.246578 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f35f97ba-300d-4589-b2ce-334e22cf08e4 None None] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:36:15.251546 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a81e463c-d069-4340-adc7-4f7c663693ac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '42d15fc0515945acacbbffcf2356c7a5', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:36:15.252506 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a81e463c-d069-4340-adc7-4f7c663693ac admin admin] REQUEST_METHOD: `DELETE` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:36:15.252849 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a81e463c-d069-4340-adc7-4f7c663693ac admin admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:36:15.253448 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a81e463c-d069-4340-adc7-4f7c663693ac admin admin] PATH_INFO: `/v3/users/3528104f3e914d62a10e80e13d3c97de` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:36:15.254696 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a81e463c-d069-4340-adc7-4f7c663693ac admin admin] RBAC: Authorizing `identity:delete_user(user_id=3528104f3e914d62a10e80e13d3c97de)` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:36:15.256637 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a81e463c-d069-4340-adc7-4f7c663693ac admin admin] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:36:15.306972 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.notifications [None req-a81e463c-d069-4340-adc7-4f7c663693ac admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3528104f3e914d62a10e80e13d3c97de'} {{(pid=80244) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 08:36:15.313855 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.notifications [None req-a81e463c-d069-4340-adc7-4f7c663693ac admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3528104f3e914d62a10e80e13d3c97de'} {{(pid=80244) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 08:36:15.328369 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.notifications [None req-a81e463c-d069-4340-adc7-4f7c663693ac admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3528104f3e914d62a10e80e13d3c97de'} {{(pid=80244) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 08:36:15.329923 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.notifications [None req-a81e463c-d069-4340-adc7-4f7c663693ac admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3528104f3e914d62a10e80e13d3c97de'} {{(pid=80244) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 08:36:15.337143 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.notifications [None req-a81e463c-d069-4340-adc7-4f7c663693ac admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3528104f3e914d62a10e80e13d3c97de'} {{(pid=80244) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 08:36:15.343197 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 1306/2620] 10.222.0.47 () {58 vars in 1223 bytes} [Thu Jul 21 08:36:15 2022] DELETE /identity/v3/users/3528104f3e914d62a10e80e13d3c97de => generated 0 bytes in 114 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 08:36:15.576208 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7b4355b-e410-4fa3-8eba-bf77b868ec3f service swift] Authenticating user token {{(pid=80245) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:36:15.588206 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7b4355b-e410-4fa3-8eba-bf77b868ec3f service swift] Validating token access rules against request {{(pid=80245) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:36:15.653740 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ccd6db3-194b-4556-a2ae-32b99af2eb13 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '42d15fc0515945acacbbffcf2356c7a5', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80245) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:36:15.654605 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ccd6db3-194b-4556-a2ae-32b99af2eb13 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:36:15.654886 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ccd6db3-194b-4556-a2ae-32b99af2eb13 admin admin] SCRIPT_NAME: `/identity` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:36:15.655226 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ccd6db3-194b-4556-a2ae-32b99af2eb13 admin admin] PATH_INFO: `/v3/projects/b27a1d777abf4d139048716280a4bb4f` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:36:15.656433 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ccd6db3-194b-4556-a2ae-32b99af2eb13 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b27a1d777abf4d139048716280a4bb4f)` {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:36:15.658328 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ccd6db3-194b-4556-a2ae-32b99af2eb13 admin admin] RBAC: Authorization granted {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:36:15.689896 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG dbcounter [-] [80245] Writing DB stats keystone:SELECT=99,keystone:DELETE=1 {{(pid=80245) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:36:15.707387 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.notifications [None req-3ccd6db3-194b-4556-a2ae-32b99af2eb13 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b27a1d777abf4d139048716280a4bb4f', 'request_id': 'req-3ccd6db3-194b-4556-a2ae-32b99af2eb13'} {{(pid=80245) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 08:36:15.713818 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.notifications [None req-3ccd6db3-194b-4556-a2ae-32b99af2eb13 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b27a1d777abf4d139048716280a4bb4f', 'request_id': 'req-3ccd6db3-194b-4556-a2ae-32b99af2eb13'} {{(pid=80245) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 08:36:15.719955 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.notifications [None req-3ccd6db3-194b-4556-a2ae-32b99af2eb13 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b27a1d777abf4d139048716280a4bb4f', 'request_id': 'req-3ccd6db3-194b-4556-a2ae-32b99af2eb13'} {{(pid=80245) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 08:36:15.732719 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.notifications [None req-3ccd6db3-194b-4556-a2ae-32b99af2eb13 admin admin] The token cache is being invalidate because project b27a1d777abf4d139048716280a4bb4f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80245) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 21 08:36:15.733092 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.notifications [None req-3ccd6db3-194b-4556-a2ae-32b99af2eb13 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80245) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 08:36:15.733781 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.notifications [None req-3ccd6db3-194b-4556-a2ae-32b99af2eb13 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80245) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 08:36:15.735406 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: [pid: 80245|app: 0|req: 1315/2621] 10.222.0.47 () {58 vars in 1232 bytes} [Thu Jul 21 08:36:15 2022] DELETE /identity/v3/projects/b27a1d777abf4d139048716280a4bb4f => generated 0 bytes in 160 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 08:36:15.740385 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f362e1ca-f950-42e5-881e-1917adad2420 None None] REQUEST_METHOD: `POST` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:36:15.741310 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f362e1ca-f950-42e5-881e-1917adad2420 None None] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:36:15.741660 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f362e1ca-f950-42e5-881e-1917adad2420 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:36:15.761837 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.auth.core [None req-f362e1ca-f950-42e5-881e-1917adad2420 None None] MFA Rules not processed for user `2dce40b8d74c41ec94cc36e6ff05d412`. Rule list: `[]` (Enabled: `True`). {{(pid=80244) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 08:36:15.763053 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.fernet_utils [None req-f362e1ca-f950-42e5-881e-1917adad2420 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80244) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 08:36:15.834627 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 1307/2622] 10.222.0.47 () {58 vars in 957 bytes} [Thu Jul 21 08:36:15 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 08:36:15.842796 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88f49c17-0583-488e-8b55-5916a194dd2a None None] REQUEST_METHOD: `POST` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:36:15.843164 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88f49c17-0583-488e-8b55-5916a194dd2a None None] SCRIPT_NAME: `/identity` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:36:15.843531 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88f49c17-0583-488e-8b55-5916a194dd2a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:36:15.867585 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.auth.core [None req-88f49c17-0583-488e-8b55-5916a194dd2a None None] MFA Rules not processed for user `2dce40b8d74c41ec94cc36e6ff05d412`. Rule list: `[]` (Enabled: `True`). {{(pid=80245) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 08:36:15.869388 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.fernet_utils [None req-88f49c17-0583-488e-8b55-5916a194dd2a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80245) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 08:36:15.879620 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: [pid: 80245|app: 0|req: 1316/2623] 10.222.0.47 () {58 vars in 957 bytes} [Thu Jul 21 08:36:15 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 08:36:15.883859 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f362e1ca-f950-42e5-881e-1917adad2420 None None] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:36:15.904706 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f362e1ca-f950-42e5-881e-1917adad2420 None None] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:36:15.921519 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb0243ce-dce9-4dde-9f1d-3366ea693b27 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '42d15fc0515945acacbbffcf2356c7a5', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:36:15.922224 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb0243ce-dce9-4dde-9f1d-3366ea693b27 admin admin] REQUEST_METHOD: `GET` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:36:15.922513 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb0243ce-dce9-4dde-9f1d-3366ea693b27 admin admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:36:15.922784 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb0243ce-dce9-4dde-9f1d-3366ea693b27 admin admin] PATH_INFO: `/v3/domains` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:36:15.923585 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb0243ce-dce9-4dde-9f1d-3366ea693b27 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80244) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 08:36:15.923896 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb0243ce-dce9-4dde-9f1d-3366ea693b27 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:36:15.925543 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb0243ce-dce9-4dde-9f1d-3366ea693b27 admin admin] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:36:15.941794 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 1308/2624] 10.222.0.47 () {58 vars in 1152 bytes} [Thu Jul 21 08:36:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 58 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 08:36:15.942600 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88f49c17-0583-488e-8b55-5916a194dd2a None None] Authenticating user token {{(pid=80245) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:36:15.955927 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88f49c17-0583-488e-8b55-5916a194dd2a None None] Validating token access rules against request {{(pid=80245) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:36:15.968197 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68404822-5e0d-4b54-a8fa-55ea2c35695e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '42d15fc0515945acacbbffcf2356c7a5', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80245) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:36:15.968614 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68404822-5e0d-4b54-a8fa-55ea2c35695e admin admin] REQUEST_METHOD: `POST` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:36:15.968954 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68404822-5e0d-4b54-a8fa-55ea2c35695e admin admin] SCRIPT_NAME: `/identity` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:36:15.969367 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68404822-5e0d-4b54-a8fa-55ea2c35695e admin admin] PATH_INFO: `/v3/projects` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:36:15.970342 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68404822-5e0d-4b54-a8fa-55ea2c35695e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:36:15.975385 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68404822-5e0d-4b54-a8fa-55ea2c35695e admin admin] RBAC: Authorization granted {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:36:16.022010 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: [pid: 80245|app: 0|req: 1317/2625] 10.222.0.47 () {60 vars in 1152 bytes} [Thu Jul 21 08:36:15 2022] POST /identity/v3/projects => generated 383 bytes in 80 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 21 08:36:16.026355 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb0243ce-dce9-4dde-9f1d-3366ea693b27 admin admin] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:36:16.034256 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG dbcounter [-] [80244] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80244) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:36:16.038813 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb0243ce-dce9-4dde-9f1d-3366ea693b27 admin admin] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:36:16.129173 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26cf84ea-9ece-41db-ad14-2c21f6ef0e4b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '42d15fc0515945acacbbffcf2356c7a5', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:36:16.130251 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26cf84ea-9ece-41db-ad14-2c21f6ef0e4b admin admin] REQUEST_METHOD: `POST` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:36:16.130618 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26cf84ea-9ece-41db-ad14-2c21f6ef0e4b admin admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:36:16.130966 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26cf84ea-9ece-41db-ad14-2c21f6ef0e4b admin admin] PATH_INFO: `/v3/users` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:36:16.131833 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26cf84ea-9ece-41db-ad14-2c21f6ef0e4b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:36:16.134341 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26cf84ea-9ece-41db-ad14-2c21f6ef0e4b admin admin] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:36:16.158532 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 1309/2626] 10.222.0.47 () {60 vars in 1143 bytes} [Thu Jul 21 08:36:16 2022] POST /identity/v3/users => generated 292 bytes in 132 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 08:36:16.162513 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68404822-5e0d-4b54-a8fa-55ea2c35695e admin admin] Authenticating user token {{(pid=80245) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:36:16.174750 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68404822-5e0d-4b54-a8fa-55ea2c35695e admin admin] Validating token access rules against request {{(pid=80245) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:36:16.179562 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31c8f653-f397-4e8c-ac66-34e459ba6d28 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '42d15fc0515945acacbbffcf2356c7a5', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80245) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:36:16.180251 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31c8f653-f397-4e8c-ac66-34e459ba6d28 admin admin] REQUEST_METHOD: `GET` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:36:16.180534 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31c8f653-f397-4e8c-ac66-34e459ba6d28 admin admin] SCRIPT_NAME: `/identity` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:36:16.180810 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31c8f653-f397-4e8c-ac66-34e459ba6d28 admin admin] PATH_INFO: `/v3/roles` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:36:16.181586 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31c8f653-f397-4e8c-ac66-34e459ba6d28 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:36:16.184530 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31c8f653-f397-4e8c-ac66-34e459ba6d28 admin admin] RBAC: Authorization granted {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:36:16.196975 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: [pid: 80245|app: 0|req: 1318/2627] 10.222.0.47 () {58 vars in 1121 bytes} [Thu Jul 21 08:36:16 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 08:36:16.200609 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26cf84ea-9ece-41db-ad14-2c21f6ef0e4b admin admin] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:36:16.211499 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26cf84ea-9ece-41db-ad14-2c21f6ef0e4b admin admin] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:36:16.217793 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9535ecff-6b0f-4548-beec-a0c71c48d961 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '42d15fc0515945acacbbffcf2356c7a5', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:36:16.218396 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9535ecff-6b0f-4548-beec-a0c71c48d961 admin admin] REQUEST_METHOD: `GET` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:36:16.218689 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9535ecff-6b0f-4548-beec-a0c71c48d961 admin admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:36:16.218933 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9535ecff-6b0f-4548-beec-a0c71c48d961 admin admin] PATH_INFO: `/v3/roles` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:36:16.220116 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9535ecff-6b0f-4548-beec-a0c71c48d961 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:36:16.221321 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9535ecff-6b0f-4548-beec-a0c71c48d961 admin admin] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:36:16.232768 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 1310/2628] 10.222.0.47 () {58 vars in 1121 bytes} [Thu Jul 21 08:36:16 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 21 08:36:16.236938 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31c8f653-f397-4e8c-ac66-34e459ba6d28 admin admin] Authenticating user token {{(pid=80245) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:36:16.247660 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31c8f653-f397-4e8c-ac66-34e459ba6d28 admin admin] Validating token access rules against request {{(pid=80245) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:36:16.252982 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abdb1826-f8ef-41de-8199-acaf08524207 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '42d15fc0515945acacbbffcf2356c7a5', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80245) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:36:16.254141 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abdb1826-f8ef-41de-8199-acaf08524207 admin admin] REQUEST_METHOD: `PUT` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:36:16.254425 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abdb1826-f8ef-41de-8199-acaf08524207 admin admin] SCRIPT_NAME: `/identity` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:36:16.254703 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abdb1826-f8ef-41de-8199-acaf08524207 admin admin] PATH_INFO: `/v3/projects/ca704726a8064214b0507190fe51650f/users/7baa0b2ddca4431b8d006873ca90ab2c/roles/3a074738221e4601b5205715ba87d240` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:36:16.271261 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abdb1826-f8ef-41de-8199-acaf08524207 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ca704726a8064214b0507190fe51650f, user_id=7baa0b2ddca4431b8d006873ca90ab2c, role_id=3a074738221e4601b5205715ba87d240)` {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:36:16.273968 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abdb1826-f8ef-41de-8199-acaf08524207 admin admin] RBAC: Authorization granted {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:36:16.287894 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: [pid: 80245|app: 0|req: 1319/2629] 10.222.0.47 () {60 vars in 1482 bytes} [Thu Jul 21 08:36:16 2022] PUT /identity/v3/projects/ca704726a8064214b0507190fe51650f/users/7baa0b2ddca4431b8d006873ca90ab2c/roles/3a074738221e4601b5205715ba87d240 => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 21 08:36:16.293695 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a409a165-e934-4645-af17-ce4c3839f986 None None] REQUEST_METHOD: `POST` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:36:16.293988 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a409a165-e934-4645-af17-ce4c3839f986 None None] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:36:16.294276 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a409a165-e934-4645-af17-ce4c3839f986 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:36:16.312262 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.auth.core [None req-a409a165-e934-4645-af17-ce4c3839f986 None None] MFA Rules not processed for user `2dce40b8d74c41ec94cc36e6ff05d412`. Rule list: `[]` (Enabled: `True`). {{(pid=80244) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 08:36:16.313483 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.fernet_utils [None req-a409a165-e934-4645-af17-ce4c3839f986 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80244) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 08:36:16.374014 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 1311/2630] 10.222.0.47 () {58 vars in 957 bytes} [Thu Jul 21 08:36:16 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 08:36:16.383979 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abdb1826-f8ef-41de-8199-acaf08524207 admin admin] Authenticating user token {{(pid=80245) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:36:16.386208 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.fernet_utils [None req-abdb1826-f8ef-41de-8199-acaf08524207 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80245) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 08:36:16.450707 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abdb1826-f8ef-41de-8199-acaf08524207 admin admin] Validating token access rules against request {{(pid=80245) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:36:16.454508 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67db040d-f2bd-4538-b818-37b03186a201 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20ca3c3bd3774691b3374cfbfec6e494', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4faef13aa50e4e36b6c265fb1398fa14', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80245) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:36:16.455157 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67db040d-f2bd-4538-b818-37b03186a201 service neutron] REQUEST_METHOD: `GET` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:36:16.455470 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67db040d-f2bd-4538-b818-37b03186a201 service neutron] SCRIPT_NAME: `/identity` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:36:16.455751 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67db040d-f2bd-4538-b818-37b03186a201 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:36:16.465815 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67db040d-f2bd-4538-b818-37b03186a201 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:36:16.467270 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67db040d-f2bd-4538-b818-37b03186a201 service neutron] RBAC: Authorization granted {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:36:16.481678 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: [pid: 80245|app: 0|req: 1320/2631] 10.222.0.47 () {60 vars in 1365 bytes} [Thu Jul 21 08:36:16 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 98 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 08:36:21.639844 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a409a165-e934-4645-af17-ce4c3839f986 None None] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:36:21.641448 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.fernet_utils [None req-a409a165-e934-4645-af17-ce4c3839f986 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80244) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 08:36:21.657658 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a409a165-e934-4645-af17-ce4c3839f986 None None] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:36:21.668805 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a25b0598-c010-40f4-88b5-5e4accad576a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '42d15fc0515945acacbbffcf2356c7a5', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:36:21.669967 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a25b0598-c010-40f4-88b5-5e4accad576a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:36:21.670348 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a25b0598-c010-40f4-88b5-5e4accad576a admin admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:36:21.670766 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a25b0598-c010-40f4-88b5-5e4accad576a admin admin] PATH_INFO: `/v3/users/8249ebd6b2dc4811ad315d54030d927e` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:36:21.672649 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a25b0598-c010-40f4-88b5-5e4accad576a admin admin] RBAC: Authorizing `identity:delete_user(user_id=8249ebd6b2dc4811ad315d54030d927e)` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:36:21.674903 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a25b0598-c010-40f4-88b5-5e4accad576a admin admin] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:36:21.736613 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.notifications [None req-a25b0598-c010-40f4-88b5-5e4accad576a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8249ebd6b2dc4811ad315d54030d927e'} {{(pid=80244) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 08:36:21.748134 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.notifications [None req-a25b0598-c010-40f4-88b5-5e4accad576a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8249ebd6b2dc4811ad315d54030d927e'} {{(pid=80244) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 08:36:21.769909 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.notifications [None req-a25b0598-c010-40f4-88b5-5e4accad576a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8249ebd6b2dc4811ad315d54030d927e'} {{(pid=80244) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 08:36:21.770641 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.notifications [None req-a25b0598-c010-40f4-88b5-5e4accad576a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8249ebd6b2dc4811ad315d54030d927e'} {{(pid=80244) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 08:36:21.777063 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG dbcounter [-] [80244] Writing DB stats keystone:SELECT=86,keystone:INSERT=4,keystone:DELETE=10 {{(pid=80244) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:36:21.779343 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.notifications [None req-a25b0598-c010-40f4-88b5-5e4accad576a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8249ebd6b2dc4811ad315d54030d927e'} {{(pid=80244) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 08:36:21.790926 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 1312/2632] 10.222.0.47 () {58 vars in 1223 bytes} [Thu Jul 21 08:36:21 2022] DELETE /identity/v3/users/8249ebd6b2dc4811ad315d54030d927e => generated 0 bytes in 148 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 08:36:21.992175 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67db040d-f2bd-4538-b818-37b03186a201 service neutron] Authenticating user token {{(pid=80245) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:36:22.012415 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67db040d-f2bd-4538-b818-37b03186a201 service neutron] Validating token access rules against request {{(pid=80245) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:36:22.090338 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG dbcounter [-] [80245] Writing DB stats keystone:DELETE=6,keystone:SELECT=91,keystone:INSERT=3 {{(pid=80245) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:36:22.112234 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8057cf55-b9dc-4ddb-906c-c50d8cdb5783 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '42d15fc0515945acacbbffcf2356c7a5', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80245) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:36:22.113544 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8057cf55-b9dc-4ddb-906c-c50d8cdb5783 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:36:22.114162 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8057cf55-b9dc-4ddb-906c-c50d8cdb5783 admin admin] SCRIPT_NAME: `/identity` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:36:22.114836 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8057cf55-b9dc-4ddb-906c-c50d8cdb5783 admin admin] PATH_INFO: `/v3/projects/a8ccb394622d4b24930f8b6396671857` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:36:22.116672 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8057cf55-b9dc-4ddb-906c-c50d8cdb5783 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a8ccb394622d4b24930f8b6396671857)` {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:36:22.119742 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8057cf55-b9dc-4ddb-906c-c50d8cdb5783 admin admin] RBAC: Authorization granted {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:36:22.190843 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.notifications [None req-8057cf55-b9dc-4ddb-906c-c50d8cdb5783 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a8ccb394622d4b24930f8b6396671857', 'request_id': 'req-8057cf55-b9dc-4ddb-906c-c50d8cdb5783'} {{(pid=80245) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 08:36:22.198781 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.notifications [None req-8057cf55-b9dc-4ddb-906c-c50d8cdb5783 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a8ccb394622d4b24930f8b6396671857', 'request_id': 'req-8057cf55-b9dc-4ddb-906c-c50d8cdb5783'} {{(pid=80245) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 08:36:22.205142 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.notifications [None req-8057cf55-b9dc-4ddb-906c-c50d8cdb5783 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a8ccb394622d4b24930f8b6396671857', 'request_id': 'req-8057cf55-b9dc-4ddb-906c-c50d8cdb5783'} {{(pid=80245) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 08:36:22.233020 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.notifications [None req-8057cf55-b9dc-4ddb-906c-c50d8cdb5783 admin admin] The token cache is being invalidate because project a8ccb394622d4b24930f8b6396671857 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80245) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 21 08:36:22.233492 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.notifications [None req-8057cf55-b9dc-4ddb-906c-c50d8cdb5783 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80245) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 08:36:22.234326 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.notifications [None req-8057cf55-b9dc-4ddb-906c-c50d8cdb5783 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80245) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 08:36:22.236453 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: [pid: 80245|app: 0|req: 1321/2633] 10.222.0.47 () {58 vars in 1232 bytes} [Thu Jul 21 08:36:21 2022] DELETE /identity/v3/projects/a8ccb394622d4b24930f8b6396671857 => generated 0 bytes in 245 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 08:36:22.240372 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a25b0598-c010-40f4-88b5-5e4accad576a admin admin] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:36:22.242177 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.fernet_utils [None req-a25b0598-c010-40f4-88b5-5e4accad576a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80244) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 08:36:22.257139 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a25b0598-c010-40f4-88b5-5e4accad576a admin admin] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:36:22.330880 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-794e14e0-57ed-45fb-b93e-d601ddd5cbfd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '42d15fc0515945acacbbffcf2356c7a5', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:36:22.331809 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-794e14e0-57ed-45fb-b93e-d601ddd5cbfd admin admin] REQUEST_METHOD: `DELETE` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:36:22.332306 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-794e14e0-57ed-45fb-b93e-d601ddd5cbfd admin admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:36:22.332546 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-794e14e0-57ed-45fb-b93e-d601ddd5cbfd admin admin] PATH_INFO: `/v3/users/edcf4542e2cf43bfb62a41c82b3a3384` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:36:22.334335 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-794e14e0-57ed-45fb-b93e-d601ddd5cbfd admin admin] RBAC: Authorizing `identity:delete_user(user_id=edcf4542e2cf43bfb62a41c82b3a3384)` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:36:22.335813 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-794e14e0-57ed-45fb-b93e-d601ddd5cbfd admin admin] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:36:22.413473 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.notifications [None req-794e14e0-57ed-45fb-b93e-d601ddd5cbfd admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'edcf4542e2cf43bfb62a41c82b3a3384'} {{(pid=80244) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 08:36:22.424667 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.notifications [None req-794e14e0-57ed-45fb-b93e-d601ddd5cbfd admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'edcf4542e2cf43bfb62a41c82b3a3384'} {{(pid=80244) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 08:36:22.441033 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.notifications [None req-794e14e0-57ed-45fb-b93e-d601ddd5cbfd admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'edcf4542e2cf43bfb62a41c82b3a3384'} {{(pid=80244) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 08:36:22.442035 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.notifications [None req-794e14e0-57ed-45fb-b93e-d601ddd5cbfd admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'edcf4542e2cf43bfb62a41c82b3a3384'} {{(pid=80244) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 08:36:22.449885 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.notifications [None req-794e14e0-57ed-45fb-b93e-d601ddd5cbfd admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'edcf4542e2cf43bfb62a41c82b3a3384'} {{(pid=80244) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 08:36:22.456219 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 1313/2634] 10.222.0.47 () {58 vars in 1223 bytes} [Thu Jul 21 08:36:22 2022] DELETE /identity/v3/users/edcf4542e2cf43bfb62a41c82b3a3384 => generated 0 bytes in 216 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 08:36:22.694153 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8057cf55-b9dc-4ddb-906c-c50d8cdb5783 admin admin] Authenticating user token {{(pid=80245) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:36:22.695733 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.fernet_utils [None req-8057cf55-b9dc-4ddb-906c-c50d8cdb5783 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80245) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 08:36:22.724580 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8057cf55-b9dc-4ddb-906c-c50d8cdb5783 admin admin] Validating token access rules against request {{(pid=80245) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:36:22.811074 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-783adad7-c937-4aad-96f8-161bedbb6dd5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '42d15fc0515945acacbbffcf2356c7a5', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80245) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:36:22.812027 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-783adad7-c937-4aad-96f8-161bedbb6dd5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:36:22.812467 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-783adad7-c937-4aad-96f8-161bedbb6dd5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:36:22.812747 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-783adad7-c937-4aad-96f8-161bedbb6dd5 admin admin] PATH_INFO: `/v3/projects/da18c71220bf49458a8826a45a82d2e1` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:36:22.814167 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-783adad7-c937-4aad-96f8-161bedbb6dd5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=da18c71220bf49458a8826a45a82d2e1)` {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:36:22.816204 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-783adad7-c937-4aad-96f8-161bedbb6dd5 admin admin] RBAC: Authorization granted {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:36:22.887304 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.notifications [None req-783adad7-c937-4aad-96f8-161bedbb6dd5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'da18c71220bf49458a8826a45a82d2e1', 'request_id': 'req-783adad7-c937-4aad-96f8-161bedbb6dd5'} {{(pid=80245) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 08:36:22.901028 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.notifications [None req-783adad7-c937-4aad-96f8-161bedbb6dd5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'da18c71220bf49458a8826a45a82d2e1', 'request_id': 'req-783adad7-c937-4aad-96f8-161bedbb6dd5'} {{(pid=80245) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 08:36:22.908793 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.notifications [None req-783adad7-c937-4aad-96f8-161bedbb6dd5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'da18c71220bf49458a8826a45a82d2e1', 'request_id': 'req-783adad7-c937-4aad-96f8-161bedbb6dd5'} {{(pid=80245) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 08:36:22.926480 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.notifications [None req-783adad7-c937-4aad-96f8-161bedbb6dd5 admin admin] The token cache is being invalidate because project da18c71220bf49458a8826a45a82d2e1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80245) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 21 08:36:22.927202 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.notifications [None req-783adad7-c937-4aad-96f8-161bedbb6dd5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80245) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 08:36:22.928057 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.notifications [None req-783adad7-c937-4aad-96f8-161bedbb6dd5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80245) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 08:36:22.930262 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: [pid: 80245|app: 0|req: 1322/2635] 10.222.0.47 () {58 vars in 1232 bytes} [Thu Jul 21 08:36:22 2022] DELETE /identity/v3/projects/da18c71220bf49458a8826a45a82d2e1 => generated 0 bytes in 237 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 08:36:23.017381 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42e467a3-423d-46e2-a021-312611a815b1 None None] REQUEST_METHOD: `POST` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:36:23.018222 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42e467a3-423d-46e2-a021-312611a815b1 None None] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:36:23.018222 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42e467a3-423d-46e2-a021-312611a815b1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:36:23.066880 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.auth.core [None req-42e467a3-423d-46e2-a021-312611a815b1 None None] MFA Rules not processed for user `7baa0b2ddca4431b8d006873ca90ab2c`. Rule list: `[]` (Enabled: `True`). {{(pid=80244) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 08:36:23.069818 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.fernet_utils [None req-42e467a3-423d-46e2-a021-312611a815b1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80244) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 08:36:23.152309 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 1314/2636] 10.222.0.47 () {58 vars in 957 bytes} [Thu Jul 21 08:36:23 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 136 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 08:36:23.170964 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-783adad7-c937-4aad-96f8-161bedbb6dd5 admin admin] Authenticating user token {{(pid=80245) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:36:23.173371 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.fernet_utils [None req-783adad7-c937-4aad-96f8-161bedbb6dd5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80245) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 08:36:23.199314 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG dbcounter [-] [80245] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=80245) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:36:23.272801 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-783adad7-c937-4aad-96f8-161bedbb6dd5 admin admin] Validating token access rules against request {{(pid=80245) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:36:23.279727 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1341993c-6f0a-4fe4-9856-1d871ace94b3 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2b5b99a36442148fe743269054e991', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4faef13aa50e4e36b6c265fb1398fa14', 'project_domain_id': 'default', 'roles': ['member', 'service', 'ResellerAdmin', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80245) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:36:23.280572 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1341993c-6f0a-4fe4-9856-1d871ace94b3 service nova] REQUEST_METHOD: `GET` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:36:23.280971 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1341993c-6f0a-4fe4-9856-1d871ace94b3 service nova] SCRIPT_NAME: `/identity` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:36:23.281448 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1341993c-6f0a-4fe4-9856-1d871ace94b3 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:36:23.290497 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1341993c-6f0a-4fe4-9856-1d871ace94b3 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:36:23.292300 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1341993c-6f0a-4fe4-9856-1d871ace94b3 service nova] RBAC: Authorization granted {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:36:23.302107 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: [pid: 80245|app: 0|req: 1323/2637] 10.222.0.47 () {60 vars in 1365 bytes} [Thu Jul 21 08:36:23 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 132 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 08:36:24.858881 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d042fced-e661-4893-9a7b-c77e85c395c9 None None] REQUEST_METHOD: `POST` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:36:24.859470 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d042fced-e661-4893-9a7b-c77e85c395c9 None None] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:36:24.859655 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d042fced-e661-4893-9a7b-c77e85c395c9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:36:24.867506 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG dbcounter [-] [80244] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80244) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:36:24.879097 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.auth.core [None req-d042fced-e661-4893-9a7b-c77e85c395c9 None None] MFA Rules not processed for user `7baa0b2ddca4431b8d006873ca90ab2c`. Rule list: `[]` (Enabled: `True`). {{(pid=80244) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 08:36:24.880429 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.fernet_utils [None req-d042fced-e661-4893-9a7b-c77e85c395c9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80244) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 08:36:24.889208 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 1315/2638] 10.222.0.47 () {58 vars in 957 bytes} [Thu Jul 21 08:36:24 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 08:36:33.302480 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG dbcounter [-] [80245] Writing DB stats keystone:SELECT=28 {{(pid=80245) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:36:34.866056 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1341993c-6f0a-4fe4-9856-1d871ace94b3 service nova] Authenticating user token {{(pid=80245) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:36:34.867719 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.fernet_utils [None req-1341993c-6f0a-4fe4-9856-1d871ace94b3 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80245) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 08:36:34.883097 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG dbcounter [-] [80244] Writing DB stats keystone:SELECT=1 {{(pid=80244) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:36:34.889713 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1341993c-6f0a-4fe4-9856-1d871ace94b3 service nova] Validating token access rules against request {{(pid=80245) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:36:34.895578 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc362c79-c92b-46d2-9510-c3c458f40b4b service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2b5b99a36442148fe743269054e991', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4faef13aa50e4e36b6c265fb1398fa14', 'project_domain_id': 'default', 'roles': ['member', 'service', 'ResellerAdmin', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80245) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:36:34.896715 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc362c79-c92b-46d2-9510-c3c458f40b4b service nova] REQUEST_METHOD: `GET` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:36:34.896715 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc362c79-c92b-46d2-9510-c3c458f40b4b service nova] SCRIPT_NAME: `/identity` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:36:34.896941 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc362c79-c92b-46d2-9510-c3c458f40b4b service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:36:34.899160 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.fernet_utils [None req-dc362c79-c92b-46d2-9510-c3c458f40b4b service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80245) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 08:36:34.912928 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc362c79-c92b-46d2-9510-c3c458f40b4b service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:36:34.914544 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc362c79-c92b-46d2-9510-c3c458f40b4b service nova] RBAC: Authorization granted {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:36:34.926070 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: [pid: 80245|app: 0|req: 1324/2639] 10.222.0.47 () {60 vars in 1365 bytes} [Thu Jul 21 08:36:34 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 60 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 08:36:44.921160 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG dbcounter [-] [80245] Writing DB stats keystone:SELECT=6 {{(pid=80245) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:36:45.532440 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92b9b5ca-0977-4e2e-aecc-d01f46acf3b7 None None] REQUEST_METHOD: `POST` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:36:45.532958 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92b9b5ca-0977-4e2e-aecc-d01f46acf3b7 None None] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:36:45.533061 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92b9b5ca-0977-4e2e-aecc-d01f46acf3b7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:36:45.551782 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.auth.core [None req-92b9b5ca-0977-4e2e-aecc-d01f46acf3b7 None None] MFA Rules not processed for user `2ef7f1e91e554a1f88eb6a63e60a475f`. Rule list: `[]` (Enabled: `True`). {{(pid=80244) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 08:36:45.554065 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.fernet_utils [None req-92b9b5ca-0977-4e2e-aecc-d01f46acf3b7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80244) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 08:36:45.615143 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 1316/2640] 10.222.0.47 () {58 vars in 1009 bytes} [Thu Jul 21 08:36:45 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 08:36:45.627980 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc362c79-c92b-46d2-9510-c3c458f40b4b service nova] Authenticating user token {{(pid=80245) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:36:45.629718 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.fernet_utils [None req-dc362c79-c92b-46d2-9510-c3c458f40b4b service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80245) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 08:36:45.703429 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc362c79-c92b-46d2-9510-c3c458f40b4b service nova] Validating token access rules against request {{(pid=80245) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:36:45.708713 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae562067-966f-46d5-9bf4-e9e10f60e988 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '302351411fc84248a6b166b25d21609f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4faef13aa50e4e36b6c265fb1398fa14', 'project_domain_id': 'default', 'roles': ['service', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80245) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:36:45.709396 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae562067-966f-46d5-9bf4-e9e10f60e988 service swift] REQUEST_METHOD: `GET` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:36:45.709710 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae562067-966f-46d5-9bf4-e9e10f60e988 service swift] SCRIPT_NAME: `/identity` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:36:45.710081 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae562067-966f-46d5-9bf4-e9e10f60e988 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:36:45.718768 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae562067-966f-46d5-9bf4-e9e10f60e988 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:36:45.720393 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae562067-966f-46d5-9bf4-e9e10f60e988 service swift] RBAC: Authorization granted {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:36:45.728629 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: [pid: 80245|app: 0|req: 1325/2641] 10.222.0.47 () {60 vars in 1384 bytes} [Thu Jul 21 08:36:45 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 101 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 21 08:36:55.607425 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG dbcounter [-] [80244] Writing DB stats keystone:SELECT=25 {{(pid=80244) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:36:55.725276 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG dbcounter [-] [80245] Writing DB stats keystone:SELECT=32 {{(pid=80245) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:37:00.102978 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92b9b5ca-0977-4e2e-aecc-d01f46acf3b7 None None] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:37:00.104674 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.fernet_utils [None req-92b9b5ca-0977-4e2e-aecc-d01f46acf3b7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80244) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 08:37:00.166177 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92b9b5ca-0977-4e2e-aecc-d01f46acf3b7 None None] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:37:00.170218 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2638d116-33b2-48f6-ac97-a84db5f5bc1b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20ca3c3bd3774691b3374cfbfec6e494', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4faef13aa50e4e36b6c265fb1398fa14', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:37:00.170912 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2638d116-33b2-48f6-ac97-a84db5f5bc1b service neutron] REQUEST_METHOD: `GET` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:37:00.171207 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2638d116-33b2-48f6-ac97-a84db5f5bc1b service neutron] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:37:00.171518 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2638d116-33b2-48f6-ac97-a84db5f5bc1b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:37:00.173307 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.fernet_utils [None req-2638d116-33b2-48f6-ac97-a84db5f5bc1b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80244) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 08:37:00.183658 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2638d116-33b2-48f6-ac97-a84db5f5bc1b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:37:00.185090 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2638d116-33b2-48f6-ac97-a84db5f5bc1b service neutron] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:37:00.193286 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 1317/2642] 10.222.0.47 () {60 vars in 1365 bytes} [Thu Jul 21 08:37:00 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 08:37:00.297840 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a00aa2f-4b1a-4df6-b468-c129c2feafc5 None None] REQUEST_METHOD: `GET` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:37:00.298420 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a00aa2f-4b1a-4df6-b468-c129c2feafc5 None None] SCRIPT_NAME: `/identity` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:37:00.298657 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a00aa2f-4b1a-4df6-b468-c129c2feafc5 None None] PATH_INFO: `/` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:37:00.300454 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: [pid: 80245|app: 0|req: 1326/2643] 10.222.0.47 () {54 vars in 921 bytes} [Thu Jul 21 08:37:00 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 08:37:00.307098 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89b78be4-0ecb-4785-a273-813b5782a34d None None] REQUEST_METHOD: `POST` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:37:00.307372 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89b78be4-0ecb-4785-a273-813b5782a34d None None] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:37:00.307778 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89b78be4-0ecb-4785-a273-813b5782a34d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:37:00.326827 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.auth.core [None req-89b78be4-0ecb-4785-a273-813b5782a34d None None] MFA Rules not processed for user `20ca3c3bd3774691b3374cfbfec6e494`. Rule list: `[]` (Enabled: `True`). {{(pid=80244) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 08:37:00.328068 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.fernet_utils [None req-89b78be4-0ecb-4785-a273-813b5782a34d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80244) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 08:37:00.335491 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 1318/2644] 10.222.0.47 () {58 vars in 1018 bytes} [Thu Jul 21 08:37:00 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 08:37:00.348429 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a00aa2f-4b1a-4df6-b468-c129c2feafc5 None None] Authenticating user token {{(pid=80245) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:37:00.350552 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.fernet_utils [None req-8a00aa2f-4b1a-4df6-b468-c129c2feafc5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80245) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 08:37:00.367816 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a00aa2f-4b1a-4df6-b468-c129c2feafc5 None None] Validating token access rules against request {{(pid=80245) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:37:00.378963 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f21aa611-d877-4444-b07a-8f29e75e252a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20ca3c3bd3774691b3374cfbfec6e494', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4faef13aa50e4e36b6c265fb1398fa14', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80245) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:37:00.379659 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f21aa611-d877-4444-b07a-8f29e75e252a service neutron] REQUEST_METHOD: `GET` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:37:00.380133 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f21aa611-d877-4444-b07a-8f29e75e252a service neutron] SCRIPT_NAME: `/identity` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:37:00.380692 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f21aa611-d877-4444-b07a-8f29e75e252a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:37:00.390059 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f21aa611-d877-4444-b07a-8f29e75e252a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:37:00.392019 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f21aa611-d877-4444-b07a-8f29e75e252a service neutron] RBAC: Authorization granted {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:37:00.405285 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: [pid: 80245|app: 0|req: 1327/2645] 10.222.0.47 () {60 vars in 1365 bytes} [Thu Jul 21 08:37:00 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 57 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 08:37:00.497272 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26809117-52f6-4b1f-97a1-9e45ce2846d8 None None] REQUEST_METHOD: `GET` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:37:00.497678 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26809117-52f6-4b1f-97a1-9e45ce2846d8 None None] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:37:00.498110 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26809117-52f6-4b1f-97a1-9e45ce2846d8 None None] PATH_INFO: `/` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:37:00.499573 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 1319/2646] 10.222.0.47 () {54 vars in 921 bytes} [Thu Jul 21 08:37:00 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 08:37:00.506034 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-354516a5-6efd-41b3-89ef-645e4bbdfd21 None None] REQUEST_METHOD: `POST` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:37:00.506334 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-354516a5-6efd-41b3-89ef-645e4bbdfd21 None None] SCRIPT_NAME: `/identity` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:37:00.506623 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-354516a5-6efd-41b3-89ef-645e4bbdfd21 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:37:00.530158 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.auth.core [None req-354516a5-6efd-41b3-89ef-645e4bbdfd21 None None] MFA Rules not processed for user `ac2b5b99a36442148fe743269054e991`. Rule list: `[]` (Enabled: `True`). {{(pid=80245) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 08:37:00.531466 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.fernet_utils [None req-354516a5-6efd-41b3-89ef-645e4bbdfd21 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80245) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 08:37:00.542122 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: [pid: 80245|app: 0|req: 1328/2647] 10.222.0.47 () {58 vars in 1018 bytes} [Thu Jul 21 08:37:00 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 08:37:00.557626 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26809117-52f6-4b1f-97a1-9e45ce2846d8 None None] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:37:00.569562 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26809117-52f6-4b1f-97a1-9e45ce2846d8 None None] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:37:00.573912 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f83c64c9-7437-4888-ab5a-e0e1bf322d72 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20ca3c3bd3774691b3374cfbfec6e494', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4faef13aa50e4e36b6c265fb1398fa14', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:37:00.574483 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f83c64c9-7437-4888-ab5a-e0e1bf322d72 service neutron] REQUEST_METHOD: `GET` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:37:00.574850 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f83c64c9-7437-4888-ab5a-e0e1bf322d72 service neutron] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:37:00.575207 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f83c64c9-7437-4888-ab5a-e0e1bf322d72 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:37:00.586883 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f83c64c9-7437-4888-ab5a-e0e1bf322d72 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:37:00.588383 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f83c64c9-7437-4888-ab5a-e0e1bf322d72 service neutron] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:37:00.601265 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 1320/2648] 10.222.0.47 () {60 vars in 1365 bytes} [Thu Jul 21 08:37:00 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 08:37:02.640425 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd6403d1-41b2-4711-aa6e-0f91e565c89f None None] REQUEST_METHOD: `POST` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:37:02.640983 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd6403d1-41b2-4711-aa6e-0f91e565c89f None None] SCRIPT_NAME: `/identity` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:37:02.640983 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd6403d1-41b2-4711-aa6e-0f91e565c89f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:37:02.660575 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.auth.core [None req-cd6403d1-41b2-4711-aa6e-0f91e565c89f None None] MFA Rules not processed for user `2ef7f1e91e554a1f88eb6a63e60a475f`. Rule list: `[]` (Enabled: `True`). {{(pid=80245) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 08:37:02.661854 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.fernet_utils [None req-cd6403d1-41b2-4711-aa6e-0f91e565c89f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80245) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 08:37:02.669268 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: [pid: 80245|app: 0|req: 1329/2649] 10.222.0.47 () {58 vars in 1009 bytes} [Thu Jul 21 08:37:02 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 08:37:02.681932 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f83c64c9-7437-4888-ab5a-e0e1bf322d72 service neutron] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:37:02.694499 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f83c64c9-7437-4888-ab5a-e0e1bf322d72 service neutron] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:37:02.700545 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-100d98ed-a77e-42a6-a69b-ea8ed5f905a2 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '302351411fc84248a6b166b25d21609f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4faef13aa50e4e36b6c265fb1398fa14', 'project_domain_id': 'default', 'roles': ['service', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:37:02.701291 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-100d98ed-a77e-42a6-a69b-ea8ed5f905a2 service swift] REQUEST_METHOD: `GET` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:37:02.701642 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-100d98ed-a77e-42a6-a69b-ea8ed5f905a2 service swift] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:37:02.701985 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-100d98ed-a77e-42a6-a69b-ea8ed5f905a2 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:37:02.711649 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-100d98ed-a77e-42a6-a69b-ea8ed5f905a2 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:37:02.713386 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-100d98ed-a77e-42a6-a69b-ea8ed5f905a2 service swift] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:37:02.731473 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 1321/2650] 10.222.0.47 () {60 vars in 1384 bytes} [Thu Jul 21 08:37:02 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 50 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 21 08:37:12.654356 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG dbcounter [-] [80245] Writing DB stats keystone:SELECT=12 {{(pid=80245) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:37:12.719550 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG dbcounter [-] [80244] Writing DB stats keystone:SELECT=41 {{(pid=80244) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:37:19.312538 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aac88767-c6e7-4a63-9700-50ae3217c2a7 None None] REQUEST_METHOD: `POST` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:37:19.313012 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aac88767-c6e7-4a63-9700-50ae3217c2a7 None None] SCRIPT_NAME: `/identity` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:37:19.313096 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aac88767-c6e7-4a63-9700-50ae3217c2a7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:37:19.332001 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.auth.core [None req-aac88767-c6e7-4a63-9700-50ae3217c2a7 None None] MFA Rules not processed for user `2ef7f1e91e554a1f88eb6a63e60a475f`. Rule list: `[]` (Enabled: `True`). {{(pid=80245) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 08:37:19.333280 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.fernet_utils [None req-aac88767-c6e7-4a63-9700-50ae3217c2a7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80245) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 08:37:19.341101 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: [pid: 80245|app: 0|req: 1330/2651] 10.222.0.47 () {58 vars in 967 bytes} [Thu Jul 21 08:37:19 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 08:37:19.363199 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-100d98ed-a77e-42a6-a69b-ea8ed5f905a2 service swift] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:37:19.374278 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-100d98ed-a77e-42a6-a69b-ea8ed5f905a2 service swift] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:37:19.379595 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f5f6075-2339-48d9-ab29-fdab476aa9e6 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '302351411fc84248a6b166b25d21609f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4faef13aa50e4e36b6c265fb1398fa14', 'project_domain_id': 'default', 'roles': ['service', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:37:19.380243 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f5f6075-2339-48d9-ab29-fdab476aa9e6 service swift] REQUEST_METHOD: `GET` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:37:19.380528 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f5f6075-2339-48d9-ab29-fdab476aa9e6 service swift] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:37:19.380807 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f5f6075-2339-48d9-ab29-fdab476aa9e6 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:37:19.389960 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f5f6075-2339-48d9-ab29-fdab476aa9e6 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:37:19.391669 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f5f6075-2339-48d9-ab29-fdab476aa9e6 service swift] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:37:19.400736 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 1322/2652] 10.222.0.47 () {60 vars in 1384 bytes} [Thu Jul 21 08:37:19 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 21 08:37:24.346386 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dabd7811-a792-4e8d-89fe-5c4065793fae None None] REQUEST_METHOD: `POST` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:37:24.346897 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dabd7811-a792-4e8d-89fe-5c4065793fae None None] SCRIPT_NAME: `/identity` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:37:24.346995 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dabd7811-a792-4e8d-89fe-5c4065793fae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:37:24.365122 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.auth.core [None req-dabd7811-a792-4e8d-89fe-5c4065793fae None None] MFA Rules not processed for user `7baa0b2ddca4431b8d006873ca90ab2c`. Rule list: `[]` (Enabled: `True`). {{(pid=80245) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 08:37:24.366327 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.fernet_utils [None req-dabd7811-a792-4e8d-89fe-5c4065793fae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80245) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 08:37:24.373318 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: [pid: 80245|app: 0|req: 1331/2653] 10.222.0.47 () {58 vars in 957 bytes} [Thu Jul 21 08:37:24 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 08:37:29.397741 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG dbcounter [-] [80244] Writing DB stats keystone:SELECT=6 {{(pid=80244) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:37:34.359723 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG dbcounter [-] [80245] Writing DB stats keystone:SELECT=6 {{(pid=80245) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:38:06.487776 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f5f6075-2339-48d9-ab29-fdab476aa9e6 service swift] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:38:06.497340 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f5f6075-2339-48d9-ab29-fdab476aa9e6 service swift] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:38:06.500874 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab77d10c-a543-4a62-a27d-acfe6aebecea service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20ca3c3bd3774691b3374cfbfec6e494', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4faef13aa50e4e36b6c265fb1398fa14', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:38:06.501512 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab77d10c-a543-4a62-a27d-acfe6aebecea service neutron] REQUEST_METHOD: `GET` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:38:06.501792 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab77d10c-a543-4a62-a27d-acfe6aebecea service neutron] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:38:06.502065 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab77d10c-a543-4a62-a27d-acfe6aebecea service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:38:06.504178 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.fernet_utils [None req-ab77d10c-a543-4a62-a27d-acfe6aebecea service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80244) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 08:38:06.573619 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab77d10c-a543-4a62-a27d-acfe6aebecea service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:38:06.575042 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab77d10c-a543-4a62-a27d-acfe6aebecea service neutron] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:38:06.585169 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 1323/2654] 10.222.0.47 () {60 vars in 1365 bytes} [Thu Jul 21 08:38:06 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 98 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 08:38:09.684296 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dabd7811-a792-4e8d-89fe-5c4065793fae None None] Authenticating user token {{(pid=80245) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:38:09.686646 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.fernet_utils [None req-dabd7811-a792-4e8d-89fe-5c4065793fae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80245) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 08:38:09.745292 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab77d10c-a543-4a62-a27d-acfe6aebecea service neutron] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:38:09.747854 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.fernet_utils [None req-ab77d10c-a543-4a62-a27d-acfe6aebecea service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80244) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 08:38:09.838431 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dabd7811-a792-4e8d-89fe-5c4065793fae None None] Validating token access rules against request {{(pid=80245) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:38:09.843722 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84808845-68f2-4f32-8555-2c9979a77770 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3379b32543474d0a8e897484e85c7d04', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4faef13aa50e4e36b6c265fb1398fa14', 'project_domain_id': 'default', 'roles': ['service', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80245) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:38:09.844507 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84808845-68f2-4f32-8555-2c9979a77770 service placement] REQUEST_METHOD: `GET` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:38:09.844801 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84808845-68f2-4f32-8555-2c9979a77770 service placement] SCRIPT_NAME: `/identity` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:38:09.845090 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84808845-68f2-4f32-8555-2c9979a77770 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:38:09.846502 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab77d10c-a543-4a62-a27d-acfe6aebecea service neutron] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:38:09.847595 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.fernet_utils [None req-84808845-68f2-4f32-8555-2c9979a77770 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80245) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 08:38:09.851673 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65bf8756-5288-4c63-96f7-7bb9c75538f3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '42d15fc0515945acacbbffcf2356c7a5', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:38:09.852610 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65bf8756-5288-4c63-96f7-7bb9c75538f3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:38:09.852958 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65bf8756-5288-4c63-96f7-7bb9c75538f3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:38:09.853296 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65bf8756-5288-4c63-96f7-7bb9c75538f3 admin admin] PATH_INFO: `/v3/users/ad56309db7044cb2a86651d4149cd090` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:38:09.854621 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65bf8756-5288-4c63-96f7-7bb9c75538f3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ad56309db7044cb2a86651d4149cd090)` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:38:09.856576 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65bf8756-5288-4c63-96f7-7bb9c75538f3 admin admin] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:38:09.859386 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84808845-68f2-4f32-8555-2c9979a77770 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:38:09.860970 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84808845-68f2-4f32-8555-2c9979a77770 service placement] RBAC: Authorization granted {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:38:09.871272 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: [pid: 80245|app: 0|req: 1332/2655] 10.222.0.47 () {60 vars in 1365 bytes} [Thu Jul 21 08:38:09 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 188 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 08:38:09.914705 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.notifications [None req-65bf8756-5288-4c63-96f7-7bb9c75538f3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ad56309db7044cb2a86651d4149cd090'} {{(pid=80244) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 08:38:09.922315 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.notifications [None req-65bf8756-5288-4c63-96f7-7bb9c75538f3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ad56309db7044cb2a86651d4149cd090'} {{(pid=80244) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 08:38:09.935191 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.notifications [None req-65bf8756-5288-4c63-96f7-7bb9c75538f3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ad56309db7044cb2a86651d4149cd090'} {{(pid=80244) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 08:38:09.935931 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.notifications [None req-65bf8756-5288-4c63-96f7-7bb9c75538f3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ad56309db7044cb2a86651d4149cd090'} {{(pid=80244) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 08:38:09.945129 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.notifications [None req-65bf8756-5288-4c63-96f7-7bb9c75538f3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ad56309db7044cb2a86651d4149cd090'} {{(pid=80244) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 08:38:09.951591 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 1324/2656] 10.222.0.47 () {58 vars in 1223 bytes} [Thu Jul 21 08:38:09 2022] DELETE /identity/v3/users/ad56309db7044cb2a86651d4149cd090 => generated 0 bytes in 207 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 08:38:10.121879 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84808845-68f2-4f32-8555-2c9979a77770 service placement] Authenticating user token {{(pid=80245) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:38:10.133452 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84808845-68f2-4f32-8555-2c9979a77770 service placement] Validating token access rules against request {{(pid=80245) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:38:10.190194 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac3d20a8-5ccd-435c-bab8-f21a3577d6a9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '42d15fc0515945acacbbffcf2356c7a5', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80245) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:38:10.191739 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac3d20a8-5ccd-435c-bab8-f21a3577d6a9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:38:10.192024 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac3d20a8-5ccd-435c-bab8-f21a3577d6a9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:38:10.192302 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac3d20a8-5ccd-435c-bab8-f21a3577d6a9 admin admin] PATH_INFO: `/v3/projects/1bae82711d354d908d096edaae82c2c5` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:38:10.193493 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac3d20a8-5ccd-435c-bab8-f21a3577d6a9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=1bae82711d354d908d096edaae82c2c5)` {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:38:10.195472 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac3d20a8-5ccd-435c-bab8-f21a3577d6a9 admin admin] RBAC: Authorization granted {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:38:10.239846 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG dbcounter [-] [80245] Writing DB stats keystone:SELECT=97,keystone:DELETE=3 {{(pid=80245) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:38:10.244258 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.notifications [None req-ac3d20a8-5ccd-435c-bab8-f21a3577d6a9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1bae82711d354d908d096edaae82c2c5', 'request_id': 'req-ac3d20a8-5ccd-435c-bab8-f21a3577d6a9'} {{(pid=80245) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 08:38:10.252238 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.notifications [None req-ac3d20a8-5ccd-435c-bab8-f21a3577d6a9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1bae82711d354d908d096edaae82c2c5', 'request_id': 'req-ac3d20a8-5ccd-435c-bab8-f21a3577d6a9'} {{(pid=80245) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 08:38:10.259802 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.notifications [None req-ac3d20a8-5ccd-435c-bab8-f21a3577d6a9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1bae82711d354d908d096edaae82c2c5', 'request_id': 'req-ac3d20a8-5ccd-435c-bab8-f21a3577d6a9'} {{(pid=80245) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 08:38:10.274910 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.notifications [None req-ac3d20a8-5ccd-435c-bab8-f21a3577d6a9 admin admin] The token cache is being invalidate because project 1bae82711d354d908d096edaae82c2c5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80245) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 21 08:38:10.275209 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.notifications [None req-ac3d20a8-5ccd-435c-bab8-f21a3577d6a9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80245) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 08:38:10.275873 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.notifications [None req-ac3d20a8-5ccd-435c-bab8-f21a3577d6a9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80245) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 08:38:10.277572 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: [pid: 80245|app: 0|req: 1333/2657] 10.222.0.47 () {58 vars in 1232 bytes} [Thu Jul 21 08:38:10 2022] DELETE /identity/v3/projects/1bae82711d354d908d096edaae82c2c5 => generated 0 bytes in 156 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 21 08:38:10.283272 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d23ae628-42f2-401d-9edd-34ae8d708b7a None None] REQUEST_METHOD: `POST` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:38:10.283612 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d23ae628-42f2-401d-9edd-34ae8d708b7a None None] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:38:10.283896 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d23ae628-42f2-401d-9edd-34ae8d708b7a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:38:10.288617 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG dbcounter [-] [80244] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80244) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:38:10.347450 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.auth.core [None req-d23ae628-42f2-401d-9edd-34ae8d708b7a None None] MFA Rules not processed for user `2dce40b8d74c41ec94cc36e6ff05d412`. Rule list: `[]` (Enabled: `True`). {{(pid=80244) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 08:38:10.348690 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.fernet_utils [None req-d23ae628-42f2-401d-9edd-34ae8d708b7a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80244) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 08:38:10.408858 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 1325/2658] 10.222.0.47 () {58 vars in 957 bytes} [Thu Jul 21 08:38:10 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 08:38:10.416871 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18ad299a-e350-40fb-b990-71e899a8eafb None None] REQUEST_METHOD: `POST` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:38:10.417163 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18ad299a-e350-40fb-b990-71e899a8eafb None None] SCRIPT_NAME: `/identity` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:38:10.417439 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18ad299a-e350-40fb-b990-71e899a8eafb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:38:10.436184 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.auth.core [None req-18ad299a-e350-40fb-b990-71e899a8eafb None None] MFA Rules not processed for user `2dce40b8d74c41ec94cc36e6ff05d412`. Rule list: `[]` (Enabled: `True`). {{(pid=80245) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 08:38:10.437545 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.fernet_utils [None req-18ad299a-e350-40fb-b990-71e899a8eafb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80245) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 08:38:10.445713 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: [pid: 80245|app: 0|req: 1334/2659] 10.222.0.47 () {58 vars in 957 bytes} [Thu Jul 21 08:38:10 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 08:38:10.449939 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d23ae628-42f2-401d-9edd-34ae8d708b7a None None] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:38:10.459773 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d23ae628-42f2-401d-9edd-34ae8d708b7a None None] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:38:10.464325 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a00ff9b6-39ab-4ea8-910f-816684cc3e98 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '42d15fc0515945acacbbffcf2356c7a5', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:38:10.464977 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a00ff9b6-39ab-4ea8-910f-816684cc3e98 admin admin] REQUEST_METHOD: `GET` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:38:10.465256 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a00ff9b6-39ab-4ea8-910f-816684cc3e98 admin admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:38:10.465526 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a00ff9b6-39ab-4ea8-910f-816684cc3e98 admin admin] PATH_INFO: `/v3/domains` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:38:10.466270 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a00ff9b6-39ab-4ea8-910f-816684cc3e98 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80244) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 08:38:10.466574 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a00ff9b6-39ab-4ea8-910f-816684cc3e98 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:38:10.468212 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a00ff9b6-39ab-4ea8-910f-816684cc3e98 admin admin] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:38:10.480404 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 1326/2660] 10.222.0.47 () {58 vars in 1152 bytes} [Thu Jul 21 08:38:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 08:38:10.484801 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18ad299a-e350-40fb-b990-71e899a8eafb None None] Authenticating user token {{(pid=80245) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:38:10.494793 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18ad299a-e350-40fb-b990-71e899a8eafb None None] Validating token access rules against request {{(pid=80245) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:38:10.499534 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ca406bf-8f87-4728-a5dc-1ad5f2e6f078 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '42d15fc0515945acacbbffcf2356c7a5', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80245) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:38:10.500190 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ca406bf-8f87-4728-a5dc-1ad5f2e6f078 admin admin] REQUEST_METHOD: `POST` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:38:10.500472 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ca406bf-8f87-4728-a5dc-1ad5f2e6f078 admin admin] SCRIPT_NAME: `/identity` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:38:10.500751 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ca406bf-8f87-4728-a5dc-1ad5f2e6f078 admin admin] PATH_INFO: `/v3/projects` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:38:10.501450 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ca406bf-8f87-4728-a5dc-1ad5f2e6f078 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:38:10.503656 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ca406bf-8f87-4728-a5dc-1ad5f2e6f078 admin admin] RBAC: Authorization granted {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:38:10.529655 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: [pid: 80245|app: 0|req: 1335/2661] 10.222.0.47 () {60 vars in 1152 bytes} [Thu Jul 21 08:38:10 2022] POST /identity/v3/projects => generated 375 bytes in 45 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 08:38:10.533325 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a00ff9b6-39ab-4ea8-910f-816684cc3e98 admin admin] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:38:10.543929 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a00ff9b6-39ab-4ea8-910f-816684cc3e98 admin admin] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:38:10.610121 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eaf0d29-a2b6-4f45-9612-cc30f27f84c9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '42d15fc0515945acacbbffcf2356c7a5', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:38:10.610804 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eaf0d29-a2b6-4f45-9612-cc30f27f84c9 admin admin] REQUEST_METHOD: `POST` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:38:10.611091 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eaf0d29-a2b6-4f45-9612-cc30f27f84c9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:38:10.611403 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eaf0d29-a2b6-4f45-9612-cc30f27f84c9 admin admin] PATH_INFO: `/v3/users` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:38:10.612146 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4eaf0d29-a2b6-4f45-9612-cc30f27f84c9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:38:10.615022 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4eaf0d29-a2b6-4f45-9612-cc30f27f84c9 admin admin] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:38:10.635344 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 1327/2662] 10.222.0.47 () {60 vars in 1143 bytes} [Thu Jul 21 08:38:10 2022] POST /identity/v3/users => generated 288 bytes in 102 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 08:38:10.639867 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ca406bf-8f87-4728-a5dc-1ad5f2e6f078 admin admin] Authenticating user token {{(pid=80245) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:38:10.650124 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ca406bf-8f87-4728-a5dc-1ad5f2e6f078 admin admin] Validating token access rules against request {{(pid=80245) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:38:10.654887 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57dcddab-6849-4006-95cc-85772789bebb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '42d15fc0515945acacbbffcf2356c7a5', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80245) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:38:10.655594 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57dcddab-6849-4006-95cc-85772789bebb admin admin] REQUEST_METHOD: `GET` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:38:10.655877 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57dcddab-6849-4006-95cc-85772789bebb admin admin] SCRIPT_NAME: `/identity` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:38:10.656171 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57dcddab-6849-4006-95cc-85772789bebb admin admin] PATH_INFO: `/v3/roles` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:38:10.656870 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57dcddab-6849-4006-95cc-85772789bebb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:38:10.658492 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57dcddab-6849-4006-95cc-85772789bebb admin admin] RBAC: Authorization granted {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:38:10.670205 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: [pid: 80245|app: 0|req: 1336/2663] 10.222.0.47 () {58 vars in 1121 bytes} [Thu Jul 21 08:38:10 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 21 08:38:10.674301 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eaf0d29-a2b6-4f45-9612-cc30f27f84c9 admin admin] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:38:10.684754 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eaf0d29-a2b6-4f45-9612-cc30f27f84c9 admin admin] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:38:10.689529 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fa28f6a-2a9c-4c99-8652-800aea2f5798 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '42d15fc0515945acacbbffcf2356c7a5', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:38:10.690177 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fa28f6a-2a9c-4c99-8652-800aea2f5798 admin admin] REQUEST_METHOD: `GET` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:38:10.690489 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fa28f6a-2a9c-4c99-8652-800aea2f5798 admin admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:38:10.690774 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fa28f6a-2a9c-4c99-8652-800aea2f5798 admin admin] PATH_INFO: `/v3/roles` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:38:10.691536 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fa28f6a-2a9c-4c99-8652-800aea2f5798 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:38:10.693127 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fa28f6a-2a9c-4c99-8652-800aea2f5798 admin admin] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:38:10.702879 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 1328/2664] 10.222.0.47 () {58 vars in 1121 bytes} [Thu Jul 21 08:38:10 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 08:38:10.707638 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57dcddab-6849-4006-95cc-85772789bebb admin admin] Authenticating user token {{(pid=80245) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:38:10.717711 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57dcddab-6849-4006-95cc-85772789bebb admin admin] Validating token access rules against request {{(pid=80245) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:38:10.723646 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-128f60a7-841d-4c15-ab24-ceaea003e0a2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '42d15fc0515945acacbbffcf2356c7a5', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80245) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:38:10.724535 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-128f60a7-841d-4c15-ab24-ceaea003e0a2 admin admin] REQUEST_METHOD: `PUT` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:38:10.724818 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-128f60a7-841d-4c15-ab24-ceaea003e0a2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:38:10.725095 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-128f60a7-841d-4c15-ab24-ceaea003e0a2 admin admin] PATH_INFO: `/v3/projects/d588f87ef1d248528ec37cd9b5d43dcc/users/591ec12e19db47139d3a73484abf2f0d/roles/3a074738221e4601b5205715ba87d240` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:38:10.739747 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-128f60a7-841d-4c15-ab24-ceaea003e0a2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d588f87ef1d248528ec37cd9b5d43dcc, user_id=591ec12e19db47139d3a73484abf2f0d, role_id=3a074738221e4601b5205715ba87d240)` {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:38:10.742378 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-128f60a7-841d-4c15-ab24-ceaea003e0a2 admin admin] RBAC: Authorization granted {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:38:10.755279 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: [pid: 80245|app: 0|req: 1337/2665] 10.222.0.47 () {60 vars in 1482 bytes} [Thu Jul 21 08:38:10 2022] PUT /identity/v3/projects/d588f87ef1d248528ec37cd9b5d43dcc/users/591ec12e19db47139d3a73484abf2f0d/roles/3a074738221e4601b5205715ba87d240 => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 08:38:10.760647 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-365b09b6-63ec-4753-a242-0bff732ddbc8 None None] REQUEST_METHOD: `POST` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:38:10.761198 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-365b09b6-63ec-4753-a242-0bff732ddbc8 None None] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:38:10.761360 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-365b09b6-63ec-4753-a242-0bff732ddbc8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:38:10.779522 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.auth.core [None req-365b09b6-63ec-4753-a242-0bff732ddbc8 None None] MFA Rules not processed for user `2dce40b8d74c41ec94cc36e6ff05d412`. Rule list: `[]` (Enabled: `True`). {{(pid=80244) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 08:38:10.780753 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.fernet_utils [None req-365b09b6-63ec-4753-a242-0bff732ddbc8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80244) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 08:38:10.813511 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG dbcounter [-] [80244] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80244) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:38:10.839453 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 1329/2666] 10.222.0.47 () {58 vars in 957 bytes} [Thu Jul 21 08:38:10 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 08:38:10.849661 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-128f60a7-841d-4c15-ab24-ceaea003e0a2 admin admin] Authenticating user token {{(pid=80245) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:38:10.851377 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.fernet_utils [None req-128f60a7-841d-4c15-ab24-ceaea003e0a2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80245) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 08:38:10.916775 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-128f60a7-841d-4c15-ab24-ceaea003e0a2 admin admin] Validating token access rules against request {{(pid=80245) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:38:10.920564 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e47fa06e-837d-4595-855e-235135a46b6f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20ca3c3bd3774691b3374cfbfec6e494', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4faef13aa50e4e36b6c265fb1398fa14', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80245) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:38:10.921157 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e47fa06e-837d-4595-855e-235135a46b6f service neutron] REQUEST_METHOD: `GET` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:38:10.921444 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e47fa06e-837d-4595-855e-235135a46b6f service neutron] SCRIPT_NAME: `/identity` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:38:10.921724 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e47fa06e-837d-4595-855e-235135a46b6f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:38:10.930741 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e47fa06e-837d-4595-855e-235135a46b6f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:38:10.932204 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e47fa06e-837d-4595-855e-235135a46b6f service neutron] RBAC: Authorization granted {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:38:10.941705 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: [pid: 80245|app: 0|req: 1338/2667] 10.222.0.47 () {60 vars in 1365 bytes} [Thu Jul 21 08:38:10 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 08:38:14.971567 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb26e76b-d46d-4eb5-9982-ad2579c79bff None None] REQUEST_METHOD: `POST` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:38:14.972107 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb26e76b-d46d-4eb5-9982-ad2579c79bff None None] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:38:14.972107 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb26e76b-d46d-4eb5-9982-ad2579c79bff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:38:15.001959 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.auth.core [None req-bb26e76b-d46d-4eb5-9982-ad2579c79bff None None] MFA Rules not processed for user `591ec12e19db47139d3a73484abf2f0d`. Rule list: `[]` (Enabled: `True`). {{(pid=80244) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 08:38:15.004112 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.fernet_utils [None req-bb26e76b-d46d-4eb5-9982-ad2579c79bff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80244) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 08:38:15.074725 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 1330/2668] 10.222.0.47 () {58 vars in 957 bytes} [Thu Jul 21 08:38:14 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 08:38:15.089624 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e47fa06e-837d-4595-855e-235135a46b6f service neutron] Authenticating user token {{(pid=80245) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:38:15.091145 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.fernet_utils [None req-e47fa06e-837d-4595-855e-235135a46b6f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80245) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 08:38:15.167572 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG dbcounter [-] [80245] Writing DB stats keystone:SELECT=93,keystone:DELETE=4,keystone:INSERT=3 {{(pid=80245) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:38:15.204957 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e47fa06e-837d-4595-855e-235135a46b6f service neutron] Validating token access rules against request {{(pid=80245) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:38:15.211118 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b893ea4-4730-49a5-8ca4-0ec11565357a service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2b5b99a36442148fe743269054e991', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4faef13aa50e4e36b6c265fb1398fa14', 'project_domain_id': 'default', 'roles': ['member', 'service', 'ResellerAdmin', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80245) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:38:15.211809 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b893ea4-4730-49a5-8ca4-0ec11565357a service nova] REQUEST_METHOD: `GET` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:38:15.212309 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b893ea4-4730-49a5-8ca4-0ec11565357a service nova] SCRIPT_NAME: `/identity` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:38:15.212410 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b893ea4-4730-49a5-8ca4-0ec11565357a service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:38:15.221011 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b893ea4-4730-49a5-8ca4-0ec11565357a service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:38:15.223714 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b893ea4-4730-49a5-8ca4-0ec11565357a service nova] RBAC: Authorization granted {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:38:15.233675 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: [pid: 80245|app: 0|req: 1339/2669] 10.222.0.47 () {60 vars in 1365 bytes} [Thu Jul 21 08:38:15 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 145 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 08:38:16.589484 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d679477-3156-431d-8d09-acbf5f4a8db0 None None] REQUEST_METHOD: `POST` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:38:16.589991 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d679477-3156-431d-8d09-acbf5f4a8db0 None None] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:38:16.590116 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d679477-3156-431d-8d09-acbf5f4a8db0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:38:16.607990 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.auth.core [None req-2d679477-3156-431d-8d09-acbf5f4a8db0 None None] MFA Rules not processed for user `591ec12e19db47139d3a73484abf2f0d`. Rule list: `[]` (Enabled: `True`). {{(pid=80244) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 08:38:16.609462 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.fernet_utils [None req-2d679477-3156-431d-8d09-acbf5f4a8db0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80244) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 08:38:16.617230 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 1331/2670] 10.222.0.47 () {58 vars in 957 bytes} [Thu Jul 21 08:38:16 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 08:38:17.817885 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b893ea4-4730-49a5-8ca4-0ec11565357a service nova] Authenticating user token {{(pid=80245) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:38:17.819390 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.fernet_utils [None req-6b893ea4-4730-49a5-8ca4-0ec11565357a service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80245) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 08:38:17.925180 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b893ea4-4730-49a5-8ca4-0ec11565357a service nova] Validating token access rules against request {{(pid=80245) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:38:17.932116 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48485a00-ba54-4805-b215-0592a32d0740 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3379b32543474d0a8e897484e85c7d04', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4faef13aa50e4e36b6c265fb1398fa14', 'project_domain_id': 'default', 'roles': ['service', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80245) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:38:17.932940 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48485a00-ba54-4805-b215-0592a32d0740 service placement] REQUEST_METHOD: `GET` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:38:17.933318 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48485a00-ba54-4805-b215-0592a32d0740 service placement] SCRIPT_NAME: `/identity` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:38:17.933619 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48485a00-ba54-4805-b215-0592a32d0740 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:38:17.935467 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.fernet_utils [None req-48485a00-ba54-4805-b215-0592a32d0740 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80245) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 08:38:17.952896 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48485a00-ba54-4805-b215-0592a32d0740 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:38:17.959183 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48485a00-ba54-4805-b215-0592a32d0740 service placement] RBAC: Authorization granted {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:38:17.980047 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: [pid: 80245|app: 0|req: 1340/2671] 10.222.0.47 () {60 vars in 1365 bytes} [Thu Jul 21 08:38:17 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 162 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 08:38:26.602472 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG dbcounter [-] [80244] Writing DB stats keystone:SELECT=44 {{(pid=80244) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:38:27.972115 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG dbcounter [-] [80245] Writing DB stats keystone:SELECT=55 {{(pid=80245) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:38:34.164110 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d82b804e-e175-4528-9d35-a9d70b85cc1a None None] REQUEST_METHOD: `POST` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:38:34.164604 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d82b804e-e175-4528-9d35-a9d70b85cc1a None None] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:38:34.164706 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d82b804e-e175-4528-9d35-a9d70b85cc1a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:38:34.182374 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.auth.core [None req-d82b804e-e175-4528-9d35-a9d70b85cc1a None None] MFA Rules not processed for user `7baa0b2ddca4431b8d006873ca90ab2c`. Rule list: `[]` (Enabled: `True`). {{(pid=80244) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 08:38:34.183610 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.fernet_utils [None req-d82b804e-e175-4528-9d35-a9d70b85cc1a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80244) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 08:38:34.240466 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 1332/2672] 10.222.0.47 () {58 vars in 957 bytes} [Thu Jul 21 08:38:34 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 78 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 08:38:44.233165 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG dbcounter [-] [80244] Writing DB stats keystone:SELECT=25 {{(pid=80244) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:38:44.745787 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48485a00-ba54-4805-b215-0592a32d0740 service placement] Authenticating user token {{(pid=80245) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:38:44.765092 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48485a00-ba54-4805-b215-0592a32d0740 service placement] Validating token access rules against request {{(pid=80245) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:38:44.768791 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-017ef2a5-3e7d-450c-8f6c-51920bcbfe14 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20ca3c3bd3774691b3374cfbfec6e494', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4faef13aa50e4e36b6c265fb1398fa14', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80245) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:38:44.770268 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-017ef2a5-3e7d-450c-8f6c-51920bcbfe14 service neutron] REQUEST_METHOD: `GET` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:38:44.770554 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-017ef2a5-3e7d-450c-8f6c-51920bcbfe14 service neutron] SCRIPT_NAME: `/identity` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:38:44.770834 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-017ef2a5-3e7d-450c-8f6c-51920bcbfe14 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:38:44.772644 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.fernet_utils [None req-017ef2a5-3e7d-450c-8f6c-51920bcbfe14 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80245) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 08:38:44.830505 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-017ef2a5-3e7d-450c-8f6c-51920bcbfe14 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:38:44.831975 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-017ef2a5-3e7d-450c-8f6c-51920bcbfe14 service neutron] RBAC: Authorization granted {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:38:44.841490 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: [pid: 80245|app: 0|req: 1341/2673] 10.222.0.47 () {60 vars in 1365 bytes} [Thu Jul 21 08:38:44 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 08:38:50.060904 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d82b804e-e175-4528-9d35-a9d70b85cc1a None None] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:38:50.070803 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d82b804e-e175-4528-9d35-a9d70b85cc1a None None] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:38:50.074328 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27bb52be-2b4d-4a0a-b4b9-5452c9fb3f07 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20ca3c3bd3774691b3374cfbfec6e494', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4faef13aa50e4e36b6c265fb1398fa14', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:38:50.074974 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27bb52be-2b4d-4a0a-b4b9-5452c9fb3f07 service neutron] REQUEST_METHOD: `GET` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:38:50.075258 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27bb52be-2b4d-4a0a-b4b9-5452c9fb3f07 service neutron] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:38:50.075569 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27bb52be-2b4d-4a0a-b4b9-5452c9fb3f07 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:38:50.078380 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.fernet_utils [None req-27bb52be-2b4d-4a0a-b4b9-5452c9fb3f07 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80244) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 08:38:50.089899 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27bb52be-2b4d-4a0a-b4b9-5452c9fb3f07 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:38:50.091360 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27bb52be-2b4d-4a0a-b4b9-5452c9fb3f07 service neutron] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:38:50.101827 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 1333/2674] 10.222.0.47 () {60 vars in 1365 bytes} [Thu Jul 21 08:38:50 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 21 08:38:54.836748 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG dbcounter [-] [80245] Writing DB stats keystone:SELECT=32 {{(pid=80245) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:38:56.385665 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-017ef2a5-3e7d-450c-8f6c-51920bcbfe14 service neutron] Authenticating user token {{(pid=80245) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:38:56.387253 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.fernet_utils [None req-017ef2a5-3e7d-450c-8f6c-51920bcbfe14 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80245) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 08:38:56.407381 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-017ef2a5-3e7d-450c-8f6c-51920bcbfe14 service neutron] Validating token access rules against request {{(pid=80245) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:38:56.412870 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fd1bf95-476c-408d-99e4-0459ef140b3f service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3379b32543474d0a8e897484e85c7d04', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4faef13aa50e4e36b6c265fb1398fa14', 'project_domain_id': 'default', 'roles': ['service', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80245) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:38:56.413563 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fd1bf95-476c-408d-99e4-0459ef140b3f service placement] REQUEST_METHOD: `GET` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:38:56.413997 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fd1bf95-476c-408d-99e4-0459ef140b3f service placement] SCRIPT_NAME: `/identity` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:38:56.414274 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fd1bf95-476c-408d-99e4-0459ef140b3f service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:38:56.416159 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.fernet_utils [None req-3fd1bf95-476c-408d-99e4-0459ef140b3f service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80245) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 08:38:56.431654 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fd1bf95-476c-408d-99e4-0459ef140b3f service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:38:56.433283 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fd1bf95-476c-408d-99e4-0459ef140b3f service placement] RBAC: Authorization granted {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:38:56.450657 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: [pid: 80245|app: 0|req: 1342/2675] 10.222.0.47 () {60 vars in 1365 bytes} [Thu Jul 21 08:38:56 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 65 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 08:38:56.468424 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27bb52be-2b4d-4a0a-b4b9-5452c9fb3f07 service neutron] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:38:56.472396 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.fernet_utils [None req-27bb52be-2b4d-4a0a-b4b9-5452c9fb3f07 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80244) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 08:38:56.505873 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27bb52be-2b4d-4a0a-b4b9-5452c9fb3f07 service neutron] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:38:56.511087 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0acf1415-bed1-4a3d-9257-d749840ac0b6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '42d15fc0515945acacbbffcf2356c7a5', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:38:56.512031 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0acf1415-bed1-4a3d-9257-d749840ac0b6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:38:56.512312 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0acf1415-bed1-4a3d-9257-d749840ac0b6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:38:56.512589 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0acf1415-bed1-4a3d-9257-d749840ac0b6 admin admin] PATH_INFO: `/v3/users/45688a2eafb840e9b2b4c56bbee69375` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:38:56.513813 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0acf1415-bed1-4a3d-9257-d749840ac0b6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=45688a2eafb840e9b2b4c56bbee69375)` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:38:56.515910 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0acf1415-bed1-4a3d-9257-d749840ac0b6 admin admin] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:38:56.569788 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.notifications [None req-0acf1415-bed1-4a3d-9257-d749840ac0b6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '45688a2eafb840e9b2b4c56bbee69375'} {{(pid=80244) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 08:38:56.577479 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.notifications [None req-0acf1415-bed1-4a3d-9257-d749840ac0b6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '45688a2eafb840e9b2b4c56bbee69375'} {{(pid=80244) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 08:38:56.591006 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.notifications [None req-0acf1415-bed1-4a3d-9257-d749840ac0b6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '45688a2eafb840e9b2b4c56bbee69375'} {{(pid=80244) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 08:38:56.591745 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.notifications [None req-0acf1415-bed1-4a3d-9257-d749840ac0b6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '45688a2eafb840e9b2b4c56bbee69375'} {{(pid=80244) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 08:38:56.599536 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.notifications [None req-0acf1415-bed1-4a3d-9257-d749840ac0b6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '45688a2eafb840e9b2b4c56bbee69375'} {{(pid=80244) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 08:38:56.605739 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 1334/2676] 10.222.0.47 () {58 vars in 1223 bytes} [Thu Jul 21 08:38:56 2022] DELETE /identity/v3/users/45688a2eafb840e9b2b4c56bbee69375 => generated 0 bytes in 138 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 08:38:56.795214 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fd1bf95-476c-408d-99e4-0459ef140b3f service placement] Authenticating user token {{(pid=80245) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:38:56.807116 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fd1bf95-476c-408d-99e4-0459ef140b3f service placement] Validating token access rules against request {{(pid=80245) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:38:56.874979 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f50a09c4-6b34-49ab-97f8-015ec3e1600d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '42d15fc0515945acacbbffcf2356c7a5', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80245) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:38:56.875812 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f50a09c4-6b34-49ab-97f8-015ec3e1600d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:38:56.876099 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f50a09c4-6b34-49ab-97f8-015ec3e1600d admin admin] SCRIPT_NAME: `/identity` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:38:56.876374 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f50a09c4-6b34-49ab-97f8-015ec3e1600d admin admin] PATH_INFO: `/v3/projects/ccb36757791d463e92faf5dbaa0ff13a` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:38:56.877612 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f50a09c4-6b34-49ab-97f8-015ec3e1600d admin admin] RBAC: Authorizing `identity:delete_project(project_id=ccb36757791d463e92faf5dbaa0ff13a)` {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:38:56.879971 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f50a09c4-6b34-49ab-97f8-015ec3e1600d admin admin] RBAC: Authorization granted {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:38:56.926370 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.notifications [None req-f50a09c4-6b34-49ab-97f8-015ec3e1600d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ccb36757791d463e92faf5dbaa0ff13a', 'request_id': 'req-f50a09c4-6b34-49ab-97f8-015ec3e1600d'} {{(pid=80245) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 08:38:56.932442 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.notifications [None req-f50a09c4-6b34-49ab-97f8-015ec3e1600d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ccb36757791d463e92faf5dbaa0ff13a', 'request_id': 'req-f50a09c4-6b34-49ab-97f8-015ec3e1600d'} {{(pid=80245) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 08:38:56.937914 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.notifications [None req-f50a09c4-6b34-49ab-97f8-015ec3e1600d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ccb36757791d463e92faf5dbaa0ff13a', 'request_id': 'req-f50a09c4-6b34-49ab-97f8-015ec3e1600d'} {{(pid=80245) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 08:38:56.950134 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.notifications [None req-f50a09c4-6b34-49ab-97f8-015ec3e1600d admin admin] The token cache is being invalidate because project ccb36757791d463e92faf5dbaa0ff13a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80245) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 21 08:38:56.950423 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.notifications [None req-f50a09c4-6b34-49ab-97f8-015ec3e1600d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80245) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 08:38:56.951063 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.notifications [None req-f50a09c4-6b34-49ab-97f8-015ec3e1600d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80245) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 08:38:56.952702 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: [pid: 80245|app: 0|req: 1343/2677] 10.222.0.47 () {58 vars in 1232 bytes} [Thu Jul 21 08:38:56 2022] DELETE /identity/v3/projects/ccb36757791d463e92faf5dbaa0ff13a => generated 0 bytes in 158 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 08:38:56.957290 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0acf1415-bed1-4a3d-9257-d749840ac0b6 admin admin] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:38:56.958688 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.fernet_utils [None req-0acf1415-bed1-4a3d-9257-d749840ac0b6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80244) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 08:38:56.971522 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0acf1415-bed1-4a3d-9257-d749840ac0b6 admin admin] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:38:57.027577 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5676cc6f-80fe-46b1-8ba6-8927ded98898 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '42d15fc0515945acacbbffcf2356c7a5', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:38:57.028433 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5676cc6f-80fe-46b1-8ba6-8927ded98898 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:38:57.028711 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5676cc6f-80fe-46b1-8ba6-8927ded98898 admin admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:38:57.028983 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5676cc6f-80fe-46b1-8ba6-8927ded98898 admin admin] PATH_INFO: `/v3/users/4512e5eb630943a996a43ea931656696` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:38:57.030161 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5676cc6f-80fe-46b1-8ba6-8927ded98898 admin admin] RBAC: Authorizing `identity:delete_user(user_id=4512e5eb630943a996a43ea931656696)` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:38:57.032089 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5676cc6f-80fe-46b1-8ba6-8927ded98898 admin admin] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:38:57.080047 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.notifications [None req-5676cc6f-80fe-46b1-8ba6-8927ded98898 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4512e5eb630943a996a43ea931656696'} {{(pid=80244) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 08:38:57.089675 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.notifications [None req-5676cc6f-80fe-46b1-8ba6-8927ded98898 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4512e5eb630943a996a43ea931656696'} {{(pid=80244) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 08:38:57.096692 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG dbcounter [-] [80244] Writing DB stats keystone:SELECT=79,keystone:DELETE=19,keystone:INSERT=2 {{(pid=80244) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:38:57.106210 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.notifications [None req-5676cc6f-80fe-46b1-8ba6-8927ded98898 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4512e5eb630943a996a43ea931656696'} {{(pid=80244) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 08:38:57.107038 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.notifications [None req-5676cc6f-80fe-46b1-8ba6-8927ded98898 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4512e5eb630943a996a43ea931656696'} {{(pid=80244) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 08:38:57.115380 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.notifications [None req-5676cc6f-80fe-46b1-8ba6-8927ded98898 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4512e5eb630943a996a43ea931656696'} {{(pid=80244) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 08:38:57.121976 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 1335/2678] 10.222.0.47 () {58 vars in 1223 bytes} [Thu Jul 21 08:38:56 2022] DELETE /identity/v3/users/4512e5eb630943a996a43ea931656696 => generated 0 bytes in 166 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 08:38:57.293554 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f50a09c4-6b34-49ab-97f8-015ec3e1600d admin admin] Authenticating user token {{(pid=80245) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:38:57.295095 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.fernet_utils [None req-f50a09c4-6b34-49ab-97f8-015ec3e1600d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80245) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 08:38:57.308561 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f50a09c4-6b34-49ab-97f8-015ec3e1600d admin admin] Validating token access rules against request {{(pid=80245) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:38:57.368283 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43a36745-cfc2-41ee-88fb-099ed9abdcf4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '42d15fc0515945acacbbffcf2356c7a5', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80245) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:38:57.369162 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43a36745-cfc2-41ee-88fb-099ed9abdcf4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:38:57.369446 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43a36745-cfc2-41ee-88fb-099ed9abdcf4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:38:57.369723 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43a36745-cfc2-41ee-88fb-099ed9abdcf4 admin admin] PATH_INFO: `/v3/projects/56b8aca439294e93b494a7d8c3177e91` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:38:57.370909 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43a36745-cfc2-41ee-88fb-099ed9abdcf4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=56b8aca439294e93b494a7d8c3177e91)` {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:38:57.372870 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43a36745-cfc2-41ee-88fb-099ed9abdcf4 admin admin] RBAC: Authorization granted {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:38:57.398063 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG dbcounter [-] [80245] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80245) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:38:57.430289 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.notifications [None req-43a36745-cfc2-41ee-88fb-099ed9abdcf4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '56b8aca439294e93b494a7d8c3177e91', 'request_id': 'req-43a36745-cfc2-41ee-88fb-099ed9abdcf4'} {{(pid=80245) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 08:38:57.436772 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.notifications [None req-43a36745-cfc2-41ee-88fb-099ed9abdcf4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '56b8aca439294e93b494a7d8c3177e91', 'request_id': 'req-43a36745-cfc2-41ee-88fb-099ed9abdcf4'} {{(pid=80245) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 08:38:57.443221 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.notifications [None req-43a36745-cfc2-41ee-88fb-099ed9abdcf4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '56b8aca439294e93b494a7d8c3177e91', 'request_id': 'req-43a36745-cfc2-41ee-88fb-099ed9abdcf4'} {{(pid=80245) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 08:38:57.457518 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.notifications [None req-43a36745-cfc2-41ee-88fb-099ed9abdcf4 admin admin] The token cache is being invalidate because project 56b8aca439294e93b494a7d8c3177e91 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80245) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 21 08:38:57.457814 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.notifications [None req-43a36745-cfc2-41ee-88fb-099ed9abdcf4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80245) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 08:38:57.458465 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.notifications [None req-43a36745-cfc2-41ee-88fb-099ed9abdcf4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80245) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 08:38:57.460086 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: [pid: 80245|app: 0|req: 1344/2679] 10.222.0.47 () {58 vars in 1232 bytes} [Thu Jul 21 08:38:57 2022] DELETE /identity/v3/projects/56b8aca439294e93b494a7d8c3177e91 => generated 0 bytes in 167 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 08:39:07.118051 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG dbcounter [-] [80244] Writing DB stats keystone:SELECT=9,keystone:DELETE=1 {{(pid=80244) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:39:07.453088 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG dbcounter [-] [80245] Writing DB stats keystone:SELECT=16,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80245) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:39:39.755127 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9ddbdfd-bacb-43c9-9520-dedd148a63e0 None None] REQUEST_METHOD: `POST` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:39:39.755771 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9ddbdfd-bacb-43c9-9520-dedd148a63e0 None None] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:39:39.755883 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9ddbdfd-bacb-43c9-9520-dedd148a63e0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:39:39.773383 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.auth.core [None req-d9ddbdfd-bacb-43c9-9520-dedd148a63e0 None None] MFA Rules not processed for user `7baa0b2ddca4431b8d006873ca90ab2c`. Rule list: `[]` (Enabled: `True`). {{(pid=80244) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 08:39:39.774569 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.fernet_utils [None req-d9ddbdfd-bacb-43c9-9520-dedd148a63e0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80244) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 08:39:39.827756 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 1336/2680] 10.222.0.47 () {58 vars in 957 bytes} [Thu Jul 21 08:39:39 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 74 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 08:39:49.821215 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG dbcounter [-] [80244] Writing DB stats keystone:SELECT=25 {{(pid=80244) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:40:13.260457 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43a36745-cfc2-41ee-88fb-099ed9abdcf4 admin admin] Authenticating user token {{(pid=80245) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:40:13.261972 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.fernet_utils [None req-43a36745-cfc2-41ee-88fb-099ed9abdcf4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80245) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 08:40:13.330200 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43a36745-cfc2-41ee-88fb-099ed9abdcf4 admin admin] Validating token access rules against request {{(pid=80245) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:40:13.334328 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f87b2df7-2ec0-448a-a46f-a9bbb53f1ed5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20ca3c3bd3774691b3374cfbfec6e494', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4faef13aa50e4e36b6c265fb1398fa14', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80245) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:40:13.334974 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f87b2df7-2ec0-448a-a46f-a9bbb53f1ed5 service neutron] REQUEST_METHOD: `GET` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:40:13.335258 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f87b2df7-2ec0-448a-a46f-a9bbb53f1ed5 service neutron] SCRIPT_NAME: `/identity` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:40:13.335560 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f87b2df7-2ec0-448a-a46f-a9bbb53f1ed5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:40:13.337305 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.fernet_utils [None req-f87b2df7-2ec0-448a-a46f-a9bbb53f1ed5 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80245) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 08:40:13.408081 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f87b2df7-2ec0-448a-a46f-a9bbb53f1ed5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:40:13.409511 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f87b2df7-2ec0-448a-a46f-a9bbb53f1ed5 service neutron] RBAC: Authorization granted {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:40:13.420977 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: [pid: 80245|app: 0|req: 1345/2681] 10.222.0.47 () {60 vars in 1365 bytes} [Thu Jul 21 08:40:13 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 161 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 08:40:16.926946 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9ddbdfd-bacb-43c9-9520-dedd148a63e0 None None] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:40:16.928538 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.fernet_utils [None req-d9ddbdfd-bacb-43c9-9520-dedd148a63e0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80244) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 08:40:16.943230 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9ddbdfd-bacb-43c9-9520-dedd148a63e0 None None] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:40:16.946835 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff563e30-b3c7-4414-8e4c-0d7751533cf1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20ca3c3bd3774691b3374cfbfec6e494', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4faef13aa50e4e36b6c265fb1398fa14', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:40:16.947523 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff563e30-b3c7-4414-8e4c-0d7751533cf1 service neutron] REQUEST_METHOD: `GET` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:40:16.947806 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff563e30-b3c7-4414-8e4c-0d7751533cf1 service neutron] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:40:16.948084 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff563e30-b3c7-4414-8e4c-0d7751533cf1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:40:16.949897 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.fernet_utils [None req-ff563e30-b3c7-4414-8e4c-0d7751533cf1 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80244) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 08:40:16.963191 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff563e30-b3c7-4414-8e4c-0d7751533cf1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:40:16.964650 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff563e30-b3c7-4414-8e4c-0d7751533cf1 service neutron] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:40:16.976045 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 1337/2682] 10.222.0.47 () {60 vars in 1365 bytes} [Thu Jul 21 08:40:16 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 51 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 08:40:23.416073 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG dbcounter [-] [80245] Writing DB stats keystone:SELECT=59 {{(pid=80245) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:40:26.970677 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG dbcounter [-] [80244] Writing DB stats keystone:SELECT=6 {{(pid=80244) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:40:57.571609 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f87b2df7-2ec0-448a-a46f-a9bbb53f1ed5 service neutron] Authenticating user token {{(pid=80245) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:40:57.573546 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.fernet_utils [None req-f87b2df7-2ec0-448a-a46f-a9bbb53f1ed5 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80245) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 08:40:57.589137 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f87b2df7-2ec0-448a-a46f-a9bbb53f1ed5 service neutron] Validating token access rules against request {{(pid=80245) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:40:57.595222 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-337014e3-ea0d-44b9-b9a9-8c3e9addcb9f service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2b5b99a36442148fe743269054e991', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4faef13aa50e4e36b6c265fb1398fa14', 'project_domain_id': 'default', 'roles': ['member', 'service', 'ResellerAdmin', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80245) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:40:57.595933 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-337014e3-ea0d-44b9-b9a9-8c3e9addcb9f service nova] REQUEST_METHOD: `GET` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:40:57.596832 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-337014e3-ea0d-44b9-b9a9-8c3e9addcb9f service nova] SCRIPT_NAME: `/identity` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:40:57.597111 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-337014e3-ea0d-44b9-b9a9-8c3e9addcb9f service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:40:57.599760 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.fernet_utils [None req-337014e3-ea0d-44b9-b9a9-8c3e9addcb9f service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80245) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 08:40:57.613725 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-337014e3-ea0d-44b9-b9a9-8c3e9addcb9f service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:40:57.615370 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-337014e3-ea0d-44b9-b9a9-8c3e9addcb9f service nova] RBAC: Authorization granted {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:40:57.628775 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: [pid: 80245|app: 0|req: 1346/2683] 10.222.0.47 () {60 vars in 1365 bytes} [Thu Jul 21 08:40:57 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 58 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 08:41:01.226107 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45289174-2b5d-4a82-a6f4-09a2483a49e8 None None] REQUEST_METHOD: `POST` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:41:01.226680 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45289174-2b5d-4a82-a6f4-09a2483a49e8 None None] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:41:01.226680 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45289174-2b5d-4a82-a6f4-09a2483a49e8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:41:01.244495 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.auth.core [None req-45289174-2b5d-4a82-a6f4-09a2483a49e8 None None] MFA Rules not processed for user `591ec12e19db47139d3a73484abf2f0d`. Rule list: `[]` (Enabled: `True`). {{(pid=80244) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 08:41:01.245700 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.fernet_utils [None req-45289174-2b5d-4a82-a6f4-09a2483a49e8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80244) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 08:41:01.301933 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 1338/2684] 10.222.0.47 () {58 vars in 957 bytes} [Thu Jul 21 08:41:01 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 77 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 21 08:41:01.920299 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-337014e3-ea0d-44b9-b9a9-8c3e9addcb9f service nova] Authenticating user token {{(pid=80245) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:41:01.921824 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.fernet_utils [None req-337014e3-ea0d-44b9-b9a9-8c3e9addcb9f service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80245) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 08:41:01.993913 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-337014e3-ea0d-44b9-b9a9-8c3e9addcb9f service nova] Validating token access rules against request {{(pid=80245) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:41:01.999575 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b2d9b63-4070-490c-b186-61695ddc905b service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3379b32543474d0a8e897484e85c7d04', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4faef13aa50e4e36b6c265fb1398fa14', 'project_domain_id': 'default', 'roles': ['service', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80245) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:41:02.000298 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b2d9b63-4070-490c-b186-61695ddc905b service placement] REQUEST_METHOD: `GET` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:41:02.000581 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b2d9b63-4070-490c-b186-61695ddc905b service placement] SCRIPT_NAME: `/identity` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:41:02.000857 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b2d9b63-4070-490c-b186-61695ddc905b service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:41:02.002886 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.fernet_utils [None req-7b2d9b63-4070-490c-b186-61695ddc905b service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80245) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 08:41:02.014279 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b2d9b63-4070-490c-b186-61695ddc905b service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:41:02.016131 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b2d9b63-4070-490c-b186-61695ddc905b service placement] RBAC: Authorization granted {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:41:02.026399 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: [pid: 80245|app: 0|req: 1347/2685] 10.222.0.47 () {60 vars in 1365 bytes} [Thu Jul 21 08:41:01 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 107 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 08:41:04.118199 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45289174-2b5d-4a82-a6f4-09a2483a49e8 None None] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:41:04.129692 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45289174-2b5d-4a82-a6f4-09a2483a49e8 None None] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:41:04.137082 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adaf9d59-e755-4772-a5d6-930795b9df10 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20ca3c3bd3774691b3374cfbfec6e494', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4faef13aa50e4e36b6c265fb1398fa14', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:41:04.137948 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adaf9d59-e755-4772-a5d6-930795b9df10 service neutron] REQUEST_METHOD: `GET` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:41:04.138237 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adaf9d59-e755-4772-a5d6-930795b9df10 service neutron] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:41:04.138513 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adaf9d59-e755-4772-a5d6-930795b9df10 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:41:04.140843 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.fernet_utils [None req-adaf9d59-e755-4772-a5d6-930795b9df10 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80244) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 08:41:04.151988 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adaf9d59-e755-4772-a5d6-930795b9df10 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:41:04.153055 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adaf9d59-e755-4772-a5d6-930795b9df10 service neutron] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:41:04.162405 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 1339/2686] 10.222.0.47 () {60 vars in 1365 bytes} [Thu Jul 21 08:41:04 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 45 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 08:41:12.022254 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG dbcounter [-] [80245] Writing DB stats keystone:SELECT=38 {{(pid=80245) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:41:14.158907 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG dbcounter [-] [80244] Writing DB stats keystone:SELECT=31 {{(pid=80244) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:41:16.037074 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b2d9b63-4070-490c-b186-61695ddc905b service placement] Authenticating user token {{(pid=80245) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:41:16.047743 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b2d9b63-4070-490c-b186-61695ddc905b service placement] Validating token access rules against request {{(pid=80245) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:41:16.051599 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f3044ef-5c3c-48cd-b62f-c7ce0cb6f950 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20ca3c3bd3774691b3374cfbfec6e494', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4faef13aa50e4e36b6c265fb1398fa14', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80245) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:41:16.052168 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f3044ef-5c3c-48cd-b62f-c7ce0cb6f950 service neutron] REQUEST_METHOD: `GET` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:41:16.052450 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f3044ef-5c3c-48cd-b62f-c7ce0cb6f950 service neutron] SCRIPT_NAME: `/identity` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:41:16.052722 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f3044ef-5c3c-48cd-b62f-c7ce0cb6f950 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:41:16.054480 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.fernet_utils [None req-3f3044ef-5c3c-48cd-b62f-c7ce0cb6f950 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80245) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 08:41:16.126621 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f3044ef-5c3c-48cd-b62f-c7ce0cb6f950 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:41:16.128384 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f3044ef-5c3c-48cd-b62f-c7ce0cb6f950 service neutron] RBAC: Authorization granted {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:41:16.140569 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: [pid: 80245|app: 0|req: 1348/2687] 10.222.0.47 () {60 vars in 1365 bytes} [Thu Jul 21 08:41:16 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 104 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 08:41:17.319313 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adaf9d59-e755-4772-a5d6-930795b9df10 service neutron] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:41:17.320873 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.fernet_utils [None req-adaf9d59-e755-4772-a5d6-930795b9df10 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80244) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 08:41:17.335920 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adaf9d59-e755-4772-a5d6-930795b9df10 service neutron] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:41:17.340985 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d61922cf-8103-4085-8e89-351b66c75c9b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '42d15fc0515945acacbbffcf2356c7a5', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:41:17.341849 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d61922cf-8103-4085-8e89-351b66c75c9b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:41:17.342127 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d61922cf-8103-4085-8e89-351b66c75c9b admin admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:41:17.342398 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d61922cf-8103-4085-8e89-351b66c75c9b admin admin] PATH_INFO: `/v3/users/7baa0b2ddca4431b8d006873ca90ab2c` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:41:17.344008 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d61922cf-8103-4085-8e89-351b66c75c9b admin admin] RBAC: Authorizing `identity:delete_user(user_id=7baa0b2ddca4431b8d006873ca90ab2c)` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:41:17.346147 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d61922cf-8103-4085-8e89-351b66c75c9b admin admin] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:41:17.401302 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.notifications [None req-d61922cf-8103-4085-8e89-351b66c75c9b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7baa0b2ddca4431b8d006873ca90ab2c'} {{(pid=80244) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 08:41:17.410366 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.notifications [None req-d61922cf-8103-4085-8e89-351b66c75c9b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7baa0b2ddca4431b8d006873ca90ab2c'} {{(pid=80244) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 08:41:17.424805 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.notifications [None req-d61922cf-8103-4085-8e89-351b66c75c9b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7baa0b2ddca4431b8d006873ca90ab2c'} {{(pid=80244) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 08:41:17.425576 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.notifications [None req-d61922cf-8103-4085-8e89-351b66c75c9b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7baa0b2ddca4431b8d006873ca90ab2c'} {{(pid=80244) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 08:41:17.433553 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.notifications [None req-d61922cf-8103-4085-8e89-351b66c75c9b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7baa0b2ddca4431b8d006873ca90ab2c'} {{(pid=80244) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 08:41:17.440532 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 1340/2688] 10.222.0.47 () {58 vars in 1223 bytes} [Thu Jul 21 08:41:17 2022] DELETE /identity/v3/users/7baa0b2ddca4431b8d006873ca90ab2c => generated 0 bytes in 122 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 08:41:17.639511 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f3044ef-5c3c-48cd-b62f-c7ce0cb6f950 service neutron] Authenticating user token {{(pid=80245) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:41:17.650170 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f3044ef-5c3c-48cd-b62f-c7ce0cb6f950 service neutron] Validating token access rules against request {{(pid=80245) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:41:17.724345 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97c404b1-abe4-4023-852e-c221d4d21366 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '42d15fc0515945acacbbffcf2356c7a5', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80245) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:41:17.725318 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97c404b1-abe4-4023-852e-c221d4d21366 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:41:17.725699 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97c404b1-abe4-4023-852e-c221d4d21366 admin admin] SCRIPT_NAME: `/identity` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:41:17.726044 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97c404b1-abe4-4023-852e-c221d4d21366 admin admin] PATH_INFO: `/v3/projects/ca704726a8064214b0507190fe51650f` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:41:17.727375 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97c404b1-abe4-4023-852e-c221d4d21366 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ca704726a8064214b0507190fe51650f)` {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:41:17.729429 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97c404b1-abe4-4023-852e-c221d4d21366 admin admin] RBAC: Authorization granted {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:41:17.779097 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.notifications [None req-97c404b1-abe4-4023-852e-c221d4d21366 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ca704726a8064214b0507190fe51650f', 'request_id': 'req-97c404b1-abe4-4023-852e-c221d4d21366'} {{(pid=80245) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 08:41:17.786463 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.notifications [None req-97c404b1-abe4-4023-852e-c221d4d21366 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ca704726a8064214b0507190fe51650f', 'request_id': 'req-97c404b1-abe4-4023-852e-c221d4d21366'} {{(pid=80245) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 08:41:17.791965 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.notifications [None req-97c404b1-abe4-4023-852e-c221d4d21366 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ca704726a8064214b0507190fe51650f', 'request_id': 'req-97c404b1-abe4-4023-852e-c221d4d21366'} {{(pid=80245) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 08:41:17.804543 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.notifications [None req-97c404b1-abe4-4023-852e-c221d4d21366 admin admin] The token cache is being invalidate because project ca704726a8064214b0507190fe51650f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80245) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 21 08:41:17.804829 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.notifications [None req-97c404b1-abe4-4023-852e-c221d4d21366 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80245) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 08:41:17.805496 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.notifications [None req-97c404b1-abe4-4023-852e-c221d4d21366 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80245) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 08:41:17.807041 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: [pid: 80245|app: 0|req: 1349/2689] 10.222.0.47 () {58 vars in 1232 bytes} [Thu Jul 21 08:41:17 2022] DELETE /identity/v3/projects/ca704726a8064214b0507190fe51650f => generated 0 bytes in 168 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 08:41:17.812102 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07b32f8a-fc91-432a-b361-c413a9b9e10c None None] REQUEST_METHOD: `POST` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:41:17.812306 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07b32f8a-fc91-432a-b361-c413a9b9e10c None None] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:41:17.812551 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07b32f8a-fc91-432a-b361-c413a9b9e10c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:41:17.834405 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.auth.core [None req-07b32f8a-fc91-432a-b361-c413a9b9e10c None None] MFA Rules not processed for user `2dce40b8d74c41ec94cc36e6ff05d412`. Rule list: `[]` (Enabled: `True`). {{(pid=80244) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 08:41:17.835574 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.fernet_utils [None req-07b32f8a-fc91-432a-b361-c413a9b9e10c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80244) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 08:41:17.900984 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 1341/2690] 10.222.0.47 () {58 vars in 957 bytes} [Thu Jul 21 08:41:17 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 08:41:17.908750 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edbd0b52-8e30-448f-87e0-e3f1deba525a None None] REQUEST_METHOD: `POST` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:41:17.909045 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edbd0b52-8e30-448f-87e0-e3f1deba525a None None] SCRIPT_NAME: `/identity` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:41:17.909331 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edbd0b52-8e30-448f-87e0-e3f1deba525a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:41:17.927599 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.auth.core [None req-edbd0b52-8e30-448f-87e0-e3f1deba525a None None] MFA Rules not processed for user `2dce40b8d74c41ec94cc36e6ff05d412`. Rule list: `[]` (Enabled: `True`). {{(pid=80245) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 08:41:17.928816 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.fernet_utils [None req-edbd0b52-8e30-448f-87e0-e3f1deba525a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80245) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 08:41:17.936827 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: [pid: 80245|app: 0|req: 1350/2691] 10.222.0.47 () {58 vars in 957 bytes} [Thu Jul 21 08:41:17 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 08:41:17.941505 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07b32f8a-fc91-432a-b361-c413a9b9e10c None None] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:41:17.951273 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07b32f8a-fc91-432a-b361-c413a9b9e10c None None] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:41:17.956520 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b3884e7-b368-4de9-bca8-6b8cf74ce9fc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '42d15fc0515945acacbbffcf2356c7a5', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:41:17.957174 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b3884e7-b368-4de9-bca8-6b8cf74ce9fc admin admin] REQUEST_METHOD: `GET` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:41:17.957458 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b3884e7-b368-4de9-bca8-6b8cf74ce9fc admin admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:41:17.957733 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b3884e7-b368-4de9-bca8-6b8cf74ce9fc admin admin] PATH_INFO: `/v3/domains` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:41:17.958461 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b3884e7-b368-4de9-bca8-6b8cf74ce9fc admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80244) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 08:41:17.958762 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b3884e7-b368-4de9-bca8-6b8cf74ce9fc admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:41:17.960393 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b3884e7-b368-4de9-bca8-6b8cf74ce9fc admin admin] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:41:17.971872 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 1342/2692] 10.222.0.47 () {58 vars in 1152 bytes} [Thu Jul 21 08:41:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 08:41:17.976251 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edbd0b52-8e30-448f-87e0-e3f1deba525a None None] Authenticating user token {{(pid=80245) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:41:17.985537 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edbd0b52-8e30-448f-87e0-e3f1deba525a None None] Validating token access rules against request {{(pid=80245) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:41:17.990261 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16df1c22-bcd3-4483-87bd-e7f1132ba99a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '42d15fc0515945acacbbffcf2356c7a5', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80245) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:41:17.990919 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16df1c22-bcd3-4483-87bd-e7f1132ba99a admin admin] REQUEST_METHOD: `POST` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:41:17.991207 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16df1c22-bcd3-4483-87bd-e7f1132ba99a admin admin] SCRIPT_NAME: `/identity` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:41:17.991538 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16df1c22-bcd3-4483-87bd-e7f1132ba99a admin admin] PATH_INFO: `/v3/projects` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:41:17.992248 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16df1c22-bcd3-4483-87bd-e7f1132ba99a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:41:17.994138 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16df1c22-bcd3-4483-87bd-e7f1132ba99a admin admin] RBAC: Authorization granted {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:41:18.008582 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG dbcounter [-] [80245] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80245) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:41:18.032477 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: [pid: 80245|app: 0|req: 1351/2693] 10.222.0.47 () {60 vars in 1152 bytes} [Thu Jul 21 08:41:17 2022] POST /identity/v3/projects => generated 381 bytes in 57 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 08:41:18.036616 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b3884e7-b368-4de9-bca8-6b8cf74ce9fc admin admin] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:41:18.047359 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b3884e7-b368-4de9-bca8-6b8cf74ce9fc admin admin] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:41:18.104280 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ad21cb8-e576-474f-af01-d248e57e0408 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '42d15fc0515945acacbbffcf2356c7a5', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:41:18.104864 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ad21cb8-e576-474f-af01-d248e57e0408 admin admin] REQUEST_METHOD: `POST` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:41:18.105146 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ad21cb8-e576-474f-af01-d248e57e0408 admin admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:41:18.105422 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ad21cb8-e576-474f-af01-d248e57e0408 admin admin] PATH_INFO: `/v3/users` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:41:18.106116 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ad21cb8-e576-474f-af01-d248e57e0408 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:41:18.108079 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ad21cb8-e576-474f-af01-d248e57e0408 admin admin] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:41:18.114480 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG dbcounter [-] [80244] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80244) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:41:18.129463 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 1343/2694] 10.222.0.47 () {60 vars in 1143 bytes} [Thu Jul 21 08:41:18 2022] POST /identity/v3/users => generated 291 bytes in 93 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 08:41:18.133704 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16df1c22-bcd3-4483-87bd-e7f1132ba99a admin admin] Authenticating user token {{(pid=80245) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:41:18.144850 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16df1c22-bcd3-4483-87bd-e7f1132ba99a admin admin] Validating token access rules against request {{(pid=80245) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:41:18.149580 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8735234-74e0-4ad4-bd3d-c82082a1de92 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '42d15fc0515945acacbbffcf2356c7a5', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80245) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:41:18.150248 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8735234-74e0-4ad4-bd3d-c82082a1de92 admin admin] REQUEST_METHOD: `GET` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:41:18.150535 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8735234-74e0-4ad4-bd3d-c82082a1de92 admin admin] SCRIPT_NAME: `/identity` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:41:18.150814 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8735234-74e0-4ad4-bd3d-c82082a1de92 admin admin] PATH_INFO: `/v3/roles` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:41:18.151574 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8735234-74e0-4ad4-bd3d-c82082a1de92 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:41:18.153193 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8735234-74e0-4ad4-bd3d-c82082a1de92 admin admin] RBAC: Authorization granted {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:41:18.169721 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: [pid: 80245|app: 0|req: 1352/2695] 10.222.0.47 () {58 vars in 1121 bytes} [Thu Jul 21 08:41:18 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 08:41:18.173875 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ad21cb8-e576-474f-af01-d248e57e0408 admin admin] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:41:18.186741 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ad21cb8-e576-474f-af01-d248e57e0408 admin admin] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:41:18.191897 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a043fd17-0fd6-484e-ad4b-8f72a9a70f2b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '42d15fc0515945acacbbffcf2356c7a5', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:41:18.192592 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a043fd17-0fd6-484e-ad4b-8f72a9a70f2b admin admin] REQUEST_METHOD: `GET` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:41:18.192867 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a043fd17-0fd6-484e-ad4b-8f72a9a70f2b admin admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:41:18.193139 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a043fd17-0fd6-484e-ad4b-8f72a9a70f2b admin admin] PATH_INFO: `/v3/roles` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:41:18.193863 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a043fd17-0fd6-484e-ad4b-8f72a9a70f2b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:41:18.196382 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a043fd17-0fd6-484e-ad4b-8f72a9a70f2b admin admin] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:41:18.206316 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 1344/2696] 10.222.0.47 () {58 vars in 1121 bytes} [Thu Jul 21 08:41:18 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 21 08:41:18.210553 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8735234-74e0-4ad4-bd3d-c82082a1de92 admin admin] Authenticating user token {{(pid=80245) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:41:18.221004 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8735234-74e0-4ad4-bd3d-c82082a1de92 admin admin] Validating token access rules against request {{(pid=80245) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:41:18.225786 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e50a6d5-b9cd-493a-ba50-f39ce8923ac4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '42d15fc0515945acacbbffcf2356c7a5', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80245) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:41:18.226678 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e50a6d5-b9cd-493a-ba50-f39ce8923ac4 admin admin] REQUEST_METHOD: `PUT` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:41:18.226967 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e50a6d5-b9cd-493a-ba50-f39ce8923ac4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:41:18.227276 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e50a6d5-b9cd-493a-ba50-f39ce8923ac4 admin admin] PATH_INFO: `/v3/projects/2ef38c4c46b84a8abcd42f4da5400cdd/users/30db7ce3398346b2a16a72406e79c254/roles/3a074738221e4601b5205715ba87d240` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:41:18.247277 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e50a6d5-b9cd-493a-ba50-f39ce8923ac4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2ef38c4c46b84a8abcd42f4da5400cdd, user_id=30db7ce3398346b2a16a72406e79c254, role_id=3a074738221e4601b5205715ba87d240)` {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:41:18.249971 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e50a6d5-b9cd-493a-ba50-f39ce8923ac4 admin admin] RBAC: Authorization granted {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:41:18.264326 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: [pid: 80245|app: 0|req: 1353/2697] 10.222.0.47 () {60 vars in 1482 bytes} [Thu Jul 21 08:41:18 2022] PUT /identity/v3/projects/2ef38c4c46b84a8abcd42f4da5400cdd/users/30db7ce3398346b2a16a72406e79c254/roles/3a074738221e4601b5205715ba87d240 => generated 0 bytes in 54 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 08:41:18.275737 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-724cadb0-dda4-4b36-b336-dabc690227fd None None] REQUEST_METHOD: `POST` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:41:18.276027 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-724cadb0-dda4-4b36-b336-dabc690227fd None None] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:41:18.276310 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-724cadb0-dda4-4b36-b336-dabc690227fd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:41:18.302395 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.auth.core [None req-724cadb0-dda4-4b36-b336-dabc690227fd None None] MFA Rules not processed for user `30db7ce3398346b2a16a72406e79c254`. Rule list: `[]` (Enabled: `True`). {{(pid=80244) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 08:41:18.303653 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.fernet_utils [None req-724cadb0-dda4-4b36-b336-dabc690227fd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80244) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 08:41:18.367681 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 1345/2698] 10.222.0.47 () {58 vars in 957 bytes} [Thu Jul 21 08:41:18 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 08:41:18.372002 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e50a6d5-b9cd-493a-ba50-f39ce8923ac4 admin admin] Authenticating user token {{(pid=80245) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:41:18.432253 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e50a6d5-b9cd-493a-ba50-f39ce8923ac4 admin admin] Validating token access rules against request {{(pid=80245) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:41:18.436816 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4517518f-7208-4750-8cca-455481e37e14 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '42d15fc0515945acacbbffcf2356c7a5', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80245) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:41:18.437465 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4517518f-7208-4750-8cca-455481e37e14 admin admin] REQUEST_METHOD: `POST` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:41:18.437745 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4517518f-7208-4750-8cca-455481e37e14 admin admin] SCRIPT_NAME: `/identity` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:41:18.438016 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4517518f-7208-4750-8cca-455481e37e14 admin admin] PATH_INFO: `/v3/projects` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:41:18.438691 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4517518f-7208-4750-8cca-455481e37e14 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:41:18.441237 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4517518f-7208-4750-8cca-455481e37e14 admin admin] RBAC: Authorization granted {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:41:18.465533 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: [pid: 80245|app: 0|req: 1354/2699] 10.222.0.47 () {60 vars in 1152 bytes} [Thu Jul 21 08:41:18 2022] POST /identity/v3/projects => generated 381 bytes in 94 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 08:41:18.469760 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-724cadb0-dda4-4b36-b336-dabc690227fd None None] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:41:18.480387 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-724cadb0-dda4-4b36-b336-dabc690227fd None None] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:41:18.541892 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e44c9521-a0c3-4811-b512-8348b6b8c276 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '42d15fc0515945acacbbffcf2356c7a5', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:41:18.542570 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e44c9521-a0c3-4811-b512-8348b6b8c276 admin admin] REQUEST_METHOD: `POST` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:41:18.542857 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e44c9521-a0c3-4811-b512-8348b6b8c276 admin admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:41:18.543132 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e44c9521-a0c3-4811-b512-8348b6b8c276 admin admin] PATH_INFO: `/v3/users` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:41:18.543870 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e44c9521-a0c3-4811-b512-8348b6b8c276 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:41:18.545793 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e44c9521-a0c3-4811-b512-8348b6b8c276 admin admin] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:41:18.566268 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 1346/2700] 10.222.0.47 () {60 vars in 1143 bytes} [Thu Jul 21 08:41:18 2022] POST /identity/v3/users => generated 297 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 08:41:18.570994 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4517518f-7208-4750-8cca-455481e37e14 admin admin] Authenticating user token {{(pid=80245) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:41:18.581837 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4517518f-7208-4750-8cca-455481e37e14 admin admin] Validating token access rules against request {{(pid=80245) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:41:18.586451 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4374ab5f-94ab-4b49-980a-45e0c80c56fd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '42d15fc0515945acacbbffcf2356c7a5', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80245) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:41:18.587115 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4374ab5f-94ab-4b49-980a-45e0c80c56fd admin admin] REQUEST_METHOD: `GET` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:41:18.587434 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4374ab5f-94ab-4b49-980a-45e0c80c56fd admin admin] SCRIPT_NAME: `/identity` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:41:18.587716 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4374ab5f-94ab-4b49-980a-45e0c80c56fd admin admin] PATH_INFO: `/v3/roles` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:41:18.588444 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4374ab5f-94ab-4b49-980a-45e0c80c56fd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:41:18.590100 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4374ab5f-94ab-4b49-980a-45e0c80c56fd admin admin] RBAC: Authorization granted {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:41:18.601691 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: [pid: 80245|app: 0|req: 1355/2701] 10.222.0.47 () {58 vars in 1121 bytes} [Thu Jul 21 08:41:18 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 08:41:18.605818 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e44c9521-a0c3-4811-b512-8348b6b8c276 admin admin] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:41:18.616695 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e44c9521-a0c3-4811-b512-8348b6b8c276 admin admin] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:41:18.621339 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d7c7476-7191-4920-9c69-d83be684b96a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '42d15fc0515945acacbbffcf2356c7a5', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:41:18.622233 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d7c7476-7191-4920-9c69-d83be684b96a admin admin] REQUEST_METHOD: `PUT` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:41:18.622517 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d7c7476-7191-4920-9c69-d83be684b96a admin admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:41:18.622791 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d7c7476-7191-4920-9c69-d83be684b96a admin admin] PATH_INFO: `/v3/projects/a0edd3eff07c4227b273e796377f9b7b/users/91ce89fd562c42729fd8456b0a83455c/roles/e29870f6e9e2411a9db0d5bf45f32ca5` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:41:18.638959 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d7c7476-7191-4920-9c69-d83be684b96a admin admin] RBAC: Authorizing `identity:create_grant(project_id=a0edd3eff07c4227b273e796377f9b7b, user_id=91ce89fd562c42729fd8456b0a83455c, role_id=e29870f6e9e2411a9db0d5bf45f32ca5)` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:41:18.641682 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d7c7476-7191-4920-9c69-d83be684b96a admin admin] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:41:18.654827 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 1347/2702] 10.222.0.47 () {60 vars in 1482 bytes} [Thu Jul 21 08:41:18 2022] PUT /identity/v3/projects/a0edd3eff07c4227b273e796377f9b7b/users/91ce89fd562c42729fd8456b0a83455c/roles/e29870f6e9e2411a9db0d5bf45f32ca5 => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 08:41:18.661753 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4374ab5f-94ab-4b49-980a-45e0c80c56fd admin admin] Authenticating user token {{(pid=80245) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:41:18.672171 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4374ab5f-94ab-4b49-980a-45e0c80c56fd admin admin] Validating token access rules against request {{(pid=80245) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:41:18.727532 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e9e8ffa-a6a0-4e9b-a78a-581cfc2e2089 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '42d15fc0515945acacbbffcf2356c7a5', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80245) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:41:18.728206 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e9e8ffa-a6a0-4e9b-a78a-581cfc2e2089 admin admin] REQUEST_METHOD: `GET` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:41:18.728497 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e9e8ffa-a6a0-4e9b-a78a-581cfc2e2089 admin admin] SCRIPT_NAME: `/identity` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:41:18.728771 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e9e8ffa-a6a0-4e9b-a78a-581cfc2e2089 admin admin] PATH_INFO: `/v3/roles` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:41:18.729485 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e9e8ffa-a6a0-4e9b-a78a-581cfc2e2089 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:41:18.731086 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e9e8ffa-a6a0-4e9b-a78a-581cfc2e2089 admin admin] RBAC: Authorization granted {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:41:18.741417 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: [pid: 80245|app: 0|req: 1356/2703] 10.222.0.47 () {58 vars in 1121 bytes} [Thu Jul 21 08:41:18 2022] GET /identity/v3/roles => generated 1420 bytes in 82 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 08:41:18.745667 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d7c7476-7191-4920-9c69-d83be684b96a admin admin] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:41:18.756292 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d7c7476-7191-4920-9c69-d83be684b96a admin admin] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:41:18.760960 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-059779ce-9764-46ff-aa47-c523228a562f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '42d15fc0515945acacbbffcf2356c7a5', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:41:18.761850 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-059779ce-9764-46ff-aa47-c523228a562f admin admin] REQUEST_METHOD: `PUT` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:41:18.762132 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-059779ce-9764-46ff-aa47-c523228a562f admin admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:41:18.762407 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-059779ce-9764-46ff-aa47-c523228a562f admin admin] PATH_INFO: `/v3/projects/a0edd3eff07c4227b273e796377f9b7b/users/91ce89fd562c42729fd8456b0a83455c/roles/e29870f6e9e2411a9db0d5bf45f32ca5` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:41:18.764419 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-059779ce-9764-46ff-aa47-c523228a562f admin admin] RBAC: Authorizing `identity:create_grant(project_id=a0edd3eff07c4227b273e796377f9b7b, user_id=91ce89fd562c42729fd8456b0a83455c, role_id=e29870f6e9e2411a9db0d5bf45f32ca5)` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:41:18.767054 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-059779ce-9764-46ff-aa47-c523228a562f admin admin] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:41:18.777228 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 1348/2704] 10.222.0.47 () {60 vars in 1482 bytes} [Thu Jul 21 08:41:18 2022] PUT /identity/v3/projects/a0edd3eff07c4227b273e796377f9b7b/users/91ce89fd562c42729fd8456b0a83455c/roles/e29870f6e9e2411a9db0d5bf45f32ca5 => generated 0 bytes in 32 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 08:41:18.787870 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed4ac42a-cff2-402c-81e2-5fd1cb2bb5f8 None None] REQUEST_METHOD: `POST` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:41:18.788166 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed4ac42a-cff2-402c-81e2-5fd1cb2bb5f8 None None] SCRIPT_NAME: `/identity` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:41:18.788448 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed4ac42a-cff2-402c-81e2-5fd1cb2bb5f8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:41:18.815000 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.auth.core [None req-ed4ac42a-cff2-402c-81e2-5fd1cb2bb5f8 None None] MFA Rules not processed for user `91ce89fd562c42729fd8456b0a83455c`. Rule list: `[]` (Enabled: `True`). {{(pid=80245) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 08:41:18.816278 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.fernet_utils [None req-ed4ac42a-cff2-402c-81e2-5fd1cb2bb5f8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80245) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 08:41:18.825448 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG dbcounter [-] [80245] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80245) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:41:18.886151 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: [pid: 80245|app: 0|req: 1357/2705] 10.222.0.47 () {58 vars in 957 bytes} [Thu Jul 21 08:41:18 2022] POST /identity/v3/auth/tokens => generated 3315 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 08:41:18.898056 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-059779ce-9764-46ff-aa47-c523228a562f admin admin] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:41:18.899815 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.fernet_utils [None req-059779ce-9764-46ff-aa47-c523228a562f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80244) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 08:41:18.923098 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG dbcounter [-] [80244] Writing DB stats keystone:INSERT=8,keystone:SELECT=92 {{(pid=80244) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:41:18.978969 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-059779ce-9764-46ff-aa47-c523228a562f admin admin] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:41:18.982655 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-302f6de7-0d98-4829-9545-678c00a8e9c3 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4f40410b7674687ab5362ac982248ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4faef13aa50e4e36b6c265fb1398fa14', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:41:18.983299 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-302f6de7-0d98-4829-9545-678c00a8e9c3 service cinder] REQUEST_METHOD: `GET` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:41:18.983608 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-302f6de7-0d98-4829-9545-678c00a8e9c3 service cinder] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:41:18.983882 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-302f6de7-0d98-4829-9545-678c00a8e9c3 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:41:18.994327 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-302f6de7-0d98-4829-9545-678c00a8e9c3 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:41:18.995785 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-302f6de7-0d98-4829-9545-678c00a8e9c3 service cinder] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:41:19.005484 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 1349/2706] 10.222.0.47 () {60 vars in 1365 bytes} [Thu Jul 21 08:41:18 2022] GET /identity/v3/auth/tokens => generated 3315 bytes in 108 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 08:41:19.075357 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed4ac42a-cff2-402c-81e2-5fd1cb2bb5f8 None None] Authenticating user token {{(pid=80245) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:41:19.085482 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed4ac42a-cff2-402c-81e2-5fd1cb2bb5f8 None None] Validating token access rules against request {{(pid=80245) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:41:19.089239 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b75f8dce-260e-4341-8378-617cb9d2f2a8 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4f40410b7674687ab5362ac982248ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4faef13aa50e4e36b6c265fb1398fa14', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80245) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:41:19.089876 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b75f8dce-260e-4341-8378-617cb9d2f2a8 service cinder] REQUEST_METHOD: `GET` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:41:19.090163 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b75f8dce-260e-4341-8378-617cb9d2f2a8 service cinder] SCRIPT_NAME: `/identity` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:41:19.090439 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b75f8dce-260e-4341-8378-617cb9d2f2a8 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:41:19.146251 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b75f8dce-260e-4341-8378-617cb9d2f2a8 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:41:19.147727 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b75f8dce-260e-4341-8378-617cb9d2f2a8 service cinder] RBAC: Authorization granted {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:41:19.156983 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: [pid: 80245|app: 0|req: 1358/2707] 10.222.0.47 () {60 vars in 1365 bytes} [Thu Jul 21 08:41:19 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 82 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 08:41:27.927809 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-302f6de7-0d98-4829-9545-678c00a8e9c3 service cinder] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:41:27.988887 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-302f6de7-0d98-4829-9545-678c00a8e9c3 service cinder] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:41:27.993394 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4b0eec9-3cb3-4111-b3f9-c1fc44fd0b58 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '42d15fc0515945acacbbffcf2356c7a5', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:41:27.994248 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4b0eec9-3cb3-4111-b3f9-c1fc44fd0b58 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:41:27.994527 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4b0eec9-3cb3-4111-b3f9-c1fc44fd0b58 admin admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:41:27.994797 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4b0eec9-3cb3-4111-b3f9-c1fc44fd0b58 admin admin] PATH_INFO: `/v3/users/30db7ce3398346b2a16a72406e79c254` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:41:27.995973 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4b0eec9-3cb3-4111-b3f9-c1fc44fd0b58 admin admin] RBAC: Authorizing `identity:delete_user(user_id=30db7ce3398346b2a16a72406e79c254)` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:41:27.997888 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4b0eec9-3cb3-4111-b3f9-c1fc44fd0b58 admin admin] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:41:28.042913 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.notifications [None req-f4b0eec9-3cb3-4111-b3f9-c1fc44fd0b58 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '30db7ce3398346b2a16a72406e79c254'} {{(pid=80244) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 08:41:28.052230 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.notifications [None req-f4b0eec9-3cb3-4111-b3f9-c1fc44fd0b58 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '30db7ce3398346b2a16a72406e79c254'} {{(pid=80244) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 08:41:28.064698 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.notifications [None req-f4b0eec9-3cb3-4111-b3f9-c1fc44fd0b58 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '30db7ce3398346b2a16a72406e79c254'} {{(pid=80244) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 08:41:28.065347 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.notifications [None req-f4b0eec9-3cb3-4111-b3f9-c1fc44fd0b58 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '30db7ce3398346b2a16a72406e79c254'} {{(pid=80244) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 08:41:28.072316 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.notifications [None req-f4b0eec9-3cb3-4111-b3f9-c1fc44fd0b58 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '30db7ce3398346b2a16a72406e79c254'} {{(pid=80244) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 08:41:28.078309 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 1350/2708] 10.222.0.47 () {58 vars in 1223 bytes} [Thu Jul 21 08:41:27 2022] DELETE /identity/v3/users/30db7ce3398346b2a16a72406e79c254 => generated 0 bytes in 151 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 08:41:28.083522 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb7525db-0220-4db2-95d7-8d023174efd0 None None] REQUEST_METHOD: `POST` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:41:28.084014 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb7525db-0220-4db2-95d7-8d023174efd0 None None] SCRIPT_NAME: `/identity` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:41:28.084133 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb7525db-0220-4db2-95d7-8d023174efd0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:41:28.101440 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.auth.core [None req-fb7525db-0220-4db2-95d7-8d023174efd0 None None] MFA Rules not processed for user `2dce40b8d74c41ec94cc36e6ff05d412`. Rule list: `[]` (Enabled: `True`). {{(pid=80245) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 08:41:28.102660 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.fernet_utils [None req-fb7525db-0220-4db2-95d7-8d023174efd0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80245) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 08:41:28.164105 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: [pid: 80245|app: 0|req: 1359/2709] 10.222.0.47 () {58 vars in 957 bytes} [Thu Jul 21 08:41:28 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 08:41:28.173930 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4b0eec9-3cb3-4111-b3f9-c1fc44fd0b58 admin admin] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:41:28.175380 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.fernet_utils [None req-f4b0eec9-3cb3-4111-b3f9-c1fc44fd0b58 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80244) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 08:41:28.201366 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG dbcounter [-] [80244] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80244) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:41:28.235709 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4b0eec9-3cb3-4111-b3f9-c1fc44fd0b58 admin admin] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:41:28.239487 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fea1c603-7eba-4bdd-abc2-142cc3774ce9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20ca3c3bd3774691b3374cfbfec6e494', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4faef13aa50e4e36b6c265fb1398fa14', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:41:28.240140 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fea1c603-7eba-4bdd-abc2-142cc3774ce9 service neutron] REQUEST_METHOD: `GET` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:41:28.240426 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fea1c603-7eba-4bdd-abc2-142cc3774ce9 service neutron] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:41:28.240704 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fea1c603-7eba-4bdd-abc2-142cc3774ce9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:41:28.249873 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fea1c603-7eba-4bdd-abc2-142cc3774ce9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:41:28.251306 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fea1c603-7eba-4bdd-abc2-142cc3774ce9 service neutron] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:41:28.260975 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 1351/2710] 10.222.0.47 () {60 vars in 1365 bytes} [Thu Jul 21 08:41:28 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 08:41:28.533973 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb7525db-0220-4db2-95d7-8d023174efd0 None None] Authenticating user token {{(pid=80245) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:41:28.535452 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.fernet_utils [None req-fb7525db-0220-4db2-95d7-8d023174efd0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80245) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 08:41:28.550757 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb7525db-0220-4db2-95d7-8d023174efd0 None None] Validating token access rules against request {{(pid=80245) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:41:28.557089 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cd6a4a9-60ab-4793-9c7c-dd599e01e5b3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '42d15fc0515945acacbbffcf2356c7a5', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80245) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:41:28.558058 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cd6a4a9-60ab-4793-9c7c-dd599e01e5b3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:41:28.558238 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cd6a4a9-60ab-4793-9c7c-dd599e01e5b3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:41:28.558553 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cd6a4a9-60ab-4793-9c7c-dd599e01e5b3 admin admin] PATH_INFO: `/v3/projects/2ef38c4c46b84a8abcd42f4da5400cdd` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:41:28.559826 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cd6a4a9-60ab-4793-9c7c-dd599e01e5b3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=2ef38c4c46b84a8abcd42f4da5400cdd)` {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:41:28.561653 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cd6a4a9-60ab-4793-9c7c-dd599e01e5b3 admin admin] RBAC: Authorization granted {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:41:28.605671 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG dbcounter [-] [80245] Writing DB stats keystone:SELECT=97,keystone:DELETE=3 {{(pid=80245) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:41:28.613227 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.notifications [None req-3cd6a4a9-60ab-4793-9c7c-dd599e01e5b3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2ef38c4c46b84a8abcd42f4da5400cdd', 'request_id': 'req-3cd6a4a9-60ab-4793-9c7c-dd599e01e5b3'} {{(pid=80245) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 08:41:28.619812 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.notifications [None req-3cd6a4a9-60ab-4793-9c7c-dd599e01e5b3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2ef38c4c46b84a8abcd42f4da5400cdd', 'request_id': 'req-3cd6a4a9-60ab-4793-9c7c-dd599e01e5b3'} {{(pid=80245) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 08:41:28.625438 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.notifications [None req-3cd6a4a9-60ab-4793-9c7c-dd599e01e5b3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2ef38c4c46b84a8abcd42f4da5400cdd', 'request_id': 'req-3cd6a4a9-60ab-4793-9c7c-dd599e01e5b3'} {{(pid=80245) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 08:41:28.638596 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.notifications [None req-3cd6a4a9-60ab-4793-9c7c-dd599e01e5b3 admin admin] The token cache is being invalidate because project 2ef38c4c46b84a8abcd42f4da5400cdd was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80245) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 21 08:41:28.638888 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.notifications [None req-3cd6a4a9-60ab-4793-9c7c-dd599e01e5b3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80245) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 08:41:28.639544 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.notifications [None req-3cd6a4a9-60ab-4793-9c7c-dd599e01e5b3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80245) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 08:41:28.641092 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: [pid: 80245|app: 0|req: 1360/2711] 10.222.0.47 () {58 vars in 1232 bytes} [Thu Jul 21 08:41:28 2022] DELETE /identity/v3/projects/2ef38c4c46b84a8abcd42f4da5400cdd => generated 0 bytes in 108 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 08:41:28.645726 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fea1c603-7eba-4bdd-abc2-142cc3774ce9 service neutron] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:41:28.655787 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fea1c603-7eba-4bdd-abc2-142cc3774ce9 service neutron] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:41:28.715864 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca8b77a6-4d58-459e-a92a-17c45ad3a421 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '42d15fc0515945acacbbffcf2356c7a5', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:41:28.716648 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca8b77a6-4d58-459e-a92a-17c45ad3a421 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:41:28.716927 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca8b77a6-4d58-459e-a92a-17c45ad3a421 admin admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:41:28.717201 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca8b77a6-4d58-459e-a92a-17c45ad3a421 admin admin] PATH_INFO: `/v3/users/91ce89fd562c42729fd8456b0a83455c` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:41:28.718375 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca8b77a6-4d58-459e-a92a-17c45ad3a421 admin admin] RBAC: Authorizing `identity:delete_user(user_id=91ce89fd562c42729fd8456b0a83455c)` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:41:28.720460 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca8b77a6-4d58-459e-a92a-17c45ad3a421 admin admin] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:41:28.766518 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.notifications [None req-ca8b77a6-4d58-459e-a92a-17c45ad3a421 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '91ce89fd562c42729fd8456b0a83455c'} {{(pid=80244) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 08:41:28.779110 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.notifications [None req-ca8b77a6-4d58-459e-a92a-17c45ad3a421 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '91ce89fd562c42729fd8456b0a83455c'} {{(pid=80244) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 08:41:28.791172 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.notifications [None req-ca8b77a6-4d58-459e-a92a-17c45ad3a421 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '91ce89fd562c42729fd8456b0a83455c'} {{(pid=80244) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 08:41:28.791833 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.notifications [None req-ca8b77a6-4d58-459e-a92a-17c45ad3a421 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '91ce89fd562c42729fd8456b0a83455c'} {{(pid=80244) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 08:41:28.799130 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.notifications [None req-ca8b77a6-4d58-459e-a92a-17c45ad3a421 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '91ce89fd562c42729fd8456b0a83455c'} {{(pid=80244) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 08:41:28.805085 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 1352/2712] 10.222.0.47 () {58 vars in 1223 bytes} [Thu Jul 21 08:41:28 2022] DELETE /identity/v3/users/91ce89fd562c42729fd8456b0a83455c => generated 0 bytes in 160 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 08:41:29.081965 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cd6a4a9-60ab-4793-9c7c-dd599e01e5b3 admin admin] Authenticating user token {{(pid=80245) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:41:29.083559 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.fernet_utils [None req-3cd6a4a9-60ab-4793-9c7c-dd599e01e5b3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80245) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 08:41:29.099073 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cd6a4a9-60ab-4793-9c7c-dd599e01e5b3 admin admin] Validating token access rules against request {{(pid=80245) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:41:29.162602 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ce1c264-877f-4f3e-9d86-b71efd38ea23 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '42d15fc0515945acacbbffcf2356c7a5', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80245) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:41:29.163514 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ce1c264-877f-4f3e-9d86-b71efd38ea23 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:41:29.163810 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ce1c264-877f-4f3e-9d86-b71efd38ea23 admin admin] SCRIPT_NAME: `/identity` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:41:29.164093 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ce1c264-877f-4f3e-9d86-b71efd38ea23 admin admin] PATH_INFO: `/v3/projects/a0edd3eff07c4227b273e796377f9b7b` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:41:29.165394 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ce1c264-877f-4f3e-9d86-b71efd38ea23 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a0edd3eff07c4227b273e796377f9b7b)` {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:41:29.167292 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ce1c264-877f-4f3e-9d86-b71efd38ea23 admin admin] RBAC: Authorization granted {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:41:29.212559 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.notifications [None req-6ce1c264-877f-4f3e-9d86-b71efd38ea23 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a0edd3eff07c4227b273e796377f9b7b', 'request_id': 'req-6ce1c264-877f-4f3e-9d86-b71efd38ea23'} {{(pid=80245) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 08:41:29.218574 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.notifications [None req-6ce1c264-877f-4f3e-9d86-b71efd38ea23 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a0edd3eff07c4227b273e796377f9b7b', 'request_id': 'req-6ce1c264-877f-4f3e-9d86-b71efd38ea23'} {{(pid=80245) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 08:41:29.224722 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.notifications [None req-6ce1c264-877f-4f3e-9d86-b71efd38ea23 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a0edd3eff07c4227b273e796377f9b7b', 'request_id': 'req-6ce1c264-877f-4f3e-9d86-b71efd38ea23'} {{(pid=80245) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 08:41:29.237674 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.notifications [None req-6ce1c264-877f-4f3e-9d86-b71efd38ea23 admin admin] The token cache is being invalidate because project a0edd3eff07c4227b273e796377f9b7b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80245) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 21 08:41:29.237966 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.notifications [None req-6ce1c264-877f-4f3e-9d86-b71efd38ea23 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80245) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 08:41:29.238578 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.notifications [None req-6ce1c264-877f-4f3e-9d86-b71efd38ea23 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80245) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 08:41:29.240162 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: [pid: 80245|app: 0|req: 1361/2713] 10.222.0.47 () {58 vars in 1232 bytes} [Thu Jul 21 08:41:29 2022] DELETE /identity/v3/projects/a0edd3eff07c4227b273e796377f9b7b => generated 0 bytes in 159 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 08:41:29.245234 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caf97e76-75eb-41bf-ae3f-407e35d29fee None None] REQUEST_METHOD: `POST` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:41:29.245936 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caf97e76-75eb-41bf-ae3f-407e35d29fee None None] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:41:29.245936 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caf97e76-75eb-41bf-ae3f-407e35d29fee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:41:29.264725 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.auth.core [None req-caf97e76-75eb-41bf-ae3f-407e35d29fee None None] MFA Rules not processed for user `2dce40b8d74c41ec94cc36e6ff05d412`. Rule list: `[]` (Enabled: `True`). {{(pid=80244) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 08:41:29.265869 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.fernet_utils [None req-caf97e76-75eb-41bf-ae3f-407e35d29fee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80244) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 08:41:29.306624 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG dbcounter [-] [80244] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80244) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:41:29.325946 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 1353/2714] 10.222.0.47 () {58 vars in 957 bytes} [Thu Jul 21 08:41:29 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 21 08:41:29.333842 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c97a0b95-888c-4b24-9cb6-7688172c3c6f None None] REQUEST_METHOD: `POST` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:41:29.334136 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c97a0b95-888c-4b24-9cb6-7688172c3c6f None None] SCRIPT_NAME: `/identity` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:41:29.334420 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c97a0b95-888c-4b24-9cb6-7688172c3c6f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:41:29.352847 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.auth.core [None req-c97a0b95-888c-4b24-9cb6-7688172c3c6f None None] MFA Rules not processed for user `2dce40b8d74c41ec94cc36e6ff05d412`. Rule list: `[]` (Enabled: `True`). {{(pid=80245) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 08:41:29.354043 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.fernet_utils [None req-c97a0b95-888c-4b24-9cb6-7688172c3c6f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80245) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 08:41:29.362552 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: [pid: 80245|app: 0|req: 1362/2715] 10.222.0.47 () {58 vars in 957 bytes} [Thu Jul 21 08:41:29 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 08:41:29.366666 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caf97e76-75eb-41bf-ae3f-407e35d29fee None None] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:41:29.376477 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caf97e76-75eb-41bf-ae3f-407e35d29fee None None] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:41:29.381421 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e4aa027-67df-4807-9a1d-2ddbb17d85d3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '42d15fc0515945acacbbffcf2356c7a5', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:41:29.382079 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e4aa027-67df-4807-9a1d-2ddbb17d85d3 admin admin] REQUEST_METHOD: `GET` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:41:29.382373 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e4aa027-67df-4807-9a1d-2ddbb17d85d3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:41:29.382644 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e4aa027-67df-4807-9a1d-2ddbb17d85d3 admin admin] PATH_INFO: `/v3/domains` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:41:29.383411 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e4aa027-67df-4807-9a1d-2ddbb17d85d3 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80244) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 08:41:29.383709 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e4aa027-67df-4807-9a1d-2ddbb17d85d3 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:41:29.385311 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e4aa027-67df-4807-9a1d-2ddbb17d85d3 admin admin] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:41:29.396964 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 1354/2716] 10.222.0.47 () {58 vars in 1152 bytes} [Thu Jul 21 08:41:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 08:41:29.401208 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c97a0b95-888c-4b24-9cb6-7688172c3c6f None None] Authenticating user token {{(pid=80245) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:41:29.410827 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c97a0b95-888c-4b24-9cb6-7688172c3c6f None None] Validating token access rules against request {{(pid=80245) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:41:29.415475 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49054260-564b-42fc-8a68-14cf94531208 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '42d15fc0515945acacbbffcf2356c7a5', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80245) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:41:29.416124 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49054260-564b-42fc-8a68-14cf94531208 admin admin] REQUEST_METHOD: `POST` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:41:29.416403 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49054260-564b-42fc-8a68-14cf94531208 admin admin] SCRIPT_NAME: `/identity` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:41:29.416676 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49054260-564b-42fc-8a68-14cf94531208 admin admin] PATH_INFO: `/v3/projects` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:41:29.417357 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49054260-564b-42fc-8a68-14cf94531208 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:41:29.419229 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49054260-564b-42fc-8a68-14cf94531208 admin admin] RBAC: Authorization granted {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:41:29.444366 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: [pid: 80245|app: 0|req: 1363/2717] 10.222.0.47 () {60 vars in 1152 bytes} [Thu Jul 21 08:41:29 2022] POST /identity/v3/projects => generated 387 bytes in 44 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 08:41:29.448590 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e4aa027-67df-4807-9a1d-2ddbb17d85d3 admin admin] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:41:29.460776 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e4aa027-67df-4807-9a1d-2ddbb17d85d3 admin admin] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:41:29.519671 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40b0ecb4-942b-4952-b31a-09b239021a16 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '42d15fc0515945acacbbffcf2356c7a5', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:41:29.520260 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40b0ecb4-942b-4952-b31a-09b239021a16 admin admin] REQUEST_METHOD: `POST` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:41:29.520540 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40b0ecb4-942b-4952-b31a-09b239021a16 admin admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:41:29.520812 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40b0ecb4-942b-4952-b31a-09b239021a16 admin admin] PATH_INFO: `/v3/users` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:41:29.521501 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40b0ecb4-942b-4952-b31a-09b239021a16 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:41:29.523415 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40b0ecb4-942b-4952-b31a-09b239021a16 admin admin] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:41:29.543116 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 1355/2718] 10.222.0.47 () {60 vars in 1143 bytes} [Thu Jul 21 08:41:29 2022] POST /identity/v3/users => generated 294 bytes in 95 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 08:41:29.547206 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49054260-564b-42fc-8a68-14cf94531208 admin admin] Authenticating user token {{(pid=80245) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:41:29.557531 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49054260-564b-42fc-8a68-14cf94531208 admin admin] Validating token access rules against request {{(pid=80245) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:41:29.562320 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc36c0e5-fd88-4aeb-a39e-1b3b858add61 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '42d15fc0515945acacbbffcf2356c7a5', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80245) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:41:29.562972 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc36c0e5-fd88-4aeb-a39e-1b3b858add61 admin admin] REQUEST_METHOD: `GET` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:41:29.563257 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc36c0e5-fd88-4aeb-a39e-1b3b858add61 admin admin] SCRIPT_NAME: `/identity` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:41:29.563594 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc36c0e5-fd88-4aeb-a39e-1b3b858add61 admin admin] PATH_INFO: `/v3/roles` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:41:29.564333 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc36c0e5-fd88-4aeb-a39e-1b3b858add61 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:41:29.565961 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc36c0e5-fd88-4aeb-a39e-1b3b858add61 admin admin] RBAC: Authorization granted {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:41:29.576859 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: [pid: 80245|app: 0|req: 1364/2719] 10.222.0.47 () {58 vars in 1121 bytes} [Thu Jul 21 08:41:29 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 08:41:29.581453 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40b0ecb4-942b-4952-b31a-09b239021a16 admin admin] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:41:29.592006 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40b0ecb4-942b-4952-b31a-09b239021a16 admin admin] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:41:29.598008 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b0c347c-f845-49b8-8d09-b55a4a20d14a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '42d15fc0515945acacbbffcf2356c7a5', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:41:29.598726 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b0c347c-f845-49b8-8d09-b55a4a20d14a admin admin] REQUEST_METHOD: `GET` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:41:29.599038 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b0c347c-f845-49b8-8d09-b55a4a20d14a admin admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:41:29.599318 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b0c347c-f845-49b8-8d09-b55a4a20d14a admin admin] PATH_INFO: `/v3/roles` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:41:29.600075 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b0c347c-f845-49b8-8d09-b55a4a20d14a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:41:29.601690 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b0c347c-f845-49b8-8d09-b55a4a20d14a admin admin] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:41:29.611855 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 1356/2720] 10.222.0.47 () {58 vars in 1121 bytes} [Thu Jul 21 08:41:29 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 08:41:29.616178 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc36c0e5-fd88-4aeb-a39e-1b3b858add61 admin admin] Authenticating user token {{(pid=80245) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:41:29.626171 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc36c0e5-fd88-4aeb-a39e-1b3b858add61 admin admin] Validating token access rules against request {{(pid=80245) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:41:29.631122 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77dcce3b-8bb2-4e42-8c81-b950536bcd28 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '42d15fc0515945acacbbffcf2356c7a5', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80245) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:41:29.632047 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77dcce3b-8bb2-4e42-8c81-b950536bcd28 admin admin] REQUEST_METHOD: `PUT` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:41:29.632361 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77dcce3b-8bb2-4e42-8c81-b950536bcd28 admin admin] SCRIPT_NAME: `/identity` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:41:29.632641 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77dcce3b-8bb2-4e42-8c81-b950536bcd28 admin admin] PATH_INFO: `/v3/projects/725bf016b8054e0c8a0995d084b49a1b/users/f0cd60dec81c411a8cf63b8b78812ac3/roles/3a074738221e4601b5205715ba87d240` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:41:29.647602 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77dcce3b-8bb2-4e42-8c81-b950536bcd28 admin admin] RBAC: Authorizing `identity:create_grant(project_id=725bf016b8054e0c8a0995d084b49a1b, user_id=f0cd60dec81c411a8cf63b8b78812ac3, role_id=3a074738221e4601b5205715ba87d240)` {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:41:29.650376 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77dcce3b-8bb2-4e42-8c81-b950536bcd28 admin admin] RBAC: Authorization granted {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:41:29.653403 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG dbcounter [-] [80245] Writing DB stats keystone:SELECT=85,keystone:DELETE=11,keystone:INSERT=4 {{(pid=80245) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:41:29.665915 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: [pid: 80245|app: 0|req: 1365/2721] 10.222.0.47 () {60 vars in 1482 bytes} [Thu Jul 21 08:41:29 2022] PUT /identity/v3/projects/725bf016b8054e0c8a0995d084b49a1b/users/f0cd60dec81c411a8cf63b8b78812ac3/roles/3a074738221e4601b5205715ba87d240 => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 21 08:41:29.678177 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecb0a26a-acf8-49c4-98ba-92e62d507ad9 None None] REQUEST_METHOD: `POST` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:41:29.678555 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecb0a26a-acf8-49c4-98ba-92e62d507ad9 None None] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:41:29.678900 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecb0a26a-acf8-49c4-98ba-92e62d507ad9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:41:29.705575 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.auth.core [None req-ecb0a26a-acf8-49c4-98ba-92e62d507ad9 None None] MFA Rules not processed for user `f0cd60dec81c411a8cf63b8b78812ac3`. Rule list: `[]` (Enabled: `True`). {{(pid=80244) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 08:41:29.707277 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.fernet_utils [None req-ecb0a26a-acf8-49c4-98ba-92e62d507ad9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80244) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 08:41:29.772008 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 1357/2722] 10.222.0.47 () {58 vars in 957 bytes} [Thu Jul 21 08:41:29 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 08:41:29.776390 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77dcce3b-8bb2-4e42-8c81-b950536bcd28 admin admin] Authenticating user token {{(pid=80245) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:41:29.842502 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77dcce3b-8bb2-4e42-8c81-b950536bcd28 admin admin] Validating token access rules against request {{(pid=80245) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:41:29.847396 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89856211-d49b-47e9-8dc5-465323c7a548 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '42d15fc0515945acacbbffcf2356c7a5', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80245) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:41:29.848065 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89856211-d49b-47e9-8dc5-465323c7a548 admin admin] REQUEST_METHOD: `POST` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:41:29.848353 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89856211-d49b-47e9-8dc5-465323c7a548 admin admin] SCRIPT_NAME: `/identity` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:41:29.848634 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89856211-d49b-47e9-8dc5-465323c7a548 admin admin] PATH_INFO: `/v3/projects` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:41:29.849337 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89856211-d49b-47e9-8dc5-465323c7a548 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:41:29.851267 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89856211-d49b-47e9-8dc5-465323c7a548 admin admin] RBAC: Authorization granted {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:41:29.878173 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: [pid: 80245|app: 0|req: 1366/2723] 10.222.0.47 () {60 vars in 1152 bytes} [Thu Jul 21 08:41:29 2022] POST /identity/v3/projects => generated 389 bytes in 102 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 08:41:29.882260 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecb0a26a-acf8-49c4-98ba-92e62d507ad9 None None] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:41:29.893220 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecb0a26a-acf8-49c4-98ba-92e62d507ad9 None None] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:41:29.931432 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG dbcounter [-] [80244] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80244) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:41:29.950676 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60b7ee6e-5ff0-48f1-8895-673ba5c370fd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '42d15fc0515945acacbbffcf2356c7a5', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:41:29.951377 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60b7ee6e-5ff0-48f1-8895-673ba5c370fd admin admin] REQUEST_METHOD: `POST` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:41:29.951669 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60b7ee6e-5ff0-48f1-8895-673ba5c370fd admin admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:41:29.951975 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60b7ee6e-5ff0-48f1-8895-673ba5c370fd admin admin] PATH_INFO: `/v3/users` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:41:29.952682 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60b7ee6e-5ff0-48f1-8895-673ba5c370fd admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:41:29.954642 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60b7ee6e-5ff0-48f1-8895-673ba5c370fd admin admin] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:41:29.974677 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 1358/2724] 10.222.0.47 () {60 vars in 1143 bytes} [Thu Jul 21 08:41:29 2022] POST /identity/v3/users => generated 301 bytes in 93 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 08:41:29.978678 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89856211-d49b-47e9-8dc5-465323c7a548 admin admin] Authenticating user token {{(pid=80245) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:41:29.989427 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89856211-d49b-47e9-8dc5-465323c7a548 admin admin] Validating token access rules against request {{(pid=80245) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:41:29.994289 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3e80e2e-6b82-4cfb-889f-f64f096b4ce8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '42d15fc0515945acacbbffcf2356c7a5', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80245) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:41:29.994988 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3e80e2e-6b82-4cfb-889f-f64f096b4ce8 admin admin] REQUEST_METHOD: `GET` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:41:29.995275 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3e80e2e-6b82-4cfb-889f-f64f096b4ce8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:41:29.995588 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3e80e2e-6b82-4cfb-889f-f64f096b4ce8 admin admin] PATH_INFO: `/v3/roles` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:41:29.996344 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3e80e2e-6b82-4cfb-889f-f64f096b4ce8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:41:29.997948 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3e80e2e-6b82-4cfb-889f-f64f096b4ce8 admin admin] RBAC: Authorization granted {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:41:30.008871 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: [pid: 80245|app: 0|req: 1367/2725] 10.222.0.47 () {58 vars in 1121 bytes} [Thu Jul 21 08:41:29 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 08:41:30.012945 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60b7ee6e-5ff0-48f1-8895-673ba5c370fd admin admin] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:41:30.024088 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60b7ee6e-5ff0-48f1-8895-673ba5c370fd admin admin] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:41:30.028417 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7939b859-06c5-4ba6-bd45-d8b2082f3c04 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '42d15fc0515945acacbbffcf2356c7a5', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:41:30.029305 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7939b859-06c5-4ba6-bd45-d8b2082f3c04 admin admin] REQUEST_METHOD: `PUT` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:41:30.029593 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7939b859-06c5-4ba6-bd45-d8b2082f3c04 admin admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:41:30.029870 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7939b859-06c5-4ba6-bd45-d8b2082f3c04 admin admin] PATH_INFO: `/v3/projects/2813944bf7f74a8d82f3efe894462991/users/2895a0f2d02f4c73ad85e2d5d3d938e4/roles/e29870f6e9e2411a9db0d5bf45f32ca5` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:41:30.045080 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7939b859-06c5-4ba6-bd45-d8b2082f3c04 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2813944bf7f74a8d82f3efe894462991, user_id=2895a0f2d02f4c73ad85e2d5d3d938e4, role_id=e29870f6e9e2411a9db0d5bf45f32ca5)` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:41:30.048359 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7939b859-06c5-4ba6-bd45-d8b2082f3c04 admin admin] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:41:30.062052 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 1359/2726] 10.222.0.47 () {60 vars in 1482 bytes} [Thu Jul 21 08:41:30 2022] PUT /identity/v3/projects/2813944bf7f74a8d82f3efe894462991/users/2895a0f2d02f4c73ad85e2d5d3d938e4/roles/e29870f6e9e2411a9db0d5bf45f32ca5 => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 08:41:30.066020 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3e80e2e-6b82-4cfb-889f-f64f096b4ce8 admin admin] Authenticating user token {{(pid=80245) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:41:30.075954 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3e80e2e-6b82-4cfb-889f-f64f096b4ce8 admin admin] Validating token access rules against request {{(pid=80245) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:41:30.131088 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5a6e892-5186-4417-8c63-a17be455e336 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '42d15fc0515945acacbbffcf2356c7a5', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80245) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:41:30.131801 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5a6e892-5186-4417-8c63-a17be455e336 admin admin] REQUEST_METHOD: `GET` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:41:30.132246 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5a6e892-5186-4417-8c63-a17be455e336 admin admin] SCRIPT_NAME: `/identity` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:41:30.132531 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5a6e892-5186-4417-8c63-a17be455e336 admin admin] PATH_INFO: `/v3/roles` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:41:30.133254 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5a6e892-5186-4417-8c63-a17be455e336 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:41:30.134912 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5a6e892-5186-4417-8c63-a17be455e336 admin admin] RBAC: Authorization granted {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:41:30.144796 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: [pid: 80245|app: 0|req: 1368/2727] 10.222.0.47 () {58 vars in 1121 bytes} [Thu Jul 21 08:41:30 2022] GET /identity/v3/roles => generated 1420 bytes in 79 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 08:41:30.148927 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7939b859-06c5-4ba6-bd45-d8b2082f3c04 admin admin] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:41:30.159556 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7939b859-06c5-4ba6-bd45-d8b2082f3c04 admin admin] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:41:30.164161 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e92fe943-54b6-48a7-910b-f5036df8201f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '42d15fc0515945acacbbffcf2356c7a5', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:41:30.165133 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e92fe943-54b6-48a7-910b-f5036df8201f admin admin] REQUEST_METHOD: `PUT` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:41:30.165416 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e92fe943-54b6-48a7-910b-f5036df8201f admin admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:41:30.165689 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e92fe943-54b6-48a7-910b-f5036df8201f admin admin] PATH_INFO: `/v3/projects/2813944bf7f74a8d82f3efe894462991/users/2895a0f2d02f4c73ad85e2d5d3d938e4/roles/e29870f6e9e2411a9db0d5bf45f32ca5` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:41:30.167825 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e92fe943-54b6-48a7-910b-f5036df8201f admin admin] RBAC: Authorizing `identity:create_grant(project_id=2813944bf7f74a8d82f3efe894462991, user_id=2895a0f2d02f4c73ad85e2d5d3d938e4, role_id=e29870f6e9e2411a9db0d5bf45f32ca5)` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:41:30.170489 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e92fe943-54b6-48a7-910b-f5036df8201f admin admin] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:41:30.181297 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 1360/2728] 10.222.0.47 () {60 vars in 1482 bytes} [Thu Jul 21 08:41:30 2022] PUT /identity/v3/projects/2813944bf7f74a8d82f3efe894462991/users/2895a0f2d02f4c73ad85e2d5d3d938e4/roles/e29870f6e9e2411a9db0d5bf45f32ca5 => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 08:41:30.191193 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adef4bc0-0403-499d-8c5a-546600f3f003 None None] REQUEST_METHOD: `POST` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:41:30.191626 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adef4bc0-0403-499d-8c5a-546600f3f003 None None] SCRIPT_NAME: `/identity` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:41:30.191911 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adef4bc0-0403-499d-8c5a-546600f3f003 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:41:30.217234 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.auth.core [None req-adef4bc0-0403-499d-8c5a-546600f3f003 None None] MFA Rules not processed for user `2895a0f2d02f4c73ad85e2d5d3d938e4`. Rule list: `[]` (Enabled: `True`). {{(pid=80245) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 08:41:30.218447 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.fernet_utils [None req-adef4bc0-0403-499d-8c5a-546600f3f003 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80245) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 08:41:30.269682 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG dbcounter [-] [80245] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80245) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:41:30.289057 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: [pid: 80245|app: 0|req: 1369/2729] 10.222.0.47 () {58 vars in 957 bytes} [Thu Jul 21 08:41:30 2022] POST /identity/v3/auth/tokens => generated 3323 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 08:41:30.300710 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e92fe943-54b6-48a7-910b-f5036df8201f admin admin] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:41:30.302604 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.fernet_utils [None req-e92fe943-54b6-48a7-910b-f5036df8201f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80244) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 08:41:30.360715 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e92fe943-54b6-48a7-910b-f5036df8201f admin admin] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:41:30.364328 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4db7da3c-1426-41f6-9098-c794f68599bb service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4f40410b7674687ab5362ac982248ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4faef13aa50e4e36b6c265fb1398fa14', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:41:30.364993 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4db7da3c-1426-41f6-9098-c794f68599bb service cinder] REQUEST_METHOD: `GET` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:41:30.365272 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4db7da3c-1426-41f6-9098-c794f68599bb service cinder] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:41:30.365543 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4db7da3c-1426-41f6-9098-c794f68599bb service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:41:30.374466 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4db7da3c-1426-41f6-9098-c794f68599bb service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:41:30.375921 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4db7da3c-1426-41f6-9098-c794f68599bb service cinder] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:41:30.385500 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 1361/2730] 10.222.0.47 () {60 vars in 1365 bytes} [Thu Jul 21 08:41:30 2022] GET /identity/v3/auth/tokens => generated 3323 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 08:41:30.513499 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adef4bc0-0403-499d-8c5a-546600f3f003 None None] Authenticating user token {{(pid=80245) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:41:30.574500 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adef4bc0-0403-499d-8c5a-546600f3f003 None None] Validating token access rules against request {{(pid=80245) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:41:30.579552 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4a8bb67-fb52-4d65-abf9-7e4dc6f824b1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '42d15fc0515945acacbbffcf2356c7a5', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80245) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:41:30.580426 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4a8bb67-fb52-4d65-abf9-7e4dc6f824b1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:41:30.580736 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4a8bb67-fb52-4d65-abf9-7e4dc6f824b1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:41:30.581010 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4a8bb67-fb52-4d65-abf9-7e4dc6f824b1 admin admin] PATH_INFO: `/v3/users/f0cd60dec81c411a8cf63b8b78812ac3` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:41:30.582206 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4a8bb67-fb52-4d65-abf9-7e4dc6f824b1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f0cd60dec81c411a8cf63b8b78812ac3)` {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:41:30.584163 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4a8bb67-fb52-4d65-abf9-7e4dc6f824b1 admin admin] RBAC: Authorization granted {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:41:30.631144 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.notifications [None req-c4a8bb67-fb52-4d65-abf9-7e4dc6f824b1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f0cd60dec81c411a8cf63b8b78812ac3'} {{(pid=80245) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 08:41:30.638294 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.notifications [None req-c4a8bb67-fb52-4d65-abf9-7e4dc6f824b1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f0cd60dec81c411a8cf63b8b78812ac3'} {{(pid=80245) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 08:41:30.650465 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.notifications [None req-c4a8bb67-fb52-4d65-abf9-7e4dc6f824b1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f0cd60dec81c411a8cf63b8b78812ac3'} {{(pid=80245) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 08:41:30.657705 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.notifications [None req-c4a8bb67-fb52-4d65-abf9-7e4dc6f824b1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f0cd60dec81c411a8cf63b8b78812ac3'} {{(pid=80245) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 08:41:30.658427 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.notifications [None req-c4a8bb67-fb52-4d65-abf9-7e4dc6f824b1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f0cd60dec81c411a8cf63b8b78812ac3'} {{(pid=80245) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 08:41:30.664588 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: [pid: 80245|app: 0|req: 1370/2731] 10.222.0.47 () {58 vars in 1223 bytes} [Thu Jul 21 08:41:30 2022] DELETE /identity/v3/users/f0cd60dec81c411a8cf63b8b78812ac3 => generated 0 bytes in 152 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 08:41:30.670220 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-443d7c1e-f2ba-4601-bdd9-e86f3a093590 None None] REQUEST_METHOD: `POST` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:41:30.670759 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-443d7c1e-f2ba-4601-bdd9-e86f3a093590 None None] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:41:30.671270 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-443d7c1e-f2ba-4601-bdd9-e86f3a093590 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:41:30.691938 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.auth.core [None req-443d7c1e-f2ba-4601-bdd9-e86f3a093590 None None] MFA Rules not processed for user `2dce40b8d74c41ec94cc36e6ff05d412`. Rule list: `[]` (Enabled: `True`). {{(pid=80244) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 08:41:30.693141 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.fernet_utils [None req-443d7c1e-f2ba-4601-bdd9-e86f3a093590 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80244) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 08:41:30.751872 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 1362/2732] 10.222.0.47 () {58 vars in 957 bytes} [Thu Jul 21 08:41:30 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 08:41:30.761970 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4a8bb67-fb52-4d65-abf9-7e4dc6f824b1 admin admin] Authenticating user token {{(pid=80245) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:41:30.763420 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.fernet_utils [None req-c4a8bb67-fb52-4d65-abf9-7e4dc6f824b1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80245) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 08:41:30.820926 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4a8bb67-fb52-4d65-abf9-7e4dc6f824b1 admin admin] Validating token access rules against request {{(pid=80245) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:41:30.824703 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0151001f-f0b3-419d-a157-51449f7e230e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20ca3c3bd3774691b3374cfbfec6e494', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4faef13aa50e4e36b6c265fb1398fa14', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80245) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:41:30.825297 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0151001f-f0b3-419d-a157-51449f7e230e service neutron] REQUEST_METHOD: `GET` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:41:30.825670 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0151001f-f0b3-419d-a157-51449f7e230e service neutron] SCRIPT_NAME: `/identity` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:41:30.825955 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0151001f-f0b3-419d-a157-51449f7e230e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:41:30.835801 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0151001f-f0b3-419d-a157-51449f7e230e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:41:30.837233 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0151001f-f0b3-419d-a157-51449f7e230e service neutron] RBAC: Authorization granted {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:41:30.846675 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: [pid: 80245|app: 0|req: 1371/2733] 10.222.0.47 () {60 vars in 1365 bytes} [Thu Jul 21 08:41:30 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 08:41:31.120758 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-443d7c1e-f2ba-4601-bdd9-e86f3a093590 None None] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:41:31.122280 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.fernet_utils [None req-443d7c1e-f2ba-4601-bdd9-e86f3a093590 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80244) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 08:41:31.136156 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-443d7c1e-f2ba-4601-bdd9-e86f3a093590 None None] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:41:31.141098 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-720fd273-b591-4fac-86ea-68a9ef9e24e0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '42d15fc0515945acacbbffcf2356c7a5', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:41:31.142048 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-720fd273-b591-4fac-86ea-68a9ef9e24e0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:41:31.142335 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-720fd273-b591-4fac-86ea-68a9ef9e24e0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:41:31.142608 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-720fd273-b591-4fac-86ea-68a9ef9e24e0 admin admin] PATH_INFO: `/v3/projects/725bf016b8054e0c8a0995d084b49a1b` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:41:31.143907 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-720fd273-b591-4fac-86ea-68a9ef9e24e0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=725bf016b8054e0c8a0995d084b49a1b)` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:41:31.145723 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-720fd273-b591-4fac-86ea-68a9ef9e24e0 admin admin] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:41:31.179970 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG dbcounter [-] [80244] Writing DB stats keystone:SELECT=93,keystone:INSERT=5,keystone:DELETE=2 {{(pid=80244) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:41:31.194956 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.notifications [None req-720fd273-b591-4fac-86ea-68a9ef9e24e0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '725bf016b8054e0c8a0995d084b49a1b', 'request_id': 'req-720fd273-b591-4fac-86ea-68a9ef9e24e0'} {{(pid=80244) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 08:41:31.202135 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.notifications [None req-720fd273-b591-4fac-86ea-68a9ef9e24e0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '725bf016b8054e0c8a0995d084b49a1b', 'request_id': 'req-720fd273-b591-4fac-86ea-68a9ef9e24e0'} {{(pid=80244) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 08:41:31.208407 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.notifications [None req-720fd273-b591-4fac-86ea-68a9ef9e24e0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '725bf016b8054e0c8a0995d084b49a1b', 'request_id': 'req-720fd273-b591-4fac-86ea-68a9ef9e24e0'} {{(pid=80244) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 08:41:31.218687 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.notifications [None req-720fd273-b591-4fac-86ea-68a9ef9e24e0 admin admin] The token cache is being invalidate because project 725bf016b8054e0c8a0995d084b49a1b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80244) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 21 08:41:31.218983 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.notifications [None req-720fd273-b591-4fac-86ea-68a9ef9e24e0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80244) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 08:41:31.219742 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.notifications [None req-720fd273-b591-4fac-86ea-68a9ef9e24e0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80244) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 08:41:31.221364 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 1363/2734] 10.222.0.47 () {58 vars in 1232 bytes} [Thu Jul 21 08:41:31 2022] DELETE /identity/v3/projects/725bf016b8054e0c8a0995d084b49a1b => generated 0 bytes in 101 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 08:41:31.225492 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0151001f-f0b3-419d-a157-51449f7e230e service neutron] Authenticating user token {{(pid=80245) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:41:31.236584 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0151001f-f0b3-419d-a157-51449f7e230e service neutron] Validating token access rules against request {{(pid=80245) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:41:31.251722 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG dbcounter [-] [80245] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80245) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:41:31.305520 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c699b61f-f32e-44d5-bb50-3ae20618e255 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '42d15fc0515945acacbbffcf2356c7a5', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80245) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:41:31.306398 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c699b61f-f32e-44d5-bb50-3ae20618e255 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:41:31.306686 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c699b61f-f32e-44d5-bb50-3ae20618e255 admin admin] SCRIPT_NAME: `/identity` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:41:31.306959 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c699b61f-f32e-44d5-bb50-3ae20618e255 admin admin] PATH_INFO: `/v3/users/2895a0f2d02f4c73ad85e2d5d3d938e4` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:41:31.308167 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c699b61f-f32e-44d5-bb50-3ae20618e255 admin admin] RBAC: Authorizing `identity:delete_user(user_id=2895a0f2d02f4c73ad85e2d5d3d938e4)` {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:41:31.310075 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c699b61f-f32e-44d5-bb50-3ae20618e255 admin admin] RBAC: Authorization granted {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:41:31.353385 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.notifications [None req-c699b61f-f32e-44d5-bb50-3ae20618e255 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2895a0f2d02f4c73ad85e2d5d3d938e4'} {{(pid=80245) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 08:41:31.360018 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.notifications [None req-c699b61f-f32e-44d5-bb50-3ae20618e255 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2895a0f2d02f4c73ad85e2d5d3d938e4'} {{(pid=80245) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 08:41:31.372076 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.notifications [None req-c699b61f-f32e-44d5-bb50-3ae20618e255 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2895a0f2d02f4c73ad85e2d5d3d938e4'} {{(pid=80245) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 08:41:31.379631 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.notifications [None req-c699b61f-f32e-44d5-bb50-3ae20618e255 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2895a0f2d02f4c73ad85e2d5d3d938e4'} {{(pid=80245) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 08:41:31.380280 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.notifications [None req-c699b61f-f32e-44d5-bb50-3ae20618e255 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2895a0f2d02f4c73ad85e2d5d3d938e4'} {{(pid=80245) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 08:41:31.386119 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: [pid: 80245|app: 0|req: 1372/2735] 10.222.0.47 () {58 vars in 1223 bytes} [Thu Jul 21 08:41:31 2022] DELETE /identity/v3/users/2895a0f2d02f4c73ad85e2d5d3d938e4 => generated 0 bytes in 161 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 08:41:31.668050 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-720fd273-b591-4fac-86ea-68a9ef9e24e0 admin admin] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:41:31.669615 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.fernet_utils [None req-720fd273-b591-4fac-86ea-68a9ef9e24e0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80244) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 08:41:31.683936 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-720fd273-b591-4fac-86ea-68a9ef9e24e0 admin admin] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:41:31.740979 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09241078-82a6-47af-998a-4778cfd2aa2f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '42d15fc0515945acacbbffcf2356c7a5', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:41:31.741855 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09241078-82a6-47af-998a-4778cfd2aa2f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:41:31.742136 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09241078-82a6-47af-998a-4778cfd2aa2f admin admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:41:31.742406 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09241078-82a6-47af-998a-4778cfd2aa2f admin admin] PATH_INFO: `/v3/projects/2813944bf7f74a8d82f3efe894462991` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:41:31.743610 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09241078-82a6-47af-998a-4778cfd2aa2f admin admin] RBAC: Authorizing `identity:delete_project(project_id=2813944bf7f74a8d82f3efe894462991)` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:41:31.745526 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09241078-82a6-47af-998a-4778cfd2aa2f admin admin] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:41:31.792714 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.notifications [None req-09241078-82a6-47af-998a-4778cfd2aa2f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2813944bf7f74a8d82f3efe894462991', 'request_id': 'req-09241078-82a6-47af-998a-4778cfd2aa2f'} {{(pid=80244) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 08:41:31.800308 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.notifications [None req-09241078-82a6-47af-998a-4778cfd2aa2f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2813944bf7f74a8d82f3efe894462991', 'request_id': 'req-09241078-82a6-47af-998a-4778cfd2aa2f'} {{(pid=80244) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 08:41:31.806486 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.notifications [None req-09241078-82a6-47af-998a-4778cfd2aa2f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2813944bf7f74a8d82f3efe894462991', 'request_id': 'req-09241078-82a6-47af-998a-4778cfd2aa2f'} {{(pid=80244) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 08:41:31.817452 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.notifications [None req-09241078-82a6-47af-998a-4778cfd2aa2f admin admin] The token cache is being invalidate because project 2813944bf7f74a8d82f3efe894462991 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80244) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 21 08:41:31.817774 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.notifications [None req-09241078-82a6-47af-998a-4778cfd2aa2f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80244) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 08:41:31.818419 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.notifications [None req-09241078-82a6-47af-998a-4778cfd2aa2f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80244) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 08:41:31.819995 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 1364/2736] 10.222.0.47 () {58 vars in 1232 bytes} [Thu Jul 21 08:41:31 2022] DELETE /identity/v3/projects/2813944bf7f74a8d82f3efe894462991 => generated 0 bytes in 153 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 08:41:31.824942 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea209fc4-4a29-4016-b57c-ab997419f937 None None] REQUEST_METHOD: `POST` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:41:31.825238 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea209fc4-4a29-4016-b57c-ab997419f937 None None] SCRIPT_NAME: `/identity` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:41:31.825518 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea209fc4-4a29-4016-b57c-ab997419f937 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:41:31.843877 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.auth.core [None req-ea209fc4-4a29-4016-b57c-ab997419f937 None None] MFA Rules not processed for user `2dce40b8d74c41ec94cc36e6ff05d412`. Rule list: `[]` (Enabled: `True`). {{(pid=80245) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 08:41:31.845735 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.fernet_utils [None req-ea209fc4-4a29-4016-b57c-ab997419f937 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80245) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 08:41:31.909792 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: [pid: 80245|app: 0|req: 1373/2737] 10.222.0.47 () {58 vars in 957 bytes} [Thu Jul 21 08:41:31 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 08:41:31.918883 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01122bd4-7bc0-43ce-98a3-6096303052d9 None None] REQUEST_METHOD: `POST` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:41:31.919170 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01122bd4-7bc0-43ce-98a3-6096303052d9 None None] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:41:31.919543 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01122bd4-7bc0-43ce-98a3-6096303052d9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:41:31.940513 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.auth.core [None req-01122bd4-7bc0-43ce-98a3-6096303052d9 None None] MFA Rules not processed for user `2dce40b8d74c41ec94cc36e6ff05d412`. Rule list: `[]` (Enabled: `True`). {{(pid=80244) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 08:41:31.941738 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.fernet_utils [None req-01122bd4-7bc0-43ce-98a3-6096303052d9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80244) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 08:41:31.949549 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 1365/2738] 10.222.0.47 () {58 vars in 957 bytes} [Thu Jul 21 08:41:31 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 08:41:31.954389 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea209fc4-4a29-4016-b57c-ab997419f937 None None] Authenticating user token {{(pid=80245) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:41:31.963998 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea209fc4-4a29-4016-b57c-ab997419f937 None None] Validating token access rules against request {{(pid=80245) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:41:31.968505 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85a70250-4178-456b-9bdb-0057f17cf7c7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '42d15fc0515945acacbbffcf2356c7a5', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80245) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:41:31.969162 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85a70250-4178-456b-9bdb-0057f17cf7c7 admin admin] REQUEST_METHOD: `GET` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:41:31.969441 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85a70250-4178-456b-9bdb-0057f17cf7c7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:41:31.970368 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85a70250-4178-456b-9bdb-0057f17cf7c7 admin admin] PATH_INFO: `/v3/domains` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:41:31.971089 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85a70250-4178-456b-9bdb-0057f17cf7c7 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80245) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 08:41:31.971429 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85a70250-4178-456b-9bdb-0057f17cf7c7 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:41:31.973026 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85a70250-4178-456b-9bdb-0057f17cf7c7 admin admin] RBAC: Authorization granted {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:41:31.984558 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: [pid: 80245|app: 0|req: 1374/2739] 10.222.0.47 () {58 vars in 1152 bytes} [Thu Jul 21 08:41:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 08:41:31.988814 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01122bd4-7bc0-43ce-98a3-6096303052d9 None None] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:41:31.997873 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01122bd4-7bc0-43ce-98a3-6096303052d9 None None] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:41:32.002687 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dbe614e-f72a-464c-963d-6e8308d49267 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '42d15fc0515945acacbbffcf2356c7a5', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:41:32.003344 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dbe614e-f72a-464c-963d-6e8308d49267 admin admin] REQUEST_METHOD: `POST` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:41:32.003631 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dbe614e-f72a-464c-963d-6e8308d49267 admin admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:41:32.003899 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dbe614e-f72a-464c-963d-6e8308d49267 admin admin] PATH_INFO: `/v3/projects` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:41:32.004591 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dbe614e-f72a-464c-963d-6e8308d49267 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:41:32.006462 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dbe614e-f72a-464c-963d-6e8308d49267 admin admin] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:41:32.031988 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 1366/2740] 10.222.0.47 () {60 vars in 1152 bytes} [Thu Jul 21 08:41:31 2022] POST /identity/v3/projects => generated 383 bytes in 44 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 21 08:41:32.036725 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85a70250-4178-456b-9bdb-0057f17cf7c7 admin admin] Authenticating user token {{(pid=80245) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:41:32.046632 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85a70250-4178-456b-9bdb-0057f17cf7c7 admin admin] Validating token access rules against request {{(pid=80245) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:41:32.064674 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG dbcounter [-] [80245] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80245) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:41:32.106754 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44131ad9-08ab-41a1-bc5a-7292b03151e2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '42d15fc0515945acacbbffcf2356c7a5', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80245) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:41:32.107463 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44131ad9-08ab-41a1-bc5a-7292b03151e2 admin admin] REQUEST_METHOD: `POST` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:41:32.107753 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44131ad9-08ab-41a1-bc5a-7292b03151e2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:41:32.108034 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44131ad9-08ab-41a1-bc5a-7292b03151e2 admin admin] PATH_INFO: `/v3/users` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:41:32.108726 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44131ad9-08ab-41a1-bc5a-7292b03151e2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:41:32.110687 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44131ad9-08ab-41a1-bc5a-7292b03151e2 admin admin] RBAC: Authorization granted {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:41:32.130771 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: [pid: 80245|app: 0|req: 1375/2741] 10.222.0.47 () {60 vars in 1143 bytes} [Thu Jul 21 08:41:32 2022] POST /identity/v3/users => generated 292 bytes in 95 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 08:41:32.134897 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dbe614e-f72a-464c-963d-6e8308d49267 admin admin] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:41:32.145747 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dbe614e-f72a-464c-963d-6e8308d49267 admin admin] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:41:32.150401 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca9daf2a-865b-4c7b-9529-fab2638d6b96 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '42d15fc0515945acacbbffcf2356c7a5', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:41:32.151057 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca9daf2a-865b-4c7b-9529-fab2638d6b96 admin admin] REQUEST_METHOD: `GET` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:41:32.151362 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca9daf2a-865b-4c7b-9529-fab2638d6b96 admin admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:41:32.151642 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca9daf2a-865b-4c7b-9529-fab2638d6b96 admin admin] PATH_INFO: `/v3/roles` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:41:32.152365 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca9daf2a-865b-4c7b-9529-fab2638d6b96 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:41:32.153984 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca9daf2a-865b-4c7b-9529-fab2638d6b96 admin admin] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:41:32.164303 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 1367/2742] 10.222.0.47 () {58 vars in 1121 bytes} [Thu Jul 21 08:41:32 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 08:41:32.168508 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44131ad9-08ab-41a1-bc5a-7292b03151e2 admin admin] Authenticating user token {{(pid=80245) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:41:32.178504 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44131ad9-08ab-41a1-bc5a-7292b03151e2 admin admin] Validating token access rules against request {{(pid=80245) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:41:32.183223 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4180284a-8057-4494-a761-3d8bf67ed0fe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '42d15fc0515945acacbbffcf2356c7a5', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80245) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:41:32.183912 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4180284a-8057-4494-a761-3d8bf67ed0fe admin admin] REQUEST_METHOD: `GET` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:41:32.184199 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4180284a-8057-4494-a761-3d8bf67ed0fe admin admin] SCRIPT_NAME: `/identity` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:41:32.184479 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4180284a-8057-4494-a761-3d8bf67ed0fe admin admin] PATH_INFO: `/v3/roles` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:41:32.185203 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4180284a-8057-4494-a761-3d8bf67ed0fe admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:41:32.186788 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4180284a-8057-4494-a761-3d8bf67ed0fe admin admin] RBAC: Authorization granted {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:41:32.197050 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: [pid: 80245|app: 0|req: 1376/2743] 10.222.0.47 () {58 vars in 1121 bytes} [Thu Jul 21 08:41:32 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 08:41:32.201198 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca9daf2a-865b-4c7b-9529-fab2638d6b96 admin admin] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:41:32.211756 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca9daf2a-865b-4c7b-9529-fab2638d6b96 admin admin] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:41:32.216349 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-641a8446-020b-41e2-a081-c6d082fc2db1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '42d15fc0515945acacbbffcf2356c7a5', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:41:32.217213 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-641a8446-020b-41e2-a081-c6d082fc2db1 admin admin] REQUEST_METHOD: `PUT` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:41:32.217491 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-641a8446-020b-41e2-a081-c6d082fc2db1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:41:32.217762 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-641a8446-020b-41e2-a081-c6d082fc2db1 admin admin] PATH_INFO: `/v3/projects/32f22361d9e14522a2a370e465f2cda1/users/6bd55b363a294dc5854385d45c920d33/roles/3a074738221e4601b5205715ba87d240` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:41:32.220825 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG dbcounter [-] [80244] Writing DB stats keystone:SELECT=85,keystone:DELETE=12,keystone:INSERT=3 {{(pid=80244) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:41:32.234825 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-641a8446-020b-41e2-a081-c6d082fc2db1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=32f22361d9e14522a2a370e465f2cda1, user_id=6bd55b363a294dc5854385d45c920d33, role_id=3a074738221e4601b5205715ba87d240)` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:41:32.237662 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-641a8446-020b-41e2-a081-c6d082fc2db1 admin admin] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:41:32.251401 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 1368/2744] 10.222.0.47 () {60 vars in 1482 bytes} [Thu Jul 21 08:41:32 2022] PUT /identity/v3/projects/32f22361d9e14522a2a370e465f2cda1/users/6bd55b363a294dc5854385d45c920d33/roles/3a074738221e4601b5205715ba87d240 => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 08:41:32.311485 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47c88244-47bd-4b03-a6f7-b889dad6800c None None] REQUEST_METHOD: `POST` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:41:32.312031 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47c88244-47bd-4b03-a6f7-b889dad6800c None None] SCRIPT_NAME: `/identity` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:41:32.312031 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47c88244-47bd-4b03-a6f7-b889dad6800c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:41:32.339904 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.auth.core [None req-47c88244-47bd-4b03-a6f7-b889dad6800c None None] MFA Rules not processed for user `6bd55b363a294dc5854385d45c920d33`. Rule list: `[]` (Enabled: `True`). {{(pid=80245) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 08:41:32.341176 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.fernet_utils [None req-47c88244-47bd-4b03-a6f7-b889dad6800c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80245) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 08:41:32.404276 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: [pid: 80245|app: 0|req: 1377/2745] 10.222.0.47 () {58 vars in 957 bytes} [Thu Jul 21 08:41:32 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 08:41:32.408872 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-641a8446-020b-41e2-a081-c6d082fc2db1 admin admin] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:41:32.468847 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-641a8446-020b-41e2-a081-c6d082fc2db1 admin admin] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:41:32.473453 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42a10fe3-960f-41ce-ad19-36e3dbebea26 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '42d15fc0515945acacbbffcf2356c7a5', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:41:32.474111 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42a10fe3-960f-41ce-ad19-36e3dbebea26 admin admin] REQUEST_METHOD: `POST` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:41:32.474391 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42a10fe3-960f-41ce-ad19-36e3dbebea26 admin admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:41:32.474660 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42a10fe3-960f-41ce-ad19-36e3dbebea26 admin admin] PATH_INFO: `/v3/projects` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:41:32.475363 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42a10fe3-960f-41ce-ad19-36e3dbebea26 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:41:32.477266 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42a10fe3-960f-41ce-ad19-36e3dbebea26 admin admin] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:41:32.503144 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 1369/2746] 10.222.0.47 () {60 vars in 1152 bytes} [Thu Jul 21 08:41:32 2022] POST /identity/v3/projects => generated 383 bytes in 95 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 08:41:32.507373 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47c88244-47bd-4b03-a6f7-b889dad6800c None None] Authenticating user token {{(pid=80245) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:41:32.518724 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47c88244-47bd-4b03-a6f7-b889dad6800c None None] Validating token access rules against request {{(pid=80245) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:41:32.574019 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0eb2f7e-8c13-4f14-a1f0-655af68b79d7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '42d15fc0515945acacbbffcf2356c7a5', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80245) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:41:32.574692 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0eb2f7e-8c13-4f14-a1f0-655af68b79d7 admin admin] REQUEST_METHOD: `POST` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:41:32.574973 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0eb2f7e-8c13-4f14-a1f0-655af68b79d7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:41:32.575247 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0eb2f7e-8c13-4f14-a1f0-655af68b79d7 admin admin] PATH_INFO: `/v3/users` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:41:32.575973 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0eb2f7e-8c13-4f14-a1f0-655af68b79d7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:41:32.577865 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0eb2f7e-8c13-4f14-a1f0-655af68b79d7 admin admin] RBAC: Authorization granted {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:41:32.597532 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: [pid: 80245|app: 0|req: 1378/2747] 10.222.0.47 () {60 vars in 1143 bytes} [Thu Jul 21 08:41:32 2022] POST /identity/v3/users => generated 298 bytes in 91 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 08:41:32.601719 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42a10fe3-960f-41ce-ad19-36e3dbebea26 admin admin] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:41:32.612587 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42a10fe3-960f-41ce-ad19-36e3dbebea26 admin admin] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:41:32.617261 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0bed811-c860-40e7-9784-bd5b4e01a4b6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '42d15fc0515945acacbbffcf2356c7a5', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:41:32.617926 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0bed811-c860-40e7-9784-bd5b4e01a4b6 admin admin] REQUEST_METHOD: `GET` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:41:32.618211 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0bed811-c860-40e7-9784-bd5b4e01a4b6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:41:32.618487 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0bed811-c860-40e7-9784-bd5b4e01a4b6 admin admin] PATH_INFO: `/v3/roles` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:41:32.619215 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0bed811-c860-40e7-9784-bd5b4e01a4b6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:41:32.620848 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0bed811-c860-40e7-9784-bd5b4e01a4b6 admin admin] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:41:32.632182 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 1370/2748] 10.222.0.47 () {58 vars in 1121 bytes} [Thu Jul 21 08:41:32 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 08:41:32.636321 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0eb2f7e-8c13-4f14-a1f0-655af68b79d7 admin admin] Authenticating user token {{(pid=80245) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:41:32.646280 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0eb2f7e-8c13-4f14-a1f0-655af68b79d7 admin admin] Validating token access rules against request {{(pid=80245) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:41:32.650907 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62602867-89db-449c-b5a0-105d98be1e74 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '42d15fc0515945acacbbffcf2356c7a5', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80245) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:41:32.651827 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62602867-89db-449c-b5a0-105d98be1e74 admin admin] REQUEST_METHOD: `PUT` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:41:32.652110 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62602867-89db-449c-b5a0-105d98be1e74 admin admin] SCRIPT_NAME: `/identity` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:41:32.652384 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62602867-89db-449c-b5a0-105d98be1e74 admin admin] PATH_INFO: `/v3/projects/be11a72f1c2c4506b29d39df79aafae4/users/932bc5516b2340b19c662f6282c7db6b/roles/e29870f6e9e2411a9db0d5bf45f32ca5` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:41:32.666988 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62602867-89db-449c-b5a0-105d98be1e74 admin admin] RBAC: Authorizing `identity:create_grant(project_id=be11a72f1c2c4506b29d39df79aafae4, user_id=932bc5516b2340b19c662f6282c7db6b, role_id=e29870f6e9e2411a9db0d5bf45f32ca5)` {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:41:32.670166 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62602867-89db-449c-b5a0-105d98be1e74 admin admin] RBAC: Authorization granted {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:41:32.672444 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG dbcounter [-] [80245] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=80245) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:41:32.686076 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: [pid: 80245|app: 0|req: 1379/2749] 10.222.0.47 () {60 vars in 1482 bytes} [Thu Jul 21 08:41:32 2022] PUT /identity/v3/projects/be11a72f1c2c4506b29d39df79aafae4/users/932bc5516b2340b19c662f6282c7db6b/roles/e29870f6e9e2411a9db0d5bf45f32ca5 => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 08:41:32.690550 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0bed811-c860-40e7-9784-bd5b4e01a4b6 admin admin] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:41:32.700720 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0bed811-c860-40e7-9784-bd5b4e01a4b6 admin admin] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:41:32.757892 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8038786-dca4-4ad2-9b49-5aab8c50c5b3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '42d15fc0515945acacbbffcf2356c7a5', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:41:32.758491 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8038786-dca4-4ad2-9b49-5aab8c50c5b3 admin admin] REQUEST_METHOD: `GET` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:41:32.758774 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8038786-dca4-4ad2-9b49-5aab8c50c5b3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:41:32.759047 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8038786-dca4-4ad2-9b49-5aab8c50c5b3 admin admin] PATH_INFO: `/v3/roles` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:41:32.759835 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8038786-dca4-4ad2-9b49-5aab8c50c5b3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:41:32.761438 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8038786-dca4-4ad2-9b49-5aab8c50c5b3 admin admin] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:41:32.771467 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 1371/2750] 10.222.0.47 () {58 vars in 1121 bytes} [Thu Jul 21 08:41:32 2022] GET /identity/v3/roles => generated 1420 bytes in 81 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 08:41:32.775681 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62602867-89db-449c-b5a0-105d98be1e74 admin admin] Authenticating user token {{(pid=80245) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:41:32.785919 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62602867-89db-449c-b5a0-105d98be1e74 admin admin] Validating token access rules against request {{(pid=80245) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:41:32.790491 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ca4ab20-2e63-4d08-aa47-eff9524196e1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '42d15fc0515945acacbbffcf2356c7a5', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80245) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:41:32.791431 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ca4ab20-2e63-4d08-aa47-eff9524196e1 admin admin] REQUEST_METHOD: `PUT` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:41:32.791717 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ca4ab20-2e63-4d08-aa47-eff9524196e1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:41:32.791990 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ca4ab20-2e63-4d08-aa47-eff9524196e1 admin admin] PATH_INFO: `/v3/projects/be11a72f1c2c4506b29d39df79aafae4/users/932bc5516b2340b19c662f6282c7db6b/roles/e29870f6e9e2411a9db0d5bf45f32ca5` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:41:32.793930 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ca4ab20-2e63-4d08-aa47-eff9524196e1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=be11a72f1c2c4506b29d39df79aafae4, user_id=932bc5516b2340b19c662f6282c7db6b, role_id=e29870f6e9e2411a9db0d5bf45f32ca5)` {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:41:32.796588 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ca4ab20-2e63-4d08-aa47-eff9524196e1 admin admin] RBAC: Authorization granted {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:41:32.807426 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: [pid: 80245|app: 0|req: 1380/2751] 10.222.0.47 () {60 vars in 1482 bytes} [Thu Jul 21 08:41:32 2022] PUT /identity/v3/projects/be11a72f1c2c4506b29d39df79aafae4/users/932bc5516b2340b19c662f6282c7db6b/roles/e29870f6e9e2411a9db0d5bf45f32ca5 => generated 0 bytes in 32 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 08:41:32.818135 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e75d7d0-ec60-4353-b4f8-9ebd1453c0ec None None] REQUEST_METHOD: `POST` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:41:32.818425 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e75d7d0-ec60-4353-b4f8-9ebd1453c0ec None None] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:41:32.818705 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e75d7d0-ec60-4353-b4f8-9ebd1453c0ec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:41:32.844347 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.auth.core [None req-1e75d7d0-ec60-4353-b4f8-9ebd1453c0ec None None] MFA Rules not processed for user `932bc5516b2340b19c662f6282c7db6b`. Rule list: `[]` (Enabled: `True`). {{(pid=80244) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 08:41:32.845562 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.fernet_utils [None req-1e75d7d0-ec60-4353-b4f8-9ebd1453c0ec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80244) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 08:41:32.890402 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG dbcounter [-] [80244] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80244) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:41:32.920118 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 1372/2752] 10.222.0.47 () {58 vars in 957 bytes} [Thu Jul 21 08:41:32 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 08:41:32.935477 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ca4ab20-2e63-4d08-aa47-eff9524196e1 admin admin] Authenticating user token {{(pid=80245) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:41:32.937178 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.fernet_utils [None req-0ca4ab20-2e63-4d08-aa47-eff9524196e1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80245) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 08:41:33.000947 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ca4ab20-2e63-4d08-aa47-eff9524196e1 admin admin] Validating token access rules against request {{(pid=80245) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:41:33.004877 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cb40edb-4397-403a-868a-892361591bb1 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4f40410b7674687ab5362ac982248ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4faef13aa50e4e36b6c265fb1398fa14', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80245) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:41:33.005522 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cb40edb-4397-403a-868a-892361591bb1 service cinder] REQUEST_METHOD: `GET` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:41:33.005802 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cb40edb-4397-403a-868a-892361591bb1 service cinder] SCRIPT_NAME: `/identity` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:41:33.006075 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cb40edb-4397-403a-868a-892361591bb1 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:41:33.062594 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cb40edb-4397-403a-868a-892361591bb1 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:41:33.064037 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cb40edb-4397-403a-868a-892361591bb1 service cinder] RBAC: Authorization granted {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:41:33.072874 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: [pid: 80245|app: 0|req: 1381/2753] 10.222.0.47 () {60 vars in 1365 bytes} [Thu Jul 21 08:41:32 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 138 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 08:41:37.222556 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e75d7d0-ec60-4353-b4f8-9ebd1453c0ec None None] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:41:37.223984 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.fernet_utils [None req-1e75d7d0-ec60-4353-b4f8-9ebd1453c0ec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80244) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 08:41:37.236468 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e75d7d0-ec60-4353-b4f8-9ebd1453c0ec None None] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:41:37.240079 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b2b7d39-6f21-4b5f-ac27-803bc1272a69 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4f40410b7674687ab5362ac982248ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4faef13aa50e4e36b6c265fb1398fa14', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:41:37.240721 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b2b7d39-6f21-4b5f-ac27-803bc1272a69 service cinder] REQUEST_METHOD: `GET` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:41:37.241007 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b2b7d39-6f21-4b5f-ac27-803bc1272a69 service cinder] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:41:37.241285 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b2b7d39-6f21-4b5f-ac27-803bc1272a69 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:41:37.250375 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b2b7d39-6f21-4b5f-ac27-803bc1272a69 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:41:37.252716 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b2b7d39-6f21-4b5f-ac27-803bc1272a69 service cinder] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:41:37.262174 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 1373/2754] 10.222.0.47 () {60 vars in 1365 bytes} [Thu Jul 21 08:41:37 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 08:41:42.113924 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cb40edb-4397-403a-868a-892361591bb1 service cinder] Authenticating user token {{(pid=80245) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:41:42.175199 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cb40edb-4397-403a-868a-892361591bb1 service cinder] Validating token access rules against request {{(pid=80245) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:41:42.179815 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4497356-84a6-469d-b2e1-9970318b950b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '42d15fc0515945acacbbffcf2356c7a5', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80245) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:41:42.180730 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4497356-84a6-469d-b2e1-9970318b950b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:41:42.181058 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4497356-84a6-469d-b2e1-9970318b950b admin admin] SCRIPT_NAME: `/identity` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:41:42.181412 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4497356-84a6-469d-b2e1-9970318b950b admin admin] PATH_INFO: `/v3/users/6bd55b363a294dc5854385d45c920d33` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:41:42.182635 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4497356-84a6-469d-b2e1-9970318b950b admin admin] RBAC: Authorizing `identity:delete_user(user_id=6bd55b363a294dc5854385d45c920d33)` {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:41:42.184580 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4497356-84a6-469d-b2e1-9970318b950b admin admin] RBAC: Authorization granted {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:41:42.222790 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG dbcounter [-] [80245] Writing DB stats keystone:INSERT=2,keystone:SELECT=93,keystone:DELETE=5 {{(pid=80245) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:41:42.233264 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.notifications [None req-e4497356-84a6-469d-b2e1-9970318b950b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6bd55b363a294dc5854385d45c920d33'} {{(pid=80245) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 08:41:42.240528 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.notifications [None req-e4497356-84a6-469d-b2e1-9970318b950b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6bd55b363a294dc5854385d45c920d33'} {{(pid=80245) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 08:41:42.253293 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.notifications [None req-e4497356-84a6-469d-b2e1-9970318b950b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6bd55b363a294dc5854385d45c920d33'} {{(pid=80245) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 08:41:42.260163 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.notifications [None req-e4497356-84a6-469d-b2e1-9970318b950b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6bd55b363a294dc5854385d45c920d33'} {{(pid=80245) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 08:41:42.260813 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.notifications [None req-e4497356-84a6-469d-b2e1-9970318b950b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6bd55b363a294dc5854385d45c920d33'} {{(pid=80245) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 08:41:42.267620 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: [pid: 80245|app: 0|req: 1382/2755] 10.222.0.47 () {58 vars in 1223 bytes} [Thu Jul 21 08:41:42 2022] DELETE /identity/v3/users/6bd55b363a294dc5854385d45c920d33 => generated 0 bytes in 154 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 08:41:42.272302 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cfdebaa-fde4-4c4b-8148-f805a89071a9 None None] REQUEST_METHOD: `POST` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:41:42.272868 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cfdebaa-fde4-4c4b-8148-f805a89071a9 None None] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:41:42.272868 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cfdebaa-fde4-4c4b-8148-f805a89071a9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:41:42.290532 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.auth.core [None req-7cfdebaa-fde4-4c4b-8148-f805a89071a9 None None] MFA Rules not processed for user `2dce40b8d74c41ec94cc36e6ff05d412`. Rule list: `[]` (Enabled: `True`). {{(pid=80244) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 08:41:42.291704 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.fernet_utils [None req-7cfdebaa-fde4-4c4b-8148-f805a89071a9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80244) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 08:41:42.353136 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 1374/2756] 10.222.0.47 () {58 vars in 957 bytes} [Thu Jul 21 08:41:42 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 08:41:42.363351 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4497356-84a6-469d-b2e1-9970318b950b admin admin] Authenticating user token {{(pid=80245) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:41:42.364807 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.fernet_utils [None req-e4497356-84a6-469d-b2e1-9970318b950b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80245) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 08:41:42.421358 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4497356-84a6-469d-b2e1-9970318b950b admin admin] Validating token access rules against request {{(pid=80245) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:41:42.425720 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a57041b-5c06-46e6-9d27-a58f93b005ba service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20ca3c3bd3774691b3374cfbfec6e494', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4faef13aa50e4e36b6c265fb1398fa14', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80245) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:41:42.426381 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a57041b-5c06-46e6-9d27-a58f93b005ba service neutron] REQUEST_METHOD: `GET` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:41:42.426669 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a57041b-5c06-46e6-9d27-a58f93b005ba service neutron] SCRIPT_NAME: `/identity` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:41:42.426976 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a57041b-5c06-46e6-9d27-a58f93b005ba service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:41:42.436363 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a57041b-5c06-46e6-9d27-a58f93b005ba service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:41:42.437797 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a57041b-5c06-46e6-9d27-a58f93b005ba service neutron] RBAC: Authorization granted {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:41:42.447828 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: [pid: 80245|app: 0|req: 1383/2757] 10.222.0.47 () {60 vars in 1365 bytes} [Thu Jul 21 08:41:42 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 08:41:42.713532 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cfdebaa-fde4-4c4b-8148-f805a89071a9 None None] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:41:42.715017 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.fernet_utils [None req-7cfdebaa-fde4-4c4b-8148-f805a89071a9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80244) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 08:41:42.727751 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cfdebaa-fde4-4c4b-8148-f805a89071a9 None None] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:41:42.732557 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9222851-d561-4c0b-8151-8629fdfcbcfb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '42d15fc0515945acacbbffcf2356c7a5', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:41:42.733429 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9222851-d561-4c0b-8151-8629fdfcbcfb admin admin] REQUEST_METHOD: `DELETE` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:41:42.733714 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9222851-d561-4c0b-8151-8629fdfcbcfb admin admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:41:42.733991 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9222851-d561-4c0b-8151-8629fdfcbcfb admin admin] PATH_INFO: `/v3/projects/32f22361d9e14522a2a370e465f2cda1` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:41:42.735844 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9222851-d561-4c0b-8151-8629fdfcbcfb admin admin] RBAC: Authorizing `identity:delete_project(project_id=32f22361d9e14522a2a370e465f2cda1)` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:41:42.737842 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9222851-d561-4c0b-8151-8629fdfcbcfb admin admin] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:41:42.786022 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.notifications [None req-a9222851-d561-4c0b-8151-8629fdfcbcfb admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '32f22361d9e14522a2a370e465f2cda1', 'request_id': 'req-a9222851-d561-4c0b-8151-8629fdfcbcfb'} {{(pid=80244) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 08:41:42.793517 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.notifications [None req-a9222851-d561-4c0b-8151-8629fdfcbcfb admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '32f22361d9e14522a2a370e465f2cda1', 'request_id': 'req-a9222851-d561-4c0b-8151-8629fdfcbcfb'} {{(pid=80244) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 08:41:42.800514 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.notifications [None req-a9222851-d561-4c0b-8151-8629fdfcbcfb admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '32f22361d9e14522a2a370e465f2cda1', 'request_id': 'req-a9222851-d561-4c0b-8151-8629fdfcbcfb'} {{(pid=80244) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 08:41:42.810898 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.notifications [None req-a9222851-d561-4c0b-8151-8629fdfcbcfb admin admin] The token cache is being invalidate because project 32f22361d9e14522a2a370e465f2cda1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80244) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 21 08:41:42.811182 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.notifications [None req-a9222851-d561-4c0b-8151-8629fdfcbcfb admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80244) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 08:41:42.811856 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.notifications [None req-a9222851-d561-4c0b-8151-8629fdfcbcfb admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80244) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 08:41:42.813518 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 1375/2758] 10.222.0.47 () {58 vars in 1232 bytes} [Thu Jul 21 08:41:42 2022] DELETE /identity/v3/projects/32f22361d9e14522a2a370e465f2cda1 => generated 0 bytes in 101 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 08:41:42.817509 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a57041b-5c06-46e6-9d27-a58f93b005ba service neutron] Authenticating user token {{(pid=80245) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:41:42.827137 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a57041b-5c06-46e6-9d27-a58f93b005ba service neutron] Validating token access rules against request {{(pid=80245) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:41:42.886951 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-231fd93b-6761-48ae-8bc4-bc05f9800d89 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '42d15fc0515945acacbbffcf2356c7a5', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80245) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:41:42.887868 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-231fd93b-6761-48ae-8bc4-bc05f9800d89 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:41:42.888155 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-231fd93b-6761-48ae-8bc4-bc05f9800d89 admin admin] SCRIPT_NAME: `/identity` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:41:42.888436 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-231fd93b-6761-48ae-8bc4-bc05f9800d89 admin admin] PATH_INFO: `/v3/users/932bc5516b2340b19c662f6282c7db6b` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:41:42.889671 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-231fd93b-6761-48ae-8bc4-bc05f9800d89 admin admin] RBAC: Authorizing `identity:delete_user(user_id=932bc5516b2340b19c662f6282c7db6b)` {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:41:42.891683 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-231fd93b-6761-48ae-8bc4-bc05f9800d89 admin admin] RBAC: Authorization granted {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:41:42.937726 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.notifications [None req-231fd93b-6761-48ae-8bc4-bc05f9800d89 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '932bc5516b2340b19c662f6282c7db6b'} {{(pid=80245) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 08:41:42.945300 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.notifications [None req-231fd93b-6761-48ae-8bc4-bc05f9800d89 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '932bc5516b2340b19c662f6282c7db6b'} {{(pid=80245) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 08:41:42.949777 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG dbcounter [-] [80245] Writing DB stats keystone:SELECT=85,keystone:DELETE=13,keystone:INSERT=2 {{(pid=80245) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:41:42.961386 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.notifications [None req-231fd93b-6761-48ae-8bc4-bc05f9800d89 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '932bc5516b2340b19c662f6282c7db6b'} {{(pid=80245) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 08:41:42.969875 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.notifications [None req-231fd93b-6761-48ae-8bc4-bc05f9800d89 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '932bc5516b2340b19c662f6282c7db6b'} {{(pid=80245) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 08:41:42.970766 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.notifications [None req-231fd93b-6761-48ae-8bc4-bc05f9800d89 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '932bc5516b2340b19c662f6282c7db6b'} {{(pid=80245) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 08:41:42.977950 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: [pid: 80245|app: 0|req: 1384/2759] 10.222.0.47 () {58 vars in 1223 bytes} [Thu Jul 21 08:41:42 2022] DELETE /identity/v3/users/932bc5516b2340b19c662f6282c7db6b => generated 0 bytes in 161 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 08:41:43.249099 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9222851-d561-4c0b-8151-8629fdfcbcfb admin admin] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:41:43.250782 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.fernet_utils [None req-a9222851-d561-4c0b-8151-8629fdfcbcfb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80244) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 08:41:43.264965 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9222851-d561-4c0b-8151-8629fdfcbcfb admin admin] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:41:43.315334 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG dbcounter [-] [80244] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80244) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:41:43.327180 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-737f6e36-e2ba-494a-8589-c1c4210c109f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '42d15fc0515945acacbbffcf2356c7a5', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:41:43.328000 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-737f6e36-e2ba-494a-8589-c1c4210c109f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:41:43.328281 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-737f6e36-e2ba-494a-8589-c1c4210c109f admin admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:41:43.329143 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-737f6e36-e2ba-494a-8589-c1c4210c109f admin admin] PATH_INFO: `/v3/projects/be11a72f1c2c4506b29d39df79aafae4` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:41:43.330350 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-737f6e36-e2ba-494a-8589-c1c4210c109f admin admin] RBAC: Authorizing `identity:delete_project(project_id=be11a72f1c2c4506b29d39df79aafae4)` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:41:43.332294 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-737f6e36-e2ba-494a-8589-c1c4210c109f admin admin] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:41:43.379480 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.notifications [None req-737f6e36-e2ba-494a-8589-c1c4210c109f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'be11a72f1c2c4506b29d39df79aafae4', 'request_id': 'req-737f6e36-e2ba-494a-8589-c1c4210c109f'} {{(pid=80244) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 08:41:43.387186 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.notifications [None req-737f6e36-e2ba-494a-8589-c1c4210c109f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'be11a72f1c2c4506b29d39df79aafae4', 'request_id': 'req-737f6e36-e2ba-494a-8589-c1c4210c109f'} {{(pid=80244) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 08:41:43.394017 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.notifications [None req-737f6e36-e2ba-494a-8589-c1c4210c109f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'be11a72f1c2c4506b29d39df79aafae4', 'request_id': 'req-737f6e36-e2ba-494a-8589-c1c4210c109f'} {{(pid=80244) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 08:41:43.404294 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.notifications [None req-737f6e36-e2ba-494a-8589-c1c4210c109f admin admin] The token cache is being invalidate because project be11a72f1c2c4506b29d39df79aafae4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80244) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 21 08:41:43.404578 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.notifications [None req-737f6e36-e2ba-494a-8589-c1c4210c109f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80244) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 08:41:43.405217 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.notifications [None req-737f6e36-e2ba-494a-8589-c1c4210c109f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80244) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 08:41:43.406865 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 1376/2760] 10.222.0.47 () {58 vars in 1232 bytes} [Thu Jul 21 08:41:43 2022] DELETE /identity/v3/projects/be11a72f1c2c4506b29d39df79aafae4 => generated 0 bytes in 158 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 08:41:43.412646 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe9beed8-9392-4a73-a0e3-ac72bb4bc4b9 None None] REQUEST_METHOD: `POST` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:41:43.413094 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe9beed8-9392-4a73-a0e3-ac72bb4bc4b9 None None] SCRIPT_NAME: `/identity` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:41:43.413191 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe9beed8-9392-4a73-a0e3-ac72bb4bc4b9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:41:43.431885 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.auth.core [None req-fe9beed8-9392-4a73-a0e3-ac72bb4bc4b9 None None] MFA Rules not processed for user `2dce40b8d74c41ec94cc36e6ff05d412`. Rule list: `[]` (Enabled: `True`). {{(pid=80245) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 08:41:43.433087 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.fernet_utils [None req-fe9beed8-9392-4a73-a0e3-ac72bb4bc4b9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80245) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 08:41:43.494124 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: [pid: 80245|app: 0|req: 1385/2761] 10.222.0.47 () {58 vars in 957 bytes} [Thu Jul 21 08:41:43 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 08:41:43.504284 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50dd6963-cbce-4d50-8db0-213956ccbc82 None None] REQUEST_METHOD: `POST` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:41:43.504576 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50dd6963-cbce-4d50-8db0-213956ccbc82 None None] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:41:43.504855 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50dd6963-cbce-4d50-8db0-213956ccbc82 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:41:43.523618 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.auth.core [None req-50dd6963-cbce-4d50-8db0-213956ccbc82 None None] MFA Rules not processed for user `2dce40b8d74c41ec94cc36e6ff05d412`. Rule list: `[]` (Enabled: `True`). {{(pid=80244) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 08:41:43.524827 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.fernet_utils [None req-50dd6963-cbce-4d50-8db0-213956ccbc82 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80244) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 08:41:43.533045 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 1377/2762] 10.222.0.47 () {58 vars in 957 bytes} [Thu Jul 21 08:41:43 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 08:41:43.537265 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe9beed8-9392-4a73-a0e3-ac72bb4bc4b9 None None] Authenticating user token {{(pid=80245) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:41:43.547036 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe9beed8-9392-4a73-a0e3-ac72bb4bc4b9 None None] Validating token access rules against request {{(pid=80245) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:41:43.551783 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79275eab-4397-46da-ac7f-440d4b744db6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '42d15fc0515945acacbbffcf2356c7a5', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80245) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:41:43.552504 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79275eab-4397-46da-ac7f-440d4b744db6 admin admin] REQUEST_METHOD: `GET` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:41:43.552792 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79275eab-4397-46da-ac7f-440d4b744db6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:41:43.553070 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79275eab-4397-46da-ac7f-440d4b744db6 admin admin] PATH_INFO: `/v3/domains` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:41:43.553879 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79275eab-4397-46da-ac7f-440d4b744db6 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80245) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 08:41:43.554193 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79275eab-4397-46da-ac7f-440d4b744db6 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:41:43.555840 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79275eab-4397-46da-ac7f-440d4b744db6 admin admin] RBAC: Authorization granted {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:41:43.569136 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: [pid: 80245|app: 0|req: 1386/2763] 10.222.0.47 () {58 vars in 1152 bytes} [Thu Jul 21 08:41:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 08:41:43.574398 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50dd6963-cbce-4d50-8db0-213956ccbc82 None None] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:41:43.584354 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50dd6963-cbce-4d50-8db0-213956ccbc82 None None] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:41:43.588993 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b52c06ed-ded2-44af-a2d1-b0f8148633d5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '42d15fc0515945acacbbffcf2356c7a5', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:41:43.589651 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b52c06ed-ded2-44af-a2d1-b0f8148633d5 admin admin] REQUEST_METHOD: `POST` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:41:43.589933 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b52c06ed-ded2-44af-a2d1-b0f8148633d5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:41:43.590209 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b52c06ed-ded2-44af-a2d1-b0f8148633d5 admin admin] PATH_INFO: `/v3/projects` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:41:43.590895 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b52c06ed-ded2-44af-a2d1-b0f8148633d5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:41:43.592836 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b52c06ed-ded2-44af-a2d1-b0f8148633d5 admin admin] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:41:43.618147 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 1378/2764] 10.222.0.47 () {60 vars in 1152 bytes} [Thu Jul 21 08:41:43 2022] POST /identity/v3/projects => generated 369 bytes in 44 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 08:41:43.622288 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79275eab-4397-46da-ac7f-440d4b744db6 admin admin] Authenticating user token {{(pid=80245) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:41:43.633085 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79275eab-4397-46da-ac7f-440d4b744db6 admin admin] Validating token access rules against request {{(pid=80245) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:41:43.688589 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa454186-ab62-4193-ad7c-bf6a77feb70e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '42d15fc0515945acacbbffcf2356c7a5', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80245) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:41:43.689259 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa454186-ab62-4193-ad7c-bf6a77feb70e admin admin] REQUEST_METHOD: `POST` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:41:43.689549 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa454186-ab62-4193-ad7c-bf6a77feb70e admin admin] SCRIPT_NAME: `/identity` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:41:43.689831 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa454186-ab62-4193-ad7c-bf6a77feb70e admin admin] PATH_INFO: `/v3/users` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:41:43.690530 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa454186-ab62-4193-ad7c-bf6a77feb70e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:41:43.692497 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa454186-ab62-4193-ad7c-bf6a77feb70e admin admin] RBAC: Authorization granted {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:41:43.712527 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: [pid: 80245|app: 0|req: 1387/2765] 10.222.0.47 () {60 vars in 1143 bytes} [Thu Jul 21 08:41:43 2022] POST /identity/v3/users => generated 285 bytes in 91 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 08:41:43.716851 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b52c06ed-ded2-44af-a2d1-b0f8148633d5 admin admin] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:41:43.727621 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b52c06ed-ded2-44af-a2d1-b0f8148633d5 admin admin] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:41:43.732289 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d7a7d0b-00eb-45d9-a849-dc11cf991344 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '42d15fc0515945acacbbffcf2356c7a5', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:41:43.732949 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d7a7d0b-00eb-45d9-a849-dc11cf991344 admin admin] REQUEST_METHOD: `GET` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:41:43.733236 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d7a7d0b-00eb-45d9-a849-dc11cf991344 admin admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:41:43.733510 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d7a7d0b-00eb-45d9-a849-dc11cf991344 admin admin] PATH_INFO: `/v3/roles` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:41:43.734231 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d7a7d0b-00eb-45d9-a849-dc11cf991344 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:41:43.735865 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d7a7d0b-00eb-45d9-a849-dc11cf991344 admin admin] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:41:43.746414 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 1379/2766] 10.222.0.47 () {58 vars in 1121 bytes} [Thu Jul 21 08:41:43 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 08:41:43.750969 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa454186-ab62-4193-ad7c-bf6a77feb70e admin admin] Authenticating user token {{(pid=80245) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:41:43.761133 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa454186-ab62-4193-ad7c-bf6a77feb70e admin admin] Validating token access rules against request {{(pid=80245) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:41:43.765679 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f42b4c4d-be72-4dd8-b135-aa4d24794baa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '42d15fc0515945acacbbffcf2356c7a5', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80245) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:41:43.766970 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f42b4c4d-be72-4dd8-b135-aa4d24794baa admin admin] REQUEST_METHOD: `GET` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:41:43.767253 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f42b4c4d-be72-4dd8-b135-aa4d24794baa admin admin] SCRIPT_NAME: `/identity` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:41:43.767556 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f42b4c4d-be72-4dd8-b135-aa4d24794baa admin admin] PATH_INFO: `/v3/roles` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:41:43.768272 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f42b4c4d-be72-4dd8-b135-aa4d24794baa admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:41:43.769854 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f42b4c4d-be72-4dd8-b135-aa4d24794baa admin admin] RBAC: Authorization granted {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:41:43.779566 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: [pid: 80245|app: 0|req: 1388/2767] 10.222.0.47 () {58 vars in 1121 bytes} [Thu Jul 21 08:41:43 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 08:41:43.783899 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d7a7d0b-00eb-45d9-a849-dc11cf991344 admin admin] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:41:43.793929 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d7a7d0b-00eb-45d9-a849-dc11cf991344 admin admin] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:41:43.799205 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7638e698-de68-4c83-b914-ff4f79d1259e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '42d15fc0515945acacbbffcf2356c7a5', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:41:43.800144 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7638e698-de68-4c83-b914-ff4f79d1259e admin admin] REQUEST_METHOD: `PUT` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:41:43.800433 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7638e698-de68-4c83-b914-ff4f79d1259e admin admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:41:43.800713 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7638e698-de68-4c83-b914-ff4f79d1259e admin admin] PATH_INFO: `/v3/projects/9b16b936342343e3be044cf9cab78b2d/users/4f807920625945f5b08541616816a25a/roles/3a074738221e4601b5205715ba87d240` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:41:43.815811 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7638e698-de68-4c83-b914-ff4f79d1259e admin admin] RBAC: Authorizing `identity:create_grant(project_id=9b16b936342343e3be044cf9cab78b2d, user_id=4f807920625945f5b08541616816a25a, role_id=3a074738221e4601b5205715ba87d240)` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:41:43.818579 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7638e698-de68-4c83-b914-ff4f79d1259e admin admin] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:41:43.831393 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 1380/2768] 10.222.0.47 () {60 vars in 1482 bytes} [Thu Jul 21 08:41:43 2022] PUT /identity/v3/projects/9b16b936342343e3be044cf9cab78b2d/users/4f807920625945f5b08541616816a25a/roles/3a074738221e4601b5205715ba87d240 => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 08:41:43.842523 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f3837d2-d77f-4d89-afc0-7224bb4c1eb6 None None] REQUEST_METHOD: `POST` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:41:43.842753 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f3837d2-d77f-4d89-afc0-7224bb4c1eb6 None None] SCRIPT_NAME: `/identity` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:41:43.843033 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f3837d2-d77f-4d89-afc0-7224bb4c1eb6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:41:43.870070 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.auth.core [None req-4f3837d2-d77f-4d89-afc0-7224bb4c1eb6 None None] MFA Rules not processed for user `4f807920625945f5b08541616816a25a`. Rule list: `[]` (Enabled: `True`). {{(pid=80245) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 08:41:43.871359 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.fernet_utils [None req-4f3837d2-d77f-4d89-afc0-7224bb4c1eb6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80245) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 08:41:43.901478 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG dbcounter [-] [80245] Writing DB stats keystone:DELETE=2,keystone:SELECT=95,keystone:INSERT=3 {{(pid=80245) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:41:43.936644 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: [pid: 80245|app: 0|req: 1389/2769] 10.222.0.47 () {58 vars in 957 bytes} [Thu Jul 21 08:41:43 2022] POST /identity/v3/auth/tokens => generated 3236 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 08:41:43.948876 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7638e698-de68-4c83-b914-ff4f79d1259e admin admin] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:41:43.950588 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.fernet_utils [None req-7638e698-de68-4c83-b914-ff4f79d1259e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80244) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 08:41:44.028688 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7638e698-de68-4c83-b914-ff4f79d1259e admin admin] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:41:44.032345 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0407262-b46c-42c1-aba3-30bd1c43992e service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9b0524aed8c74abd8949b860d6b93936', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4faef13aa50e4e36b6c265fb1398fa14', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:41:44.032993 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0407262-b46c-42c1-aba3-30bd1c43992e service glance] REQUEST_METHOD: `GET` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:41:44.033278 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0407262-b46c-42c1-aba3-30bd1c43992e service glance] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:41:44.033558 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0407262-b46c-42c1-aba3-30bd1c43992e service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:41:44.041880 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0407262-b46c-42c1-aba3-30bd1c43992e service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:41:44.043301 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0407262-b46c-42c1-aba3-30bd1c43992e service glance] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:41:44.048398 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG dbcounter [-] [80244] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80244) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:41:44.053337 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 1381/2770] 10.222.0.47 () {60 vars in 1365 bytes} [Thu Jul 21 08:41:43 2022] GET /identity/v3/auth/tokens => generated 3236 bytes in 105 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 08:41:53.933638 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG dbcounter [-] [80245] Writing DB stats keystone:SELECT=13 {{(pid=80245) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:42:10.689028 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f3837d2-d77f-4d89-afc0-7224bb4c1eb6 None None] Authenticating user token {{(pid=80245) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:42:10.751712 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f3837d2-d77f-4d89-afc0-7224bb4c1eb6 None None] Validating token access rules against request {{(pid=80245) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:42:10.756456 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45b687cf-2646-474b-b2ec-95076b631d4b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '42d15fc0515945acacbbffcf2356c7a5', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80245) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:42:10.757410 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45b687cf-2646-474b-b2ec-95076b631d4b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:42:10.757695 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45b687cf-2646-474b-b2ec-95076b631d4b admin admin] SCRIPT_NAME: `/identity` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:42:10.757972 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45b687cf-2646-474b-b2ec-95076b631d4b admin admin] PATH_INFO: `/v3/users/4f807920625945f5b08541616816a25a` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:42:10.759157 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45b687cf-2646-474b-b2ec-95076b631d4b admin admin] RBAC: Authorizing `identity:delete_user(user_id=4f807920625945f5b08541616816a25a)` {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:42:10.761119 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45b687cf-2646-474b-b2ec-95076b631d4b admin admin] RBAC: Authorization granted {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:42:10.807800 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.notifications [None req-45b687cf-2646-474b-b2ec-95076b631d4b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4f807920625945f5b08541616816a25a'} {{(pid=80245) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 08:42:10.815582 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.notifications [None req-45b687cf-2646-474b-b2ec-95076b631d4b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4f807920625945f5b08541616816a25a'} {{(pid=80245) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 08:42:10.827444 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.notifications [None req-45b687cf-2646-474b-b2ec-95076b631d4b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4f807920625945f5b08541616816a25a'} {{(pid=80245) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 08:42:10.834548 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.notifications [None req-45b687cf-2646-474b-b2ec-95076b631d4b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4f807920625945f5b08541616816a25a'} {{(pid=80245) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 08:42:10.835190 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.notifications [None req-45b687cf-2646-474b-b2ec-95076b631d4b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4f807920625945f5b08541616816a25a'} {{(pid=80245) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 08:42:10.841226 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: [pid: 80245|app: 0|req: 1390/2771] 10.222.0.47 () {58 vars in 1223 bytes} [Thu Jul 21 08:42:10 2022] DELETE /identity/v3/users/4f807920625945f5b08541616816a25a => generated 0 bytes in 153 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 08:42:10.846011 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f48aa52-0978-4f7d-a216-a8f959c168d9 None None] REQUEST_METHOD: `POST` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:42:10.846507 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f48aa52-0978-4f7d-a216-a8f959c168d9 None None] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:42:10.846626 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f48aa52-0978-4f7d-a216-a8f959c168d9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:42:10.864431 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.auth.core [None req-6f48aa52-0978-4f7d-a216-a8f959c168d9 None None] MFA Rules not processed for user `2dce40b8d74c41ec94cc36e6ff05d412`. Rule list: `[]` (Enabled: `True`). {{(pid=80244) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 08:42:10.865639 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.fernet_utils [None req-6f48aa52-0978-4f7d-a216-a8f959c168d9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80244) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 08:42:10.926331 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 1382/2772] 10.222.0.47 () {58 vars in 957 bytes} [Thu Jul 21 08:42:10 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 08:42:10.936354 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45b687cf-2646-474b-b2ec-95076b631d4b admin admin] Authenticating user token {{(pid=80245) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:42:10.938557 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.fernet_utils [None req-45b687cf-2646-474b-b2ec-95076b631d4b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80245) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 08:42:10.994836 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45b687cf-2646-474b-b2ec-95076b631d4b admin admin] Validating token access rules against request {{(pid=80245) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:42:10.998489 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-475c7e21-5483-4bd4-9ff7-7dae62e27dae service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20ca3c3bd3774691b3374cfbfec6e494', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4faef13aa50e4e36b6c265fb1398fa14', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80245) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:42:10.999139 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-475c7e21-5483-4bd4-9ff7-7dae62e27dae service neutron] REQUEST_METHOD: `GET` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:42:11.000010 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-475c7e21-5483-4bd4-9ff7-7dae62e27dae service neutron] SCRIPT_NAME: `/identity` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:42:11.000364 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-475c7e21-5483-4bd4-9ff7-7dae62e27dae service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:42:11.010610 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-475c7e21-5483-4bd4-9ff7-7dae62e27dae service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:42:11.012028 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-475c7e21-5483-4bd4-9ff7-7dae62e27dae service neutron] RBAC: Authorization granted {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:42:11.021699 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: [pid: 80245|app: 0|req: 1391/2773] 10.222.0.47 () {60 vars in 1365 bytes} [Thu Jul 21 08:42:10 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 08:42:11.290088 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f48aa52-0978-4f7d-a216-a8f959c168d9 None None] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:42:11.291592 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.fernet_utils [None req-6f48aa52-0978-4f7d-a216-a8f959c168d9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80244) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 08:42:11.304109 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f48aa52-0978-4f7d-a216-a8f959c168d9 None None] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:42:11.310619 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c43032c-dc18-4a1d-8061-8e2eecb225d1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '42d15fc0515945acacbbffcf2356c7a5', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:42:11.311573 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c43032c-dc18-4a1d-8061-8e2eecb225d1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:42:11.312344 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c43032c-dc18-4a1d-8061-8e2eecb225d1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:42:11.312617 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c43032c-dc18-4a1d-8061-8e2eecb225d1 admin admin] PATH_INFO: `/v3/projects/9b16b936342343e3be044cf9cab78b2d` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:42:11.313822 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c43032c-dc18-4a1d-8061-8e2eecb225d1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=9b16b936342343e3be044cf9cab78b2d)` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:42:11.316696 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c43032c-dc18-4a1d-8061-8e2eecb225d1 admin admin] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:42:11.366467 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.notifications [None req-4c43032c-dc18-4a1d-8061-8e2eecb225d1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9b16b936342343e3be044cf9cab78b2d', 'request_id': 'req-4c43032c-dc18-4a1d-8061-8e2eecb225d1'} {{(pid=80244) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 08:42:11.373722 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.notifications [None req-4c43032c-dc18-4a1d-8061-8e2eecb225d1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9b16b936342343e3be044cf9cab78b2d', 'request_id': 'req-4c43032c-dc18-4a1d-8061-8e2eecb225d1'} {{(pid=80244) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 08:42:11.380873 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.notifications [None req-4c43032c-dc18-4a1d-8061-8e2eecb225d1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9b16b936342343e3be044cf9cab78b2d', 'request_id': 'req-4c43032c-dc18-4a1d-8061-8e2eecb225d1'} {{(pid=80244) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 08:42:11.392127 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.notifications [None req-4c43032c-dc18-4a1d-8061-8e2eecb225d1 admin admin] The token cache is being invalidate because project 9b16b936342343e3be044cf9cab78b2d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80244) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 21 08:42:11.392411 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.notifications [None req-4c43032c-dc18-4a1d-8061-8e2eecb225d1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80244) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 08:42:11.393046 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.notifications [None req-4c43032c-dc18-4a1d-8061-8e2eecb225d1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80244) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 08:42:11.394584 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 1383/2774] 10.222.0.47 () {58 vars in 1232 bytes} [Thu Jul 21 08:42:11 2022] DELETE /identity/v3/projects/9b16b936342343e3be044cf9cab78b2d => generated 0 bytes in 105 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 08:42:11.399920 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43887e5a-5a5b-4129-89bb-e3cfbce9aca6 None None] REQUEST_METHOD: `POST` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:42:11.400204 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43887e5a-5a5b-4129-89bb-e3cfbce9aca6 None None] SCRIPT_NAME: `/identity` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:42:11.400479 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43887e5a-5a5b-4129-89bb-e3cfbce9aca6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:42:11.418945 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.auth.core [None req-43887e5a-5a5b-4129-89bb-e3cfbce9aca6 None None] MFA Rules not processed for user `2dce40b8d74c41ec94cc36e6ff05d412`. Rule list: `[]` (Enabled: `True`). {{(pid=80245) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 08:42:11.420229 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.fernet_utils [None req-43887e5a-5a5b-4129-89bb-e3cfbce9aca6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80245) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 08:42:11.437579 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG dbcounter [-] [80245] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80245) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:42:11.482238 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: [pid: 80245|app: 0|req: 1392/2775] 10.222.0.47 () {58 vars in 957 bytes} [Thu Jul 21 08:42:11 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 08:42:11.490691 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d370f1a-b019-4a96-874f-3a1a6ef1b068 None None] REQUEST_METHOD: `POST` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:42:11.490987 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d370f1a-b019-4a96-874f-3a1a6ef1b068 None None] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:42:11.491272 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d370f1a-b019-4a96-874f-3a1a6ef1b068 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:42:11.515899 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.auth.core [None req-9d370f1a-b019-4a96-874f-3a1a6ef1b068 None None] MFA Rules not processed for user `2dce40b8d74c41ec94cc36e6ff05d412`. Rule list: `[]` (Enabled: `True`). {{(pid=80244) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 08:42:11.517118 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.fernet_utils [None req-9d370f1a-b019-4a96-874f-3a1a6ef1b068 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80244) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 08:42:11.525384 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 1384/2776] 10.222.0.47 () {58 vars in 957 bytes} [Thu Jul 21 08:42:11 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 08:42:11.529743 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43887e5a-5a5b-4129-89bb-e3cfbce9aca6 None None] Authenticating user token {{(pid=80245) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:42:11.540454 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43887e5a-5a5b-4129-89bb-e3cfbce9aca6 None None] Validating token access rules against request {{(pid=80245) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:42:11.545304 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d923b46-f4ad-4008-b41e-e57ba5787790 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '42d15fc0515945acacbbffcf2356c7a5', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80245) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:42:11.546579 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d923b46-f4ad-4008-b41e-e57ba5787790 admin admin] REQUEST_METHOD: `GET` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:42:11.546956 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d923b46-f4ad-4008-b41e-e57ba5787790 admin admin] SCRIPT_NAME: `/identity` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:42:11.547468 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d923b46-f4ad-4008-b41e-e57ba5787790 admin admin] PATH_INFO: `/v3/domains` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:42:11.548388 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d923b46-f4ad-4008-b41e-e57ba5787790 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80245) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 08:42:11.548774 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d923b46-f4ad-4008-b41e-e57ba5787790 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:42:11.550443 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d923b46-f4ad-4008-b41e-e57ba5787790 admin admin] RBAC: Authorization granted {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:42:11.567376 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: [pid: 80245|app: 0|req: 1393/2777] 10.222.0.47 () {58 vars in 1152 bytes} [Thu Jul 21 08:42:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 08:42:11.570522 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d370f1a-b019-4a96-874f-3a1a6ef1b068 None None] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:42:11.580469 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d370f1a-b019-4a96-874f-3a1a6ef1b068 None None] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:42:11.585117 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5af4f3f2-8d64-4ac3-ad71-6586c234857c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '42d15fc0515945acacbbffcf2356c7a5', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:42:11.585769 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5af4f3f2-8d64-4ac3-ad71-6586c234857c admin admin] REQUEST_METHOD: `POST` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:42:11.586070 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5af4f3f2-8d64-4ac3-ad71-6586c234857c admin admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:42:11.586345 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5af4f3f2-8d64-4ac3-ad71-6586c234857c admin admin] PATH_INFO: `/v3/projects` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:42:11.587027 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5af4f3f2-8d64-4ac3-ad71-6586c234857c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:42:11.588989 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5af4f3f2-8d64-4ac3-ad71-6586c234857c admin admin] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:42:11.614988 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 1385/2778] 10.222.0.47 () {60 vars in 1152 bytes} [Thu Jul 21 08:42:11 2022] POST /identity/v3/projects => generated 363 bytes in 45 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 08:42:11.619394 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d923b46-f4ad-4008-b41e-e57ba5787790 admin admin] Authenticating user token {{(pid=80245) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:42:11.630196 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d923b46-f4ad-4008-b41e-e57ba5787790 admin admin] Validating token access rules against request {{(pid=80245) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:42:11.686445 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53ee24d9-375b-4907-b71c-649cc35a1339 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '42d15fc0515945acacbbffcf2356c7a5', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80245) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:42:11.687704 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53ee24d9-375b-4907-b71c-649cc35a1339 admin admin] REQUEST_METHOD: `POST` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:42:11.687992 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53ee24d9-375b-4907-b71c-649cc35a1339 admin admin] SCRIPT_NAME: `/identity` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:42:11.688271 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53ee24d9-375b-4907-b71c-649cc35a1339 admin admin] PATH_INFO: `/v3/users` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:42:11.688971 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53ee24d9-375b-4907-b71c-649cc35a1339 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:42:11.690955 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53ee24d9-375b-4907-b71c-649cc35a1339 admin admin] RBAC: Authorization granted {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:42:11.711864 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: [pid: 80245|app: 0|req: 1394/2779] 10.222.0.47 () {60 vars in 1143 bytes} [Thu Jul 21 08:42:11 2022] POST /identity/v3/users => generated 282 bytes in 93 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 21 08:42:11.716301 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5af4f3f2-8d64-4ac3-ad71-6586c234857c admin admin] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:42:11.726848 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5af4f3f2-8d64-4ac3-ad71-6586c234857c admin admin] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:42:11.731522 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7db4e9bc-d011-4880-96ca-0fcffebd3df5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '42d15fc0515945acacbbffcf2356c7a5', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:42:11.732179 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7db4e9bc-d011-4880-96ca-0fcffebd3df5 admin admin] REQUEST_METHOD: `GET` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:42:11.732470 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7db4e9bc-d011-4880-96ca-0fcffebd3df5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:42:11.732747 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7db4e9bc-d011-4880-96ca-0fcffebd3df5 admin admin] PATH_INFO: `/v3/roles` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:42:11.733470 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7db4e9bc-d011-4880-96ca-0fcffebd3df5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:42:11.735574 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7db4e9bc-d011-4880-96ca-0fcffebd3df5 admin admin] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:42:11.745781 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 1386/2780] 10.222.0.47 () {58 vars in 1121 bytes} [Thu Jul 21 08:42:11 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 08:42:11.750317 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53ee24d9-375b-4907-b71c-649cc35a1339 admin admin] Authenticating user token {{(pid=80245) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:42:11.760690 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53ee24d9-375b-4907-b71c-649cc35a1339 admin admin] Validating token access rules against request {{(pid=80245) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:42:11.765699 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13812c44-2c1b-44c0-969b-010270cad470 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '42d15fc0515945acacbbffcf2356c7a5', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80245) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:42:11.766382 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13812c44-2c1b-44c0-969b-010270cad470 admin admin] REQUEST_METHOD: `GET` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:42:11.766649 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13812c44-2c1b-44c0-969b-010270cad470 admin admin] SCRIPT_NAME: `/identity` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:42:11.766975 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13812c44-2c1b-44c0-969b-010270cad470 admin admin] PATH_INFO: `/v3/roles` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:42:11.767759 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13812c44-2c1b-44c0-969b-010270cad470 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:42:11.769352 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13812c44-2c1b-44c0-969b-010270cad470 admin admin] RBAC: Authorization granted {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:42:11.779389 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: [pid: 80245|app: 0|req: 1395/2781] 10.222.0.47 () {58 vars in 1121 bytes} [Thu Jul 21 08:42:11 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 08:42:11.783222 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7db4e9bc-d011-4880-96ca-0fcffebd3df5 admin admin] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:42:11.793591 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7db4e9bc-d011-4880-96ca-0fcffebd3df5 admin admin] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:42:11.798690 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-098c7487-36b8-4e42-8c26-54a748f659d6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '42d15fc0515945acacbbffcf2356c7a5', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:42:11.799591 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-098c7487-36b8-4e42-8c26-54a748f659d6 admin admin] REQUEST_METHOD: `PUT` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:42:11.799879 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-098c7487-36b8-4e42-8c26-54a748f659d6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:42:11.800157 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-098c7487-36b8-4e42-8c26-54a748f659d6 admin admin] PATH_INFO: `/v3/projects/0993312e2717405eb3fa20e28a802c3b/users/0bd219159bc841d1affb14bb0a916834/roles/3a074738221e4601b5205715ba87d240` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:42:11.815285 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-098c7487-36b8-4e42-8c26-54a748f659d6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0993312e2717405eb3fa20e28a802c3b, user_id=0bd219159bc841d1affb14bb0a916834, role_id=3a074738221e4601b5205715ba87d240)` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:42:11.818062 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-098c7487-36b8-4e42-8c26-54a748f659d6 admin admin] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:42:11.830835 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 1387/2782] 10.222.0.47 () {60 vars in 1482 bytes} [Thu Jul 21 08:42:11 2022] PUT /identity/v3/projects/0993312e2717405eb3fa20e28a802c3b/users/0bd219159bc841d1affb14bb0a916834/roles/3a074738221e4601b5205715ba87d240 => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 08:42:11.841231 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34e434b1-e738-4655-9020-97f0a6a5e82e None None] REQUEST_METHOD: `POST` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:42:11.841526 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34e434b1-e738-4655-9020-97f0a6a5e82e None None] SCRIPT_NAME: `/identity` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:42:11.841815 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34e434b1-e738-4655-9020-97f0a6a5e82e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:42:11.868022 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.auth.core [None req-34e434b1-e738-4655-9020-97f0a6a5e82e None None] MFA Rules not processed for user `0bd219159bc841d1affb14bb0a916834`. Rule list: `[]` (Enabled: `True`). {{(pid=80245) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 08:42:11.869248 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.fernet_utils [None req-34e434b1-e738-4655-9020-97f0a6a5e82e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80245) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 08:42:11.934443 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: [pid: 80245|app: 0|req: 1396/2783] 10.222.0.47 () {58 vars in 957 bytes} [Thu Jul 21 08:42:11 2022] POST /identity/v3/auth/tokens => generated 3230 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 08:42:11.946019 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-098c7487-36b8-4e42-8c26-54a748f659d6 admin admin] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:42:11.947844 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.fernet_utils [None req-098c7487-36b8-4e42-8c26-54a748f659d6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80244) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 08:42:11.969692 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG dbcounter [-] [80244] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80244) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:42:12.006846 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-098c7487-36b8-4e42-8c26-54a748f659d6 admin admin] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:42:12.010446 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc150e7c-aa4a-4b6d-b7a9-07e9f1446505 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4f40410b7674687ab5362ac982248ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4faef13aa50e4e36b6c265fb1398fa14', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:42:12.011087 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc150e7c-aa4a-4b6d-b7a9-07e9f1446505 service cinder] REQUEST_METHOD: `GET` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:42:12.011398 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc150e7c-aa4a-4b6d-b7a9-07e9f1446505 service cinder] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:42:12.011672 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc150e7c-aa4a-4b6d-b7a9-07e9f1446505 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:42:12.019933 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc150e7c-aa4a-4b6d-b7a9-07e9f1446505 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:42:12.021357 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc150e7c-aa4a-4b6d-b7a9-07e9f1446505 service cinder] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:42:12.030011 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 1388/2784] 10.222.0.47 () {60 vars in 1365 bytes} [Thu Jul 21 08:42:11 2022] GET /identity/v3/auth/tokens => generated 3230 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 08:42:21.931522 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG dbcounter [-] [80245] Writing DB stats keystone:SELECT=89,keystone:INSERT=3 {{(pid=80245) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:42:22.026201 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG dbcounter [-] [80244] Writing DB stats keystone:SELECT=17 {{(pid=80244) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:42:31.077306 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34e434b1-e738-4655-9020-97f0a6a5e82e None None] Authenticating user token {{(pid=80245) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:42:31.079814 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.fernet_utils [None req-34e434b1-e738-4655-9020-97f0a6a5e82e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80245) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 08:42:31.155108 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34e434b1-e738-4655-9020-97f0a6a5e82e None None] Validating token access rules against request {{(pid=80245) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:42:31.160190 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf54c6e0-c88f-4860-8cad-343f0853110b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '42d15fc0515945acacbbffcf2356c7a5', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80245) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:42:31.161069 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf54c6e0-c88f-4860-8cad-343f0853110b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:42:31.161349 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf54c6e0-c88f-4860-8cad-343f0853110b admin admin] SCRIPT_NAME: `/identity` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:42:31.161622 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf54c6e0-c88f-4860-8cad-343f0853110b admin admin] PATH_INFO: `/v3/users/591ec12e19db47139d3a73484abf2f0d` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:42:31.162813 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf54c6e0-c88f-4860-8cad-343f0853110b admin admin] RBAC: Authorizing `identity:delete_user(user_id=591ec12e19db47139d3a73484abf2f0d)` {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:42:31.164765 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf54c6e0-c88f-4860-8cad-343f0853110b admin admin] RBAC: Authorization granted {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:42:31.211217 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.notifications [None req-cf54c6e0-c88f-4860-8cad-343f0853110b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '591ec12e19db47139d3a73484abf2f0d'} {{(pid=80245) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 08:42:31.218644 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.notifications [None req-cf54c6e0-c88f-4860-8cad-343f0853110b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '591ec12e19db47139d3a73484abf2f0d'} {{(pid=80245) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 08:42:31.231531 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.notifications [None req-cf54c6e0-c88f-4860-8cad-343f0853110b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '591ec12e19db47139d3a73484abf2f0d'} {{(pid=80245) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 08:42:31.240493 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.notifications [None req-cf54c6e0-c88f-4860-8cad-343f0853110b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '591ec12e19db47139d3a73484abf2f0d'} {{(pid=80245) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 08:42:31.241231 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.notifications [None req-cf54c6e0-c88f-4860-8cad-343f0853110b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '591ec12e19db47139d3a73484abf2f0d'} {{(pid=80245) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 08:42:31.247182 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: [pid: 80245|app: 0|req: 1397/2785] 10.222.0.47 () {58 vars in 1223 bytes} [Thu Jul 21 08:42:31 2022] DELETE /identity/v3/users/591ec12e19db47139d3a73484abf2f0d => generated 0 bytes in 176 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 08:42:31.425736 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc150e7c-aa4a-4b6d-b7a9-07e9f1446505 service cinder] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:42:31.439659 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc150e7c-aa4a-4b6d-b7a9-07e9f1446505 service cinder] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:42:31.519021 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aa8a797-cce3-4b9b-8a4f-8e7f790b1826 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '42d15fc0515945acacbbffcf2356c7a5', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:42:31.520447 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aa8a797-cce3-4b9b-8a4f-8e7f790b1826 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:42:31.520618 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aa8a797-cce3-4b9b-8a4f-8e7f790b1826 admin admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:42:31.520922 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aa8a797-cce3-4b9b-8a4f-8e7f790b1826 admin admin] PATH_INFO: `/v3/projects/d588f87ef1d248528ec37cd9b5d43dcc` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:42:31.522152 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6aa8a797-cce3-4b9b-8a4f-8e7f790b1826 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d588f87ef1d248528ec37cd9b5d43dcc)` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:42:31.524071 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6aa8a797-cce3-4b9b-8a4f-8e7f790b1826 admin admin] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:42:31.591754 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.notifications [None req-6aa8a797-cce3-4b9b-8a4f-8e7f790b1826 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd588f87ef1d248528ec37cd9b5d43dcc', 'request_id': 'req-6aa8a797-cce3-4b9b-8a4f-8e7f790b1826'} {{(pid=80244) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 08:42:31.600778 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.notifications [None req-6aa8a797-cce3-4b9b-8a4f-8e7f790b1826 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd588f87ef1d248528ec37cd9b5d43dcc', 'request_id': 'req-6aa8a797-cce3-4b9b-8a4f-8e7f790b1826'} {{(pid=80244) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 08:42:31.608643 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.notifications [None req-6aa8a797-cce3-4b9b-8a4f-8e7f790b1826 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd588f87ef1d248528ec37cd9b5d43dcc', 'request_id': 'req-6aa8a797-cce3-4b9b-8a4f-8e7f790b1826'} {{(pid=80244) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 08:42:31.619896 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.notifications [None req-6aa8a797-cce3-4b9b-8a4f-8e7f790b1826 admin admin] The token cache is being invalidate because project d588f87ef1d248528ec37cd9b5d43dcc was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80244) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 21 08:42:31.620174 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.notifications [None req-6aa8a797-cce3-4b9b-8a4f-8e7f790b1826 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80244) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 08:42:31.620831 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.notifications [None req-6aa8a797-cce3-4b9b-8a4f-8e7f790b1826 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80244) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 08:42:31.622418 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 1389/2786] 10.222.0.47 () {58 vars in 1232 bytes} [Thu Jul 21 08:42:31 2022] DELETE /identity/v3/projects/d588f87ef1d248528ec37cd9b5d43dcc => generated 0 bytes in 197 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 08:42:31.627662 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81bf6314-ccb9-407a-9515-0d0b4bb6664a None None] REQUEST_METHOD: `POST` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:42:31.627960 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81bf6314-ccb9-407a-9515-0d0b4bb6664a None None] SCRIPT_NAME: `/identity` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:42:31.628252 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81bf6314-ccb9-407a-9515-0d0b4bb6664a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:42:31.646062 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.auth.core [None req-81bf6314-ccb9-407a-9515-0d0b4bb6664a None None] MFA Rules not processed for user `2dce40b8d74c41ec94cc36e6ff05d412`. Rule list: `[]` (Enabled: `True`). {{(pid=80245) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 08:42:31.647305 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.fernet_utils [None req-81bf6314-ccb9-407a-9515-0d0b4bb6664a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80245) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 08:42:31.707197 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: [pid: 80245|app: 0|req: 1398/2787] 10.222.0.47 () {58 vars in 957 bytes} [Thu Jul 21 08:42:31 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 08:42:31.714864 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-783ecddd-8128-4ec7-bfb7-07d28f7ce66d None None] REQUEST_METHOD: `POST` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:42:31.715442 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-783ecddd-8128-4ec7-bfb7-07d28f7ce66d None None] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:42:31.715726 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-783ecddd-8128-4ec7-bfb7-07d28f7ce66d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:42:31.735261 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.auth.core [None req-783ecddd-8128-4ec7-bfb7-07d28f7ce66d None None] MFA Rules not processed for user `2dce40b8d74c41ec94cc36e6ff05d412`. Rule list: `[]` (Enabled: `True`). {{(pid=80244) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 08:42:31.736491 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.fernet_utils [None req-783ecddd-8128-4ec7-bfb7-07d28f7ce66d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80244) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 08:42:31.744495 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 1390/2788] 10.222.0.47 () {58 vars in 957 bytes} [Thu Jul 21 08:42:31 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 08:42:31.748788 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81bf6314-ccb9-407a-9515-0d0b4bb6664a None None] Authenticating user token {{(pid=80245) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:42:31.758968 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81bf6314-ccb9-407a-9515-0d0b4bb6664a None None] Validating token access rules against request {{(pid=80245) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:42:31.763628 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f95d685-eb37-48e7-89d3-7deb8ac725e6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '42d15fc0515945acacbbffcf2356c7a5', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80245) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:42:31.764290 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f95d685-eb37-48e7-89d3-7deb8ac725e6 admin admin] REQUEST_METHOD: `GET` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:42:31.764575 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f95d685-eb37-48e7-89d3-7deb8ac725e6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:42:31.764851 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f95d685-eb37-48e7-89d3-7deb8ac725e6 admin admin] PATH_INFO: `/v3/domains` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:42:31.765581 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f95d685-eb37-48e7-89d3-7deb8ac725e6 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80245) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 08:42:31.765889 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f95d685-eb37-48e7-89d3-7deb8ac725e6 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:42:31.767521 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f95d685-eb37-48e7-89d3-7deb8ac725e6 admin admin] RBAC: Authorization granted {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:42:31.780594 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: [pid: 80245|app: 0|req: 1399/2789] 10.222.0.47 () {58 vars in 1152 bytes} [Thu Jul 21 08:42:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 08:42:31.784751 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-783ecddd-8128-4ec7-bfb7-07d28f7ce66d None None] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:42:31.794100 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-783ecddd-8128-4ec7-bfb7-07d28f7ce66d None None] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:42:31.799466 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15500d36-a4e0-43e1-9a0d-54a6553d7079 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '42d15fc0515945acacbbffcf2356c7a5', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:42:31.800123 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15500d36-a4e0-43e1-9a0d-54a6553d7079 admin admin] REQUEST_METHOD: `POST` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:42:31.800412 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15500d36-a4e0-43e1-9a0d-54a6553d7079 admin admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:42:31.800692 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15500d36-a4e0-43e1-9a0d-54a6553d7079 admin admin] PATH_INFO: `/v3/projects` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:42:31.801394 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15500d36-a4e0-43e1-9a0d-54a6553d7079 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:42:31.803350 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15500d36-a4e0-43e1-9a0d-54a6553d7079 admin admin] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:42:31.830885 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 1391/2790] 10.222.0.47 () {60 vars in 1152 bytes} [Thu Jul 21 08:42:31 2022] POST /identity/v3/projects => generated 367 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 08:42:31.835053 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f95d685-eb37-48e7-89d3-7deb8ac725e6 admin admin] Authenticating user token {{(pid=80245) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:42:31.845707 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f95d685-eb37-48e7-89d3-7deb8ac725e6 admin admin] Validating token access rules against request {{(pid=80245) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:42:31.849265 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG dbcounter [-] [80245] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80245) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:42:31.906777 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bf76974-cefc-4e58-82d7-6d6a45309af7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '42d15fc0515945acacbbffcf2356c7a5', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80245) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:42:31.907497 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bf76974-cefc-4e58-82d7-6d6a45309af7 admin admin] REQUEST_METHOD: `POST` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:42:31.907780 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bf76974-cefc-4e58-82d7-6d6a45309af7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:42:31.908056 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bf76974-cefc-4e58-82d7-6d6a45309af7 admin admin] PATH_INFO: `/v3/users` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:42:31.908764 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bf76974-cefc-4e58-82d7-6d6a45309af7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:42:31.910708 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bf76974-cefc-4e58-82d7-6d6a45309af7 admin admin] RBAC: Authorization granted {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:42:31.931382 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: [pid: 80245|app: 0|req: 1400/2791] 10.222.0.47 () {60 vars in 1143 bytes} [Thu Jul 21 08:42:31 2022] POST /identity/v3/users => generated 284 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 21 08:42:31.935409 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15500d36-a4e0-43e1-9a0d-54a6553d7079 admin admin] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:42:31.945783 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15500d36-a4e0-43e1-9a0d-54a6553d7079 admin admin] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:42:31.950400 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb4eccea-7aaf-4d7f-9686-326864f7c592 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '42d15fc0515945acacbbffcf2356c7a5', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:42:31.951072 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb4eccea-7aaf-4d7f-9686-326864f7c592 admin admin] REQUEST_METHOD: `GET` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:42:31.951353 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb4eccea-7aaf-4d7f-9686-326864f7c592 admin admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:42:31.951685 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb4eccea-7aaf-4d7f-9686-326864f7c592 admin admin] PATH_INFO: `/v3/roles` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:42:31.952414 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb4eccea-7aaf-4d7f-9686-326864f7c592 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:42:31.954114 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb4eccea-7aaf-4d7f-9686-326864f7c592 admin admin] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:42:31.964802 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 1392/2792] 10.222.0.47 () {58 vars in 1121 bytes} [Thu Jul 21 08:42:31 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 08:42:31.969381 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bf76974-cefc-4e58-82d7-6d6a45309af7 admin admin] Authenticating user token {{(pid=80245) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:42:31.979569 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bf76974-cefc-4e58-82d7-6d6a45309af7 admin admin] Validating token access rules against request {{(pid=80245) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:42:31.984949 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7009463b-d0ed-4044-b11c-c9029b937e0a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '42d15fc0515945acacbbffcf2356c7a5', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80245) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:42:31.985608 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7009463b-d0ed-4044-b11c-c9029b937e0a admin admin] REQUEST_METHOD: `GET` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:42:31.985895 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7009463b-d0ed-4044-b11c-c9029b937e0a admin admin] SCRIPT_NAME: `/identity` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:42:31.986174 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7009463b-d0ed-4044-b11c-c9029b937e0a admin admin] PATH_INFO: `/v3/roles` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:42:31.986905 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7009463b-d0ed-4044-b11c-c9029b937e0a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:42:31.988575 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7009463b-d0ed-4044-b11c-c9029b937e0a admin admin] RBAC: Authorization granted {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:42:31.998805 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: [pid: 80245|app: 0|req: 1401/2793] 10.222.0.47 () {58 vars in 1121 bytes} [Thu Jul 21 08:42:31 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 08:42:32.003115 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb4eccea-7aaf-4d7f-9686-326864f7c592 admin admin] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:42:32.015126 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb4eccea-7aaf-4d7f-9686-326864f7c592 admin admin] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:42:32.019852 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0403ccc0-d705-404a-b194-d2db1ef61b34 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '42d15fc0515945acacbbffcf2356c7a5', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:42:32.020771 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0403ccc0-d705-404a-b194-d2db1ef61b34 admin admin] REQUEST_METHOD: `PUT` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:42:32.021055 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0403ccc0-d705-404a-b194-d2db1ef61b34 admin admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:42:32.021330 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0403ccc0-d705-404a-b194-d2db1ef61b34 admin admin] PATH_INFO: `/v3/projects/20501b6c1908445ea9ac1855a1210c38/users/d996d9b3678148be9b53ecf388b1cb81/roles/3a074738221e4601b5205715ba87d240` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:42:32.036695 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0403ccc0-d705-404a-b194-d2db1ef61b34 admin admin] RBAC: Authorizing `identity:create_grant(project_id=20501b6c1908445ea9ac1855a1210c38, user_id=d996d9b3678148be9b53ecf388b1cb81, role_id=3a074738221e4601b5205715ba87d240)` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:42:32.039381 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0403ccc0-d705-404a-b194-d2db1ef61b34 admin admin] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:42:32.052460 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 1393/2794] 10.222.0.47 () {60 vars in 1482 bytes} [Thu Jul 21 08:42:32 2022] PUT /identity/v3/projects/20501b6c1908445ea9ac1855a1210c38/users/d996d9b3678148be9b53ecf388b1cb81/roles/3a074738221e4601b5205715ba87d240 => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 08:42:32.063349 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ebfb0b0-e0ad-4f86-897c-6101f349b409 None None] REQUEST_METHOD: `POST` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:42:32.063646 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ebfb0b0-e0ad-4f86-897c-6101f349b409 None None] SCRIPT_NAME: `/identity` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:42:32.063931 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ebfb0b0-e0ad-4f86-897c-6101f349b409 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:42:32.090591 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.auth.core [None req-9ebfb0b0-e0ad-4f86-897c-6101f349b409 None None] MFA Rules not processed for user `d996d9b3678148be9b53ecf388b1cb81`. Rule list: `[]` (Enabled: `True`). {{(pid=80245) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 08:42:32.091840 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.fernet_utils [None req-9ebfb0b0-e0ad-4f86-897c-6101f349b409 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80245) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 08:42:32.158356 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: [pid: 80245|app: 0|req: 1402/2795] 10.222.0.47 () {58 vars in 957 bytes} [Thu Jul 21 08:42:32 2022] POST /identity/v3/auth/tokens => generated 3234 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 08:42:32.169254 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0403ccc0-d705-404a-b194-d2db1ef61b34 admin admin] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:42:32.170965 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.fernet_utils [None req-0403ccc0-d705-404a-b194-d2db1ef61b34 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80244) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 08:42:32.202409 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG dbcounter [-] [80244] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80244) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:42:32.231605 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0403ccc0-d705-404a-b194-d2db1ef61b34 admin admin] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:42:32.235337 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dce415a-af10-47d4-8d1c-6e699018402d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4f40410b7674687ab5362ac982248ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4faef13aa50e4e36b6c265fb1398fa14', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:42:32.235998 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dce415a-af10-47d4-8d1c-6e699018402d service cinder] REQUEST_METHOD: `GET` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:42:32.236281 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dce415a-af10-47d4-8d1c-6e699018402d service cinder] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:42:32.236556 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dce415a-af10-47d4-8d1c-6e699018402d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:42:32.245118 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dce415a-af10-47d4-8d1c-6e699018402d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:42:32.250310 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dce415a-af10-47d4-8d1c-6e699018402d service cinder] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:42:32.257035 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 1394/2796] 10.222.0.47 () {60 vars in 1365 bytes} [Thu Jul 21 08:42:32 2022] GET /identity/v3/auth/tokens => generated 3234 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 08:42:40.606383 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ebfb0b0-e0ad-4f86-897c-6101f349b409 None None] Authenticating user token {{(pid=80245) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:42:40.667183 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ebfb0b0-e0ad-4f86-897c-6101f349b409 None None] Validating token access rules against request {{(pid=80245) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:42:40.672338 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65b91f7e-f39e-42f0-9c10-307b97f6bbd6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '42d15fc0515945acacbbffcf2356c7a5', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80245) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:42:40.673207 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65b91f7e-f39e-42f0-9c10-307b97f6bbd6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:42:40.673493 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65b91f7e-f39e-42f0-9c10-307b97f6bbd6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:42:40.673772 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65b91f7e-f39e-42f0-9c10-307b97f6bbd6 admin admin] PATH_INFO: `/v3/users/d996d9b3678148be9b53ecf388b1cb81` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:42:40.674950 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65b91f7e-f39e-42f0-9c10-307b97f6bbd6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d996d9b3678148be9b53ecf388b1cb81)` {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:42:40.676884 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65b91f7e-f39e-42f0-9c10-307b97f6bbd6 admin admin] RBAC: Authorization granted {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:42:40.693037 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG dbcounter [-] [80245] Writing DB stats keystone:SELECT=96,keystone:INSERT=3,keystone:DELETE=1 {{(pid=80245) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:42:40.732369 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.notifications [None req-65b91f7e-f39e-42f0-9c10-307b97f6bbd6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd996d9b3678148be9b53ecf388b1cb81'} {{(pid=80245) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 08:42:40.739955 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.notifications [None req-65b91f7e-f39e-42f0-9c10-307b97f6bbd6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd996d9b3678148be9b53ecf388b1cb81'} {{(pid=80245) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 08:42:40.752397 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.notifications [None req-65b91f7e-f39e-42f0-9c10-307b97f6bbd6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd996d9b3678148be9b53ecf388b1cb81'} {{(pid=80245) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 08:42:40.759906 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.notifications [None req-65b91f7e-f39e-42f0-9c10-307b97f6bbd6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd996d9b3678148be9b53ecf388b1cb81'} {{(pid=80245) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 08:42:40.760617 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.notifications [None req-65b91f7e-f39e-42f0-9c10-307b97f6bbd6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd996d9b3678148be9b53ecf388b1cb81'} {{(pid=80245) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 08:42:40.766617 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: [pid: 80245|app: 0|req: 1403/2797] 10.222.0.47 () {58 vars in 1223 bytes} [Thu Jul 21 08:42:40 2022] DELETE /identity/v3/users/d996d9b3678148be9b53ecf388b1cb81 => generated 0 bytes in 161 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 08:42:40.771148 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fa19a0b-ceae-47d7-ab94-b3161f38df3e None None] REQUEST_METHOD: `POST` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:42:40.771655 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fa19a0b-ceae-47d7-ab94-b3161f38df3e None None] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:42:40.771757 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fa19a0b-ceae-47d7-ab94-b3161f38df3e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:42:40.789405 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.auth.core [None req-0fa19a0b-ceae-47d7-ab94-b3161f38df3e None None] MFA Rules not processed for user `2dce40b8d74c41ec94cc36e6ff05d412`. Rule list: `[]` (Enabled: `True`). {{(pid=80244) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 08:42:40.790588 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.fernet_utils [None req-0fa19a0b-ceae-47d7-ab94-b3161f38df3e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80244) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 08:42:40.848474 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 1395/2798] 10.222.0.47 () {58 vars in 957 bytes} [Thu Jul 21 08:42:40 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 78 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 08:42:40.859224 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65b91f7e-f39e-42f0-9c10-307b97f6bbd6 admin admin] Authenticating user token {{(pid=80245) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:42:40.860888 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.fernet_utils [None req-65b91f7e-f39e-42f0-9c10-307b97f6bbd6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80245) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 08:42:40.917951 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65b91f7e-f39e-42f0-9c10-307b97f6bbd6 admin admin] Validating token access rules against request {{(pid=80245) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:42:40.922448 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cf567b5-b200-49aa-a9c7-7266b5693ffc service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20ca3c3bd3774691b3374cfbfec6e494', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4faef13aa50e4e36b6c265fb1398fa14', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80245) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:42:40.923098 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cf567b5-b200-49aa-a9c7-7266b5693ffc service neutron] REQUEST_METHOD: `GET` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:42:40.923449 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cf567b5-b200-49aa-a9c7-7266b5693ffc service neutron] SCRIPT_NAME: `/identity` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:42:40.923818 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cf567b5-b200-49aa-a9c7-7266b5693ffc service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:42:40.933458 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cf567b5-b200-49aa-a9c7-7266b5693ffc service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:42:40.934889 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cf567b5-b200-49aa-a9c7-7266b5693ffc service neutron] RBAC: Authorization granted {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:42:40.944614 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: [pid: 80245|app: 0|req: 1404/2799] 10.222.0.47 () {60 vars in 1365 bytes} [Thu Jul 21 08:42:40 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 08:42:41.230051 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fa19a0b-ceae-47d7-ab94-b3161f38df3e None None] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:42:41.231615 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.fernet_utils [None req-0fa19a0b-ceae-47d7-ab94-b3161f38df3e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80244) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 08:42:41.243659 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fa19a0b-ceae-47d7-ab94-b3161f38df3e None None] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:42:41.248752 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73f58de4-015e-48e6-902d-7474a0b4037b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '42d15fc0515945acacbbffcf2356c7a5', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:42:41.249629 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73f58de4-015e-48e6-902d-7474a0b4037b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:42:41.249913 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73f58de4-015e-48e6-902d-7474a0b4037b admin admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:42:41.250189 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73f58de4-015e-48e6-902d-7474a0b4037b admin admin] PATH_INFO: `/v3/projects/20501b6c1908445ea9ac1855a1210c38` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:42:41.251465 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73f58de4-015e-48e6-902d-7474a0b4037b admin admin] RBAC: Authorizing `identity:delete_project(project_id=20501b6c1908445ea9ac1855a1210c38)` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:42:41.253646 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73f58de4-015e-48e6-902d-7474a0b4037b admin admin] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:42:41.301773 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.notifications [None req-73f58de4-015e-48e6-902d-7474a0b4037b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '20501b6c1908445ea9ac1855a1210c38', 'request_id': 'req-73f58de4-015e-48e6-902d-7474a0b4037b'} {{(pid=80244) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 08:42:41.310046 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.notifications [None req-73f58de4-015e-48e6-902d-7474a0b4037b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '20501b6c1908445ea9ac1855a1210c38', 'request_id': 'req-73f58de4-015e-48e6-902d-7474a0b4037b'} {{(pid=80244) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 08:42:41.316449 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.notifications [None req-73f58de4-015e-48e6-902d-7474a0b4037b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '20501b6c1908445ea9ac1855a1210c38', 'request_id': 'req-73f58de4-015e-48e6-902d-7474a0b4037b'} {{(pid=80244) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 08:42:41.327200 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.notifications [None req-73f58de4-015e-48e6-902d-7474a0b4037b admin admin] The token cache is being invalidate because project 20501b6c1908445ea9ac1855a1210c38 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80244) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 21 08:42:41.327510 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.notifications [None req-73f58de4-015e-48e6-902d-7474a0b4037b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80244) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 08:42:41.328149 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.notifications [None req-73f58de4-015e-48e6-902d-7474a0b4037b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80244) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 08:42:41.329690 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 1396/2800] 10.222.0.47 () {58 vars in 1232 bytes} [Thu Jul 21 08:42:41 2022] DELETE /identity/v3/projects/20501b6c1908445ea9ac1855a1210c38 => generated 0 bytes in 100 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 08:42:41.334694 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f52ed1d3-bccf-4106-a38b-d74830f7084b None None] REQUEST_METHOD: `POST` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:42:41.334987 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f52ed1d3-bccf-4106-a38b-d74830f7084b None None] SCRIPT_NAME: `/identity` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:42:41.335266 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f52ed1d3-bccf-4106-a38b-d74830f7084b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:42:41.353700 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.auth.core [None req-f52ed1d3-bccf-4106-a38b-d74830f7084b None None] MFA Rules not processed for user `2dce40b8d74c41ec94cc36e6ff05d412`. Rule list: `[]` (Enabled: `True`). {{(pid=80245) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 08:42:41.354918 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.fernet_utils [None req-f52ed1d3-bccf-4106-a38b-d74830f7084b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80245) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 08:42:41.421904 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: [pid: 80245|app: 0|req: 1405/2801] 10.222.0.47 () {58 vars in 957 bytes} [Thu Jul 21 08:42:41 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 08:42:41.429812 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc8e9eff-2b1b-4728-ad86-817544a30ee5 None None] REQUEST_METHOD: `POST` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:42:41.430106 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc8e9eff-2b1b-4728-ad86-817544a30ee5 None None] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:42:41.430414 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc8e9eff-2b1b-4728-ad86-817544a30ee5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:42:41.449398 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.auth.core [None req-bc8e9eff-2b1b-4728-ad86-817544a30ee5 None None] MFA Rules not processed for user `2dce40b8d74c41ec94cc36e6ff05d412`. Rule list: `[]` (Enabled: `True`). {{(pid=80244) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 08:42:41.450610 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.fernet_utils [None req-bc8e9eff-2b1b-4728-ad86-817544a30ee5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80244) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 08:42:41.459246 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 1397/2802] 10.222.0.47 () {58 vars in 957 bytes} [Thu Jul 21 08:42:41 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 08:42:41.463483 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f52ed1d3-bccf-4106-a38b-d74830f7084b None None] Authenticating user token {{(pid=80245) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:42:41.473417 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f52ed1d3-bccf-4106-a38b-d74830f7084b None None] Validating token access rules against request {{(pid=80245) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:42:41.478111 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c53a9c81-6f36-42cf-b7ef-76a9e33a5896 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '42d15fc0515945acacbbffcf2356c7a5', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80245) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:42:41.478811 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c53a9c81-6f36-42cf-b7ef-76a9e33a5896 admin admin] REQUEST_METHOD: `GET` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:42:41.479098 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c53a9c81-6f36-42cf-b7ef-76a9e33a5896 admin admin] SCRIPT_NAME: `/identity` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:42:41.479411 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c53a9c81-6f36-42cf-b7ef-76a9e33a5896 admin admin] PATH_INFO: `/v3/domains` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:42:41.480138 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c53a9c81-6f36-42cf-b7ef-76a9e33a5896 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80245) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 08:42:41.480446 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c53a9c81-6f36-42cf-b7ef-76a9e33a5896 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:42:41.482063 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c53a9c81-6f36-42cf-b7ef-76a9e33a5896 admin admin] RBAC: Authorization granted {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:42:41.494386 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: [pid: 80245|app: 0|req: 1406/2803] 10.222.0.47 () {58 vars in 1152 bytes} [Thu Jul 21 08:42:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 21 08:42:41.499399 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc8e9eff-2b1b-4728-ad86-817544a30ee5 None None] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:42:41.509130 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc8e9eff-2b1b-4728-ad86-817544a30ee5 None None] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:42:41.518736 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09887e53-e03e-4a15-8e3f-3215d38a10a8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '42d15fc0515945acacbbffcf2356c7a5', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:42:41.519468 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09887e53-e03e-4a15-8e3f-3215d38a10a8 admin admin] REQUEST_METHOD: `POST` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:42:41.519755 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09887e53-e03e-4a15-8e3f-3215d38a10a8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:42:41.520030 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09887e53-e03e-4a15-8e3f-3215d38a10a8 admin admin] PATH_INFO: `/v3/projects` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:42:41.520768 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09887e53-e03e-4a15-8e3f-3215d38a10a8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:42:41.522679 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09887e53-e03e-4a15-8e3f-3215d38a10a8 admin admin] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:42:41.549392 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 1398/2804] 10.222.0.47 () {60 vars in 1152 bytes} [Thu Jul 21 08:42:41 2022] POST /identity/v3/projects => generated 387 bytes in 51 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 08:42:41.554067 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c53a9c81-6f36-42cf-b7ef-76a9e33a5896 admin admin] Authenticating user token {{(pid=80245) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:42:41.566120 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c53a9c81-6f36-42cf-b7ef-76a9e33a5896 admin admin] Validating token access rules against request {{(pid=80245) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:42:41.589894 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG dbcounter [-] [80245] Writing DB stats keystone:DELETE=9,keystone:SELECT=90,keystone:INSERT=1 {{(pid=80245) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:42:41.627888 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21ab4a73-a6b2-446c-a790-c609137eada3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '42d15fc0515945acacbbffcf2356c7a5', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80245) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:42:41.628589 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21ab4a73-a6b2-446c-a790-c609137eada3 admin admin] REQUEST_METHOD: `POST` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:42:41.628832 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21ab4a73-a6b2-446c-a790-c609137eada3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:42:41.629109 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21ab4a73-a6b2-446c-a790-c609137eada3 admin admin] PATH_INFO: `/v3/users` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:42:41.629795 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21ab4a73-a6b2-446c-a790-c609137eada3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:42:41.631759 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21ab4a73-a6b2-446c-a790-c609137eada3 admin admin] RBAC: Authorization granted {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:42:41.653587 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: [pid: 80245|app: 0|req: 1407/2805] 10.222.0.47 () {60 vars in 1143 bytes} [Thu Jul 21 08:42:41 2022] POST /identity/v3/users => generated 294 bytes in 100 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 21 08:42:41.657767 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09887e53-e03e-4a15-8e3f-3215d38a10a8 admin admin] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:42:41.668098 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09887e53-e03e-4a15-8e3f-3215d38a10a8 admin admin] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:42:41.672736 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e21ad515-faac-49fb-b707-e2c796dd64f3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '42d15fc0515945acacbbffcf2356c7a5', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:42:41.673399 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e21ad515-faac-49fb-b707-e2c796dd64f3 admin admin] REQUEST_METHOD: `GET` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:42:41.673704 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e21ad515-faac-49fb-b707-e2c796dd64f3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:42:41.673964 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e21ad515-faac-49fb-b707-e2c796dd64f3 admin admin] PATH_INFO: `/v3/roles` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:42:41.674699 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e21ad515-faac-49fb-b707-e2c796dd64f3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:42:41.677637 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e21ad515-faac-49fb-b707-e2c796dd64f3 admin admin] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:42:41.688777 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 1399/2806] 10.222.0.47 () {58 vars in 1121 bytes} [Thu Jul 21 08:42:41 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 08:42:41.692857 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21ab4a73-a6b2-446c-a790-c609137eada3 admin admin] Authenticating user token {{(pid=80245) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:42:41.703533 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21ab4a73-a6b2-446c-a790-c609137eada3 admin admin] Validating token access rules against request {{(pid=80245) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:42:41.708196 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c6d26a5-448a-4671-b124-dfff0efa6c71 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '42d15fc0515945acacbbffcf2356c7a5', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80245) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:42:41.708839 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c6d26a5-448a-4671-b124-dfff0efa6c71 admin admin] REQUEST_METHOD: `GET` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:42:41.709125 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c6d26a5-448a-4671-b124-dfff0efa6c71 admin admin] SCRIPT_NAME: `/identity` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:42:41.709407 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c6d26a5-448a-4671-b124-dfff0efa6c71 admin admin] PATH_INFO: `/v3/roles` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:42:41.710130 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c6d26a5-448a-4671-b124-dfff0efa6c71 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:42:41.711804 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c6d26a5-448a-4671-b124-dfff0efa6c71 admin admin] RBAC: Authorization granted {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:42:41.722711 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: [pid: 80245|app: 0|req: 1408/2807] 10.222.0.47 () {58 vars in 1121 bytes} [Thu Jul 21 08:42:41 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 08:42:41.727077 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e21ad515-faac-49fb-b707-e2c796dd64f3 admin admin] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:42:41.738108 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e21ad515-faac-49fb-b707-e2c796dd64f3 admin admin] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:42:41.742711 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-795d09c0-2cfb-45ec-acc1-0afecda592ef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '42d15fc0515945acacbbffcf2356c7a5', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:42:41.743627 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-795d09c0-2cfb-45ec-acc1-0afecda592ef admin admin] REQUEST_METHOD: `PUT` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:42:41.743911 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-795d09c0-2cfb-45ec-acc1-0afecda592ef admin admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:42:41.744183 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-795d09c0-2cfb-45ec-acc1-0afecda592ef admin admin] PATH_INFO: `/v3/projects/1c63960ae02f4a2096e4762bdbf081dd/users/3de58cf94b33401e85990e6610f3725d/roles/3a074738221e4601b5205715ba87d240` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:42:41.758002 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG dbcounter [-] [80244] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80244) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:42:41.763094 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-795d09c0-2cfb-45ec-acc1-0afecda592ef admin admin] RBAC: Authorizing `identity:create_grant(project_id=1c63960ae02f4a2096e4762bdbf081dd, user_id=3de58cf94b33401e85990e6610f3725d, role_id=3a074738221e4601b5205715ba87d240)` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:42:41.766702 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-795d09c0-2cfb-45ec-acc1-0afecda592ef admin admin] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:42:41.780241 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 1400/2808] 10.222.0.47 () {60 vars in 1482 bytes} [Thu Jul 21 08:42:41 2022] PUT /identity/v3/projects/1c63960ae02f4a2096e4762bdbf081dd/users/3de58cf94b33401e85990e6610f3725d/roles/3a074738221e4601b5205715ba87d240 => generated 0 bytes in 54 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 08:42:41.791791 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4df571aa-343b-4ccf-aba7-9ba2059a0680 None None] REQUEST_METHOD: `POST` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:42:41.792085 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4df571aa-343b-4ccf-aba7-9ba2059a0680 None None] SCRIPT_NAME: `/identity` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:42:41.792369 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4df571aa-343b-4ccf-aba7-9ba2059a0680 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:42:41.821680 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.auth.core [None req-4df571aa-343b-4ccf-aba7-9ba2059a0680 None None] MFA Rules not processed for user `3de58cf94b33401e85990e6610f3725d`. Rule list: `[]` (Enabled: `True`). {{(pid=80245) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 08:42:41.823317 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.fernet_utils [None req-4df571aa-343b-4ccf-aba7-9ba2059a0680 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80245) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 08:42:41.888302 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: [pid: 80245|app: 0|req: 1409/2809] 10.222.0.47 () {58 vars in 957 bytes} [Thu Jul 21 08:42:41 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 08:42:41.893195 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-795d09c0-2cfb-45ec-acc1-0afecda592ef admin admin] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:42:41.955791 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-795d09c0-2cfb-45ec-acc1-0afecda592ef admin admin] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:42:41.960492 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcc1dd0a-20ba-4750-877a-f0293751ed30 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '42d15fc0515945acacbbffcf2356c7a5', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:42:41.961153 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcc1dd0a-20ba-4750-877a-f0293751ed30 admin admin] REQUEST_METHOD: `POST` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:42:41.961455 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcc1dd0a-20ba-4750-877a-f0293751ed30 admin admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:42:41.961713 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcc1dd0a-20ba-4750-877a-f0293751ed30 admin admin] PATH_INFO: `/v3/projects` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:42:41.962409 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcc1dd0a-20ba-4750-877a-f0293751ed30 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:42:41.964343 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcc1dd0a-20ba-4750-877a-f0293751ed30 admin admin] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:42:41.993891 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 1401/2810] 10.222.0.47 () {60 vars in 1152 bytes} [Thu Jul 21 08:42:41 2022] POST /identity/v3/projects => generated 387 bytes in 101 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 08:42:41.998566 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4df571aa-343b-4ccf-aba7-9ba2059a0680 None None] Authenticating user token {{(pid=80245) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:42:42.009459 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4df571aa-343b-4ccf-aba7-9ba2059a0680 None None] Validating token access rules against request {{(pid=80245) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:42:42.067208 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f82cba50-a4de-4873-b2d9-b8960634ef06 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '42d15fc0515945acacbbffcf2356c7a5', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80245) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:42:42.067846 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f82cba50-a4de-4873-b2d9-b8960634ef06 admin admin] REQUEST_METHOD: `POST` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:42:42.068135 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f82cba50-a4de-4873-b2d9-b8960634ef06 admin admin] SCRIPT_NAME: `/identity` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:42:42.068419 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f82cba50-a4de-4873-b2d9-b8960634ef06 admin admin] PATH_INFO: `/v3/users` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:42:42.069131 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f82cba50-a4de-4873-b2d9-b8960634ef06 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:42:42.071096 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f82cba50-a4de-4873-b2d9-b8960634ef06 admin admin] RBAC: Authorization granted {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:42:42.098006 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: [pid: 80245|app: 0|req: 1410/2811] 10.222.0.47 () {60 vars in 1143 bytes} [Thu Jul 21 08:42:41 2022] POST /identity/v3/users => generated 300 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 08:42:42.099197 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcc1dd0a-20ba-4750-877a-f0293751ed30 admin admin] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:42:42.110592 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcc1dd0a-20ba-4750-877a-f0293751ed30 admin admin] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:42:42.115439 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bf940ba-5168-4475-b0f5-bd32af201c05 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '42d15fc0515945acacbbffcf2356c7a5', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:42:42.116107 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bf940ba-5168-4475-b0f5-bd32af201c05 admin admin] REQUEST_METHOD: `GET` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:42:42.116446 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bf940ba-5168-4475-b0f5-bd32af201c05 admin admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:42:42.116724 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bf940ba-5168-4475-b0f5-bd32af201c05 admin admin] PATH_INFO: `/v3/roles` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:42:42.117465 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bf940ba-5168-4475-b0f5-bd32af201c05 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:42:42.119093 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bf940ba-5168-4475-b0f5-bd32af201c05 admin admin] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:42:42.130177 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 1402/2812] 10.222.0.47 () {58 vars in 1121 bytes} [Thu Jul 21 08:42:42 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 21 08:42:42.134203 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f82cba50-a4de-4873-b2d9-b8960634ef06 admin admin] Authenticating user token {{(pid=80245) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:42:42.144398 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f82cba50-a4de-4873-b2d9-b8960634ef06 admin admin] Validating token access rules against request {{(pid=80245) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:42:42.149032 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-626bfc30-ba3c-481e-af36-859d73d1d0b3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '42d15fc0515945acacbbffcf2356c7a5', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80245) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:42:42.149911 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-626bfc30-ba3c-481e-af36-859d73d1d0b3 admin admin] REQUEST_METHOD: `PUT` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:42:42.150197 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-626bfc30-ba3c-481e-af36-859d73d1d0b3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:42:42.150472 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-626bfc30-ba3c-481e-af36-859d73d1d0b3 admin admin] PATH_INFO: `/v3/projects/9ed075d51a23430cb20df232e7066592/users/adea178335ce4b86aeb3f3f0caba9be9/roles/e29870f6e9e2411a9db0d5bf45f32ca5` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:42:42.166133 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-626bfc30-ba3c-481e-af36-859d73d1d0b3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9ed075d51a23430cb20df232e7066592, user_id=adea178335ce4b86aeb3f3f0caba9be9, role_id=e29870f6e9e2411a9db0d5bf45f32ca5)` {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:42:42.168921 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-626bfc30-ba3c-481e-af36-859d73d1d0b3 admin admin] RBAC: Authorization granted {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:42:42.177491 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG dbcounter [-] [80245] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=80245) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:42:42.183661 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: [pid: 80245|app: 0|req: 1411/2813] 10.222.0.47 () {60 vars in 1482 bytes} [Thu Jul 21 08:42:42 2022] PUT /identity/v3/projects/9ed075d51a23430cb20df232e7066592/users/adea178335ce4b86aeb3f3f0caba9be9/roles/e29870f6e9e2411a9db0d5bf45f32ca5 => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 08:42:42.187647 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bf940ba-5168-4475-b0f5-bd32af201c05 admin admin] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:42:42.197474 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bf940ba-5168-4475-b0f5-bd32af201c05 admin admin] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:42:42.256526 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5fa7e64-866a-4f2f-9c56-53260b684029 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '42d15fc0515945acacbbffcf2356c7a5', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:42:42.257199 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5fa7e64-866a-4f2f-9c56-53260b684029 admin admin] REQUEST_METHOD: `GET` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:42:42.257503 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5fa7e64-866a-4f2f-9c56-53260b684029 admin admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:42:42.257912 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5fa7e64-866a-4f2f-9c56-53260b684029 admin admin] PATH_INFO: `/v3/roles` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:42:42.258664 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5fa7e64-866a-4f2f-9c56-53260b684029 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:42:42.260435 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5fa7e64-866a-4f2f-9c56-53260b684029 admin admin] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:42:42.270745 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 1403/2814] 10.222.0.47 () {58 vars in 1121 bytes} [Thu Jul 21 08:42:42 2022] GET /identity/v3/roles => generated 1420 bytes in 84 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 08:42:42.274992 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-626bfc30-ba3c-481e-af36-859d73d1d0b3 admin admin] Authenticating user token {{(pid=80245) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:42:42.285766 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-626bfc30-ba3c-481e-af36-859d73d1d0b3 admin admin] Validating token access rules against request {{(pid=80245) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:42:42.291147 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20d582b9-4f04-4158-90cd-ec44815839ed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '42d15fc0515945acacbbffcf2356c7a5', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80245) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:42:42.292075 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20d582b9-4f04-4158-90cd-ec44815839ed admin admin] REQUEST_METHOD: `PUT` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:42:42.292365 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20d582b9-4f04-4158-90cd-ec44815839ed admin admin] SCRIPT_NAME: `/identity` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:42:42.292651 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20d582b9-4f04-4158-90cd-ec44815839ed admin admin] PATH_INFO: `/v3/projects/9ed075d51a23430cb20df232e7066592/users/adea178335ce4b86aeb3f3f0caba9be9/roles/e29870f6e9e2411a9db0d5bf45f32ca5` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:42:42.294667 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20d582b9-4f04-4158-90cd-ec44815839ed admin admin] RBAC: Authorizing `identity:create_grant(project_id=9ed075d51a23430cb20df232e7066592, user_id=adea178335ce4b86aeb3f3f0caba9be9, role_id=e29870f6e9e2411a9db0d5bf45f32ca5)` {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:42:42.297871 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20d582b9-4f04-4158-90cd-ec44815839ed admin admin] RBAC: Authorization granted {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:42:42.311460 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: [pid: 80245|app: 0|req: 1412/2815] 10.222.0.47 () {60 vars in 1482 bytes} [Thu Jul 21 08:42:42 2022] PUT /identity/v3/projects/9ed075d51a23430cb20df232e7066592/users/adea178335ce4b86aeb3f3f0caba9be9/roles/e29870f6e9e2411a9db0d5bf45f32ca5 => generated 0 bytes in 37 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 08:42:42.322092 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2cb7c8a-4f0a-41e0-ad6d-8a4544e10be8 None None] REQUEST_METHOD: `POST` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:42:42.322356 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2cb7c8a-4f0a-41e0-ad6d-8a4544e10be8 None None] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:42:42.322676 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2cb7c8a-4f0a-41e0-ad6d-8a4544e10be8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:42:42.350131 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.auth.core [None req-a2cb7c8a-4f0a-41e0-ad6d-8a4544e10be8 None None] MFA Rules not processed for user `adea178335ce4b86aeb3f3f0caba9be9`. Rule list: `[]` (Enabled: `True`). {{(pid=80244) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 08:42:42.351436 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.fernet_utils [None req-a2cb7c8a-4f0a-41e0-ad6d-8a4544e10be8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80244) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 08:42:42.397201 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG dbcounter [-] [80244] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80244) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:42:42.419926 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 1404/2816] 10.222.0.47 () {58 vars in 957 bytes} [Thu Jul 21 08:42:42 2022] POST /identity/v3/auth/tokens => generated 3321 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 08:42:42.430836 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20d582b9-4f04-4158-90cd-ec44815839ed admin admin] Authenticating user token {{(pid=80245) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:42:42.432603 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.fernet_utils [None req-20d582b9-4f04-4158-90cd-ec44815839ed admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80245) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 08:42:42.489907 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20d582b9-4f04-4158-90cd-ec44815839ed admin admin] Validating token access rules against request {{(pid=80245) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:42:42.493651 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0f838cc-3048-453b-aeb4-369b2979b4be service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4f40410b7674687ab5362ac982248ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4faef13aa50e4e36b6c265fb1398fa14', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80245) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:42:42.494304 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0f838cc-3048-453b-aeb4-369b2979b4be service cinder] REQUEST_METHOD: `GET` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:42:42.494591 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0f838cc-3048-453b-aeb4-369b2979b4be service cinder] SCRIPT_NAME: `/identity` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:42:42.494870 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0f838cc-3048-453b-aeb4-369b2979b4be service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:42:42.504343 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0f838cc-3048-453b-aeb4-369b2979b4be service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:42:42.505763 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0f838cc-3048-453b-aeb4-369b2979b4be service cinder] RBAC: Authorization granted {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:42:42.516152 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: [pid: 80245|app: 0|req: 1413/2817] 10.222.0.47 () {60 vars in 1365 bytes} [Thu Jul 21 08:42:42 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 08:42:43.040751 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2cb7c8a-4f0a-41e0-ad6d-8a4544e10be8 None None] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:42:43.101808 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2cb7c8a-4f0a-41e0-ad6d-8a4544e10be8 None None] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:42:43.106428 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-635c0f0d-b924-4048-84a9-4df09b3cc5f1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '42d15fc0515945acacbbffcf2356c7a5', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:42:43.107299 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-635c0f0d-b924-4048-84a9-4df09b3cc5f1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:42:43.107617 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-635c0f0d-b924-4048-84a9-4df09b3cc5f1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:42:43.107895 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-635c0f0d-b924-4048-84a9-4df09b3cc5f1 admin admin] PATH_INFO: `/v3/users/3de58cf94b33401e85990e6610f3725d` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:42:43.109086 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-635c0f0d-b924-4048-84a9-4df09b3cc5f1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=3de58cf94b33401e85990e6610f3725d)` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:42:43.111067 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-635c0f0d-b924-4048-84a9-4df09b3cc5f1 admin admin] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:42:43.161851 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.notifications [None req-635c0f0d-b924-4048-84a9-4df09b3cc5f1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3de58cf94b33401e85990e6610f3725d'} {{(pid=80244) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 08:42:43.170230 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.notifications [None req-635c0f0d-b924-4048-84a9-4df09b3cc5f1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3de58cf94b33401e85990e6610f3725d'} {{(pid=80244) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 08:42:43.183149 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.notifications [None req-635c0f0d-b924-4048-84a9-4df09b3cc5f1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3de58cf94b33401e85990e6610f3725d'} {{(pid=80244) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 08:42:43.183834 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.notifications [None req-635c0f0d-b924-4048-84a9-4df09b3cc5f1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3de58cf94b33401e85990e6610f3725d'} {{(pid=80244) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 08:42:43.191403 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.notifications [None req-635c0f0d-b924-4048-84a9-4df09b3cc5f1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3de58cf94b33401e85990e6610f3725d'} {{(pid=80244) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 08:42:43.197376 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 1405/2818] 10.222.0.47 () {58 vars in 1223 bytes} [Thu Jul 21 08:42:43 2022] DELETE /identity/v3/users/3de58cf94b33401e85990e6610f3725d => generated 0 bytes in 157 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 08:42:43.202552 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c65e1e69-fafb-48ba-a65e-bcbdb88e97b7 None None] REQUEST_METHOD: `POST` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:42:43.203008 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c65e1e69-fafb-48ba-a65e-bcbdb88e97b7 None None] SCRIPT_NAME: `/identity` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:42:43.203190 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c65e1e69-fafb-48ba-a65e-bcbdb88e97b7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:42:43.220561 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.auth.core [None req-c65e1e69-fafb-48ba-a65e-bcbdb88e97b7 None None] MFA Rules not processed for user `2dce40b8d74c41ec94cc36e6ff05d412`. Rule list: `[]` (Enabled: `True`). {{(pid=80245) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 08:42:43.221764 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.fernet_utils [None req-c65e1e69-fafb-48ba-a65e-bcbdb88e97b7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80245) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 08:42:43.284746 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: [pid: 80245|app: 0|req: 1414/2819] 10.222.0.47 () {58 vars in 957 bytes} [Thu Jul 21 08:42:43 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 08:42:43.294684 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-635c0f0d-b924-4048-84a9-4df09b3cc5f1 admin admin] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:42:43.296855 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.fernet_utils [None req-635c0f0d-b924-4048-84a9-4df09b3cc5f1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80244) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 08:42:43.361659 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-635c0f0d-b924-4048-84a9-4df09b3cc5f1 admin admin] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:42:43.365994 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85d89426-cdf9-4d39-87de-3e4fd3586f8b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20ca3c3bd3774691b3374cfbfec6e494', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4faef13aa50e4e36b6c265fb1398fa14', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:42:43.367006 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85d89426-cdf9-4d39-87de-3e4fd3586f8b service neutron] REQUEST_METHOD: `GET` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:42:43.367006 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85d89426-cdf9-4d39-87de-3e4fd3586f8b service neutron] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:42:43.367243 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85d89426-cdf9-4d39-87de-3e4fd3586f8b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:42:43.383053 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85d89426-cdf9-4d39-87de-3e4fd3586f8b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:42:43.383778 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85d89426-cdf9-4d39-87de-3e4fd3586f8b service neutron] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:42:43.394654 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 1406/2820] 10.222.0.47 () {60 vars in 1365 bytes} [Thu Jul 21 08:42:43 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 100 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 08:42:43.683250 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c65e1e69-fafb-48ba-a65e-bcbdb88e97b7 None None] Authenticating user token {{(pid=80245) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:42:43.685047 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.fernet_utils [None req-c65e1e69-fafb-48ba-a65e-bcbdb88e97b7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80245) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 08:42:43.697928 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c65e1e69-fafb-48ba-a65e-bcbdb88e97b7 None None] Validating token access rules against request {{(pid=80245) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:42:43.703147 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31e285d7-f02c-429e-8899-23213c7069aa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '42d15fc0515945acacbbffcf2356c7a5', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80245) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:42:43.704187 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31e285d7-f02c-429e-8899-23213c7069aa admin admin] REQUEST_METHOD: `DELETE` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:42:43.704397 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31e285d7-f02c-429e-8899-23213c7069aa admin admin] SCRIPT_NAME: `/identity` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:42:43.704739 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31e285d7-f02c-429e-8899-23213c7069aa admin admin] PATH_INFO: `/v3/projects/1c63960ae02f4a2096e4762bdbf081dd` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:42:43.705948 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31e285d7-f02c-429e-8899-23213c7069aa admin admin] RBAC: Authorizing `identity:delete_project(project_id=1c63960ae02f4a2096e4762bdbf081dd)` {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:42:43.707943 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31e285d7-f02c-429e-8899-23213c7069aa admin admin] RBAC: Authorization granted {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:42:43.759859 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.notifications [None req-31e285d7-f02c-429e-8899-23213c7069aa admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1c63960ae02f4a2096e4762bdbf081dd', 'request_id': 'req-31e285d7-f02c-429e-8899-23213c7069aa'} {{(pid=80245) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 08:42:43.766062 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.notifications [None req-31e285d7-f02c-429e-8899-23213c7069aa admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1c63960ae02f4a2096e4762bdbf081dd', 'request_id': 'req-31e285d7-f02c-429e-8899-23213c7069aa'} {{(pid=80245) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 08:42:43.771761 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.notifications [None req-31e285d7-f02c-429e-8899-23213c7069aa admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1c63960ae02f4a2096e4762bdbf081dd', 'request_id': 'req-31e285d7-f02c-429e-8899-23213c7069aa'} {{(pid=80245) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 08:42:43.784431 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.notifications [None req-31e285d7-f02c-429e-8899-23213c7069aa admin admin] The token cache is being invalidate because project 1c63960ae02f4a2096e4762bdbf081dd was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80245) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 21 08:42:43.784720 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.notifications [None req-31e285d7-f02c-429e-8899-23213c7069aa admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80245) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 08:42:43.785334 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.notifications [None req-31e285d7-f02c-429e-8899-23213c7069aa admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80245) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 08:42:43.786874 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: [pid: 80245|app: 0|req: 1415/2821] 10.222.0.47 () {58 vars in 1232 bytes} [Thu Jul 21 08:42:43 2022] DELETE /identity/v3/projects/1c63960ae02f4a2096e4762bdbf081dd => generated 0 bytes in 104 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 08:42:43.791106 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85d89426-cdf9-4d39-87de-3e4fd3586f8b service neutron] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:42:43.803697 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85d89426-cdf9-4d39-87de-3e4fd3586f8b service neutron] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:42:43.807452 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG dbcounter [-] [80244] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80244) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:42:43.862457 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e837d1d-8360-4f94-8eab-c3489fa35e28 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '42d15fc0515945acacbbffcf2356c7a5', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:42:43.863415 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e837d1d-8360-4f94-8eab-c3489fa35e28 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:42:43.863650 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e837d1d-8360-4f94-8eab-c3489fa35e28 admin admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:42:43.863928 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e837d1d-8360-4f94-8eab-c3489fa35e28 admin admin] PATH_INFO: `/v3/users/adea178335ce4b86aeb3f3f0caba9be9` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:42:43.865089 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e837d1d-8360-4f94-8eab-c3489fa35e28 admin admin] RBAC: Authorizing `identity:delete_user(user_id=adea178335ce4b86aeb3f3f0caba9be9)` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:42:43.868565 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e837d1d-8360-4f94-8eab-c3489fa35e28 admin admin] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:42:43.916849 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.notifications [None req-3e837d1d-8360-4f94-8eab-c3489fa35e28 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'adea178335ce4b86aeb3f3f0caba9be9'} {{(pid=80244) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 08:42:43.924092 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.notifications [None req-3e837d1d-8360-4f94-8eab-c3489fa35e28 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'adea178335ce4b86aeb3f3f0caba9be9'} {{(pid=80244) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 08:42:43.935856 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.notifications [None req-3e837d1d-8360-4f94-8eab-c3489fa35e28 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'adea178335ce4b86aeb3f3f0caba9be9'} {{(pid=80244) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 08:42:43.936515 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.notifications [None req-3e837d1d-8360-4f94-8eab-c3489fa35e28 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'adea178335ce4b86aeb3f3f0caba9be9'} {{(pid=80244) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 08:42:43.943423 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.notifications [None req-3e837d1d-8360-4f94-8eab-c3489fa35e28 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'adea178335ce4b86aeb3f3f0caba9be9'} {{(pid=80244) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 08:42:43.949435 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 1407/2822] 10.222.0.47 () {58 vars in 1223 bytes} [Thu Jul 21 08:42:43 2022] DELETE /identity/v3/users/adea178335ce4b86aeb3f3f0caba9be9 => generated 0 bytes in 159 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 08:42:44.240647 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31e285d7-f02c-429e-8899-23213c7069aa admin admin] Authenticating user token {{(pid=80245) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:42:44.242246 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.fernet_utils [None req-31e285d7-f02c-429e-8899-23213c7069aa admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80245) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 08:42:44.256548 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31e285d7-f02c-429e-8899-23213c7069aa admin admin] Validating token access rules against request {{(pid=80245) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:42:44.265674 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG dbcounter [-] [80245] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=80245) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:42:44.317129 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ed7f611-311e-499f-8614-68c268c2167e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '42d15fc0515945acacbbffcf2356c7a5', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80245) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:42:44.317999 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ed7f611-311e-499f-8614-68c268c2167e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:42:44.318281 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ed7f611-311e-499f-8614-68c268c2167e admin admin] SCRIPT_NAME: `/identity` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:42:44.318557 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ed7f611-311e-499f-8614-68c268c2167e admin admin] PATH_INFO: `/v3/projects/9ed075d51a23430cb20df232e7066592` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:42:44.319755 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ed7f611-311e-499f-8614-68c268c2167e admin admin] RBAC: Authorizing `identity:delete_project(project_id=9ed075d51a23430cb20df232e7066592)` {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:42:44.321645 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ed7f611-311e-499f-8614-68c268c2167e admin admin] RBAC: Authorization granted {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:42:44.372594 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.notifications [None req-4ed7f611-311e-499f-8614-68c268c2167e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9ed075d51a23430cb20df232e7066592', 'request_id': 'req-4ed7f611-311e-499f-8614-68c268c2167e'} {{(pid=80245) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 08:42:44.379686 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.notifications [None req-4ed7f611-311e-499f-8614-68c268c2167e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9ed075d51a23430cb20df232e7066592', 'request_id': 'req-4ed7f611-311e-499f-8614-68c268c2167e'} {{(pid=80245) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 08:42:44.385526 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.notifications [None req-4ed7f611-311e-499f-8614-68c268c2167e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9ed075d51a23430cb20df232e7066592', 'request_id': 'req-4ed7f611-311e-499f-8614-68c268c2167e'} {{(pid=80245) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 08:42:44.398743 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.notifications [None req-4ed7f611-311e-499f-8614-68c268c2167e admin admin] The token cache is being invalidate because project 9ed075d51a23430cb20df232e7066592 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80245) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 21 08:42:44.399037 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.notifications [None req-4ed7f611-311e-499f-8614-68c268c2167e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80245) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 08:42:44.399733 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.notifications [None req-4ed7f611-311e-499f-8614-68c268c2167e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80245) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 08:42:44.401399 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: [pid: 80245|app: 0|req: 1416/2823] 10.222.0.47 () {58 vars in 1232 bytes} [Thu Jul 21 08:42:44 2022] DELETE /identity/v3/projects/9ed075d51a23430cb20df232e7066592 => generated 0 bytes in 161 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 08:42:44.406413 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93387eea-c60f-4abe-a46b-fdb5317cc54e None None] REQUEST_METHOD: `POST` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:42:44.406888 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93387eea-c60f-4abe-a46b-fdb5317cc54e None None] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:42:44.406989 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93387eea-c60f-4abe-a46b-fdb5317cc54e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:42:44.424835 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.auth.core [None req-93387eea-c60f-4abe-a46b-fdb5317cc54e None None] MFA Rules not processed for user `2dce40b8d74c41ec94cc36e6ff05d412`. Rule list: `[]` (Enabled: `True`). {{(pid=80244) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 08:42:44.426022 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.fernet_utils [None req-93387eea-c60f-4abe-a46b-fdb5317cc54e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80244) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 08:42:44.485855 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 1408/2824] 10.222.0.47 () {58 vars in 957 bytes} [Thu Jul 21 08:42:44 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 08:42:44.493716 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57f9af86-cd44-4fb6-9ab7-efe8a4a064a2 None None] REQUEST_METHOD: `POST` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:42:44.494013 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57f9af86-cd44-4fb6-9ab7-efe8a4a064a2 None None] SCRIPT_NAME: `/identity` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:42:44.494318 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57f9af86-cd44-4fb6-9ab7-efe8a4a064a2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:42:44.513324 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.auth.core [None req-57f9af86-cd44-4fb6-9ab7-efe8a4a064a2 None None] MFA Rules not processed for user `2dce40b8d74c41ec94cc36e6ff05d412`. Rule list: `[]` (Enabled: `True`). {{(pid=80245) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 08:42:44.515184 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.fernet_utils [None req-57f9af86-cd44-4fb6-9ab7-efe8a4a064a2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80245) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 08:42:44.523176 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: [pid: 80245|app: 0|req: 1417/2825] 10.222.0.47 () {58 vars in 957 bytes} [Thu Jul 21 08:42:44 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 08:42:44.527510 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93387eea-c60f-4abe-a46b-fdb5317cc54e None None] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:42:44.537274 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93387eea-c60f-4abe-a46b-fdb5317cc54e None None] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:42:44.541828 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15ac0ca2-d0f8-467e-b961-8cec72860006 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '42d15fc0515945acacbbffcf2356c7a5', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:42:44.542502 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15ac0ca2-d0f8-467e-b961-8cec72860006 admin admin] REQUEST_METHOD: `GET` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:42:44.542790 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15ac0ca2-d0f8-467e-b961-8cec72860006 admin admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:42:44.543070 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15ac0ca2-d0f8-467e-b961-8cec72860006 admin admin] PATH_INFO: `/v3/domains` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:42:44.543879 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15ac0ca2-d0f8-467e-b961-8cec72860006 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80244) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 08:42:44.544187 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15ac0ca2-d0f8-467e-b961-8cec72860006 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:42:44.545836 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15ac0ca2-d0f8-467e-b961-8cec72860006 admin admin] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:42:44.557520 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 1409/2826] 10.222.0.47 () {58 vars in 1152 bytes} [Thu Jul 21 08:42:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 08:42:44.562461 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57f9af86-cd44-4fb6-9ab7-efe8a4a064a2 None None] Authenticating user token {{(pid=80245) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:42:44.571815 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57f9af86-cd44-4fb6-9ab7-efe8a4a064a2 None None] Validating token access rules against request {{(pid=80245) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:42:44.577057 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7211cf85-f179-4d05-81f9-90fb4aa538ed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '42d15fc0515945acacbbffcf2356c7a5', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80245) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:42:44.577704 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7211cf85-f179-4d05-81f9-90fb4aa538ed admin admin] REQUEST_METHOD: `POST` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:42:44.578011 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7211cf85-f179-4d05-81f9-90fb4aa538ed admin admin] SCRIPT_NAME: `/identity` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:42:44.578254 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7211cf85-f179-4d05-81f9-90fb4aa538ed admin admin] PATH_INFO: `/v3/projects` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:42:44.579004 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7211cf85-f179-4d05-81f9-90fb4aa538ed admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:42:44.580870 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7211cf85-f179-4d05-81f9-90fb4aa538ed admin admin] RBAC: Authorization granted {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:42:44.606475 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: [pid: 80245|app: 0|req: 1418/2827] 10.222.0.47 () {60 vars in 1152 bytes} [Thu Jul 21 08:42:44 2022] POST /identity/v3/projects => generated 371 bytes in 44 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 08:42:44.610537 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15ac0ca2-d0f8-467e-b961-8cec72860006 admin admin] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:42:44.620725 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15ac0ca2-d0f8-467e-b961-8cec72860006 admin admin] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:42:44.630437 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG dbcounter [-] [80244] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80244) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:42:44.681959 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-734ab7ad-ce59-4c65-a1f3-86de5a53e067 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '42d15fc0515945acacbbffcf2356c7a5', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:42:44.682644 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-734ab7ad-ce59-4c65-a1f3-86de5a53e067 admin admin] REQUEST_METHOD: `POST` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:42:44.682933 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-734ab7ad-ce59-4c65-a1f3-86de5a53e067 admin admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:42:44.683213 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-734ab7ad-ce59-4c65-a1f3-86de5a53e067 admin admin] PATH_INFO: `/v3/users` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:42:44.683977 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-734ab7ad-ce59-4c65-a1f3-86de5a53e067 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:42:44.686420 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-734ab7ad-ce59-4c65-a1f3-86de5a53e067 admin admin] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:42:44.711090 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 1410/2828] 10.222.0.47 () {60 vars in 1143 bytes} [Thu Jul 21 08:42:44 2022] POST /identity/v3/users => generated 286 bytes in 98 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 21 08:42:44.712250 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7211cf85-f179-4d05-81f9-90fb4aa538ed admin admin] Authenticating user token {{(pid=80245) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:42:44.722685 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7211cf85-f179-4d05-81f9-90fb4aa538ed admin admin] Validating token access rules against request {{(pid=80245) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:42:44.727424 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec3f00af-b088-485c-ac38-a3ee67cbbba0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '42d15fc0515945acacbbffcf2356c7a5', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80245) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:42:44.728097 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec3f00af-b088-485c-ac38-a3ee67cbbba0 admin admin] REQUEST_METHOD: `GET` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:42:44.728380 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec3f00af-b088-485c-ac38-a3ee67cbbba0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:42:44.728656 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec3f00af-b088-485c-ac38-a3ee67cbbba0 admin admin] PATH_INFO: `/v3/roles` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:42:44.729483 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec3f00af-b088-485c-ac38-a3ee67cbbba0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:42:44.731076 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec3f00af-b088-485c-ac38-a3ee67cbbba0 admin admin] RBAC: Authorization granted {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:42:44.742856 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: [pid: 80245|app: 0|req: 1419/2829] 10.222.0.47 () {58 vars in 1121 bytes} [Thu Jul 21 08:42:44 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 08:42:44.747069 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-734ab7ad-ce59-4c65-a1f3-86de5a53e067 admin admin] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:42:44.761715 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-734ab7ad-ce59-4c65-a1f3-86de5a53e067 admin admin] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:42:44.763658 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0724006a-3d6c-4052-b09a-c4c2db2c7634 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '42d15fc0515945acacbbffcf2356c7a5', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:42:44.764349 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0724006a-3d6c-4052-b09a-c4c2db2c7634 admin admin] REQUEST_METHOD: `GET` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:42:44.764812 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0724006a-3d6c-4052-b09a-c4c2db2c7634 admin admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:42:44.765153 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0724006a-3d6c-4052-b09a-c4c2db2c7634 admin admin] PATH_INFO: `/v3/roles` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:42:44.765896 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0724006a-3d6c-4052-b09a-c4c2db2c7634 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:42:44.767575 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0724006a-3d6c-4052-b09a-c4c2db2c7634 admin admin] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:42:44.781969 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 1411/2830] 10.222.0.47 () {58 vars in 1121 bytes} [Thu Jul 21 08:42:44 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 08:42:44.783564 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec3f00af-b088-485c-ac38-a3ee67cbbba0 admin admin] Authenticating user token {{(pid=80245) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:42:44.793612 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec3f00af-b088-485c-ac38-a3ee67cbbba0 admin admin] Validating token access rules against request {{(pid=80245) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:42:44.799557 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42ec6fdb-c9be-4f9c-bbbb-041730dd20ff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '42d15fc0515945acacbbffcf2356c7a5', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80245) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:42:44.800557 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42ec6fdb-c9be-4f9c-bbbb-041730dd20ff admin admin] REQUEST_METHOD: `PUT` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:42:44.800873 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42ec6fdb-c9be-4f9c-bbbb-041730dd20ff admin admin] SCRIPT_NAME: `/identity` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:42:44.801151 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42ec6fdb-c9be-4f9c-bbbb-041730dd20ff admin admin] PATH_INFO: `/v3/projects/4813fdf8c25f4b988cf6a2b26d5b9c71/users/8325d08910cb422487f3d2f01410e49f/roles/3a074738221e4601b5205715ba87d240` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:42:44.820001 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42ec6fdb-c9be-4f9c-bbbb-041730dd20ff admin admin] RBAC: Authorizing `identity:create_grant(project_id=4813fdf8c25f4b988cf6a2b26d5b9c71, user_id=8325d08910cb422487f3d2f01410e49f, role_id=3a074738221e4601b5205715ba87d240)` {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:42:44.822700 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42ec6fdb-c9be-4f9c-bbbb-041730dd20ff admin admin] RBAC: Authorization granted {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:42:44.843793 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: [pid: 80245|app: 0|req: 1420/2831] 10.222.0.47 () {60 vars in 1482 bytes} [Thu Jul 21 08:42:44 2022] PUT /identity/v3/projects/4813fdf8c25f4b988cf6a2b26d5b9c71/users/8325d08910cb422487f3d2f01410e49f/roles/3a074738221e4601b5205715ba87d240 => generated 0 bytes in 61 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 08:42:44.849102 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c0346c2-d559-430f-9710-d394288bc4f2 None None] REQUEST_METHOD: `POST` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:42:44.849394 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c0346c2-d559-430f-9710-d394288bc4f2 None None] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:42:44.849672 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c0346c2-d559-430f-9710-d394288bc4f2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:42:44.868941 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.auth.core [None req-0c0346c2-d559-430f-9710-d394288bc4f2 None None] MFA Rules not processed for user `2dce40b8d74c41ec94cc36e6ff05d412`. Rule list: `[]` (Enabled: `True`). {{(pid=80244) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 08:42:44.870172 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.fernet_utils [None req-0c0346c2-d559-430f-9710-d394288bc4f2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80244) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 08:42:44.930750 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 1412/2832] 10.222.0.47 () {58 vars in 957 bytes} [Thu Jul 21 08:42:44 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 08:42:44.941871 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42ec6fdb-c9be-4f9c-bbbb-041730dd20ff admin admin] Authenticating user token {{(pid=80245) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:42:44.944322 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.fernet_utils [None req-42ec6fdb-c9be-4f9c-bbbb-041730dd20ff admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80245) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 08:42:45.120201 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG dbcounter [-] [80245] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80245) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:42:45.137386 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42ec6fdb-c9be-4f9c-bbbb-041730dd20ff admin admin] Validating token access rules against request {{(pid=80245) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:42:45.141162 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9222ab6b-162e-4410-9394-b30abe47a05b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20ca3c3bd3774691b3374cfbfec6e494', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4faef13aa50e4e36b6c265fb1398fa14', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80245) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:42:45.141812 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9222ab6b-162e-4410-9394-b30abe47a05b service neutron] REQUEST_METHOD: `GET` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:42:45.142096 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9222ab6b-162e-4410-9394-b30abe47a05b service neutron] SCRIPT_NAME: `/identity` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:42:45.142375 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9222ab6b-162e-4410-9394-b30abe47a05b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:42:45.152040 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9222ab6b-162e-4410-9394-b30abe47a05b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:42:45.153071 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9222ab6b-162e-4410-9394-b30abe47a05b service neutron] RBAC: Authorization granted {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:42:45.163608 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: [pid: 80245|app: 0|req: 1421/2833] 10.222.0.47 () {60 vars in 1365 bytes} [Thu Jul 21 08:42:44 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 223 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 08:42:46.038515 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35a22f00-8dfd-4995-8a28-8323d1a52349 None None] REQUEST_METHOD: `POST` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:42:46.039062 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35a22f00-8dfd-4995-8a28-8323d1a52349 None None] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:42:46.039062 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35a22f00-8dfd-4995-8a28-8323d1a52349 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:42:46.066594 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.auth.core [None req-35a22f00-8dfd-4995-8a28-8323d1a52349 None None] MFA Rules not processed for user `8325d08910cb422487f3d2f01410e49f`. Rule list: `[]` (Enabled: `True`). {{(pid=80244) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 08:42:46.067837 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.fernet_utils [None req-35a22f00-8dfd-4995-8a28-8323d1a52349 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80244) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 08:42:46.144789 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 1413/2834] 10.222.0.47 () {58 vars in 957 bytes} [Thu Jul 21 08:42:46 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 08:42:46.149801 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9222ab6b-162e-4410-9394-b30abe47a05b service neutron] Authenticating user token {{(pid=80245) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:42:46.160040 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9222ab6b-162e-4410-9394-b30abe47a05b service neutron] Validating token access rules against request {{(pid=80245) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:42:46.164660 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19ae1c8b-7479-4bb0-9732-57f71344a4f3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '42d15fc0515945acacbbffcf2356c7a5', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80245) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:42:46.165320 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19ae1c8b-7479-4bb0-9732-57f71344a4f3 admin admin] REQUEST_METHOD: `POST` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:42:46.165603 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19ae1c8b-7479-4bb0-9732-57f71344a4f3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:42:46.165881 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19ae1c8b-7479-4bb0-9732-57f71344a4f3 admin admin] PATH_INFO: `/v3/projects` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:42:46.166609 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19ae1c8b-7479-4bb0-9732-57f71344a4f3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:42:46.168527 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19ae1c8b-7479-4bb0-9732-57f71344a4f3 admin admin] RBAC: Authorization granted {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:42:46.196283 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: [pid: 80245|app: 0|req: 1422/2835] 10.222.0.47 () {60 vars in 1152 bytes} [Thu Jul 21 08:42:46 2022] POST /identity/v3/projects => generated 371 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 08:42:46.200283 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35a22f00-8dfd-4995-8a28-8323d1a52349 None None] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:42:46.211988 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35a22f00-8dfd-4995-8a28-8323d1a52349 None None] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:42:46.230949 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG dbcounter [-] [80244] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80244) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:42:46.271875 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c837f95c-1375-4f1a-845f-f2b82b896898 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '42d15fc0515945acacbbffcf2356c7a5', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:42:46.272549 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c837f95c-1375-4f1a-845f-f2b82b896898 admin admin] REQUEST_METHOD: `POST` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:42:46.272848 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c837f95c-1375-4f1a-845f-f2b82b896898 admin admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:42:46.273104 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c837f95c-1375-4f1a-845f-f2b82b896898 admin admin] PATH_INFO: `/v3/users` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:42:46.273789 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c837f95c-1375-4f1a-845f-f2b82b896898 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:42:46.275734 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c837f95c-1375-4f1a-845f-f2b82b896898 admin admin] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:42:46.296139 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 1414/2836] 10.222.0.47 () {60 vars in 1143 bytes} [Thu Jul 21 08:42:46 2022] POST /identity/v3/users => generated 292 bytes in 96 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 08:42:46.300470 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19ae1c8b-7479-4bb0-9732-57f71344a4f3 admin admin] Authenticating user token {{(pid=80245) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:42:46.311527 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19ae1c8b-7479-4bb0-9732-57f71344a4f3 admin admin] Validating token access rules against request {{(pid=80245) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:42:46.316166 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-855a7ada-0cf4-44f9-adfc-38388bb5ba83 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '42d15fc0515945acacbbffcf2356c7a5', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80245) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:42:46.316829 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-855a7ada-0cf4-44f9-adfc-38388bb5ba83 admin admin] REQUEST_METHOD: `GET` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:42:46.317113 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-855a7ada-0cf4-44f9-adfc-38388bb5ba83 admin admin] SCRIPT_NAME: `/identity` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:42:46.317388 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-855a7ada-0cf4-44f9-adfc-38388bb5ba83 admin admin] PATH_INFO: `/v3/roles` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:42:46.318114 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-855a7ada-0cf4-44f9-adfc-38388bb5ba83 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:42:46.319813 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-855a7ada-0cf4-44f9-adfc-38388bb5ba83 admin admin] RBAC: Authorization granted {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:42:46.330607 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: [pid: 80245|app: 0|req: 1423/2837] 10.222.0.47 () {58 vars in 1121 bytes} [Thu Jul 21 08:42:46 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 08:42:46.334735 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c837f95c-1375-4f1a-845f-f2b82b896898 admin admin] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:42:46.344807 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c837f95c-1375-4f1a-845f-f2b82b896898 admin admin] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:42:46.349598 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc9b3f25-cd8f-4078-a9fe-dd9f3152821b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '42d15fc0515945acacbbffcf2356c7a5', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:42:46.350483 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc9b3f25-cd8f-4078-a9fe-dd9f3152821b admin admin] REQUEST_METHOD: `PUT` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:42:46.350775 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc9b3f25-cd8f-4078-a9fe-dd9f3152821b admin admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:42:46.351279 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc9b3f25-cd8f-4078-a9fe-dd9f3152821b admin admin] PATH_INFO: `/v3/projects/d57e7e762f164e48975b4a7fae9bc114/users/37fac0d86a9047ac8bedfae0556d82b0/roles/e29870f6e9e2411a9db0d5bf45f32ca5` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:42:46.366414 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc9b3f25-cd8f-4078-a9fe-dd9f3152821b admin admin] RBAC: Authorizing `identity:create_grant(project_id=d57e7e762f164e48975b4a7fae9bc114, user_id=37fac0d86a9047ac8bedfae0556d82b0, role_id=e29870f6e9e2411a9db0d5bf45f32ca5)` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:42:46.369119 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc9b3f25-cd8f-4078-a9fe-dd9f3152821b admin admin] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:42:46.382668 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 1415/2838] 10.222.0.47 () {60 vars in 1482 bytes} [Thu Jul 21 08:42:46 2022] PUT /identity/v3/projects/d57e7e762f164e48975b4a7fae9bc114/users/37fac0d86a9047ac8bedfae0556d82b0/roles/e29870f6e9e2411a9db0d5bf45f32ca5 => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 08:42:46.386610 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-855a7ada-0cf4-44f9-adfc-38388bb5ba83 admin admin] Authenticating user token {{(pid=80245) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:42:46.396530 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-855a7ada-0cf4-44f9-adfc-38388bb5ba83 admin admin] Validating token access rules against request {{(pid=80245) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:42:46.451210 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8f4512e-1a1c-4408-a9e0-e7dca03bfaa0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '42d15fc0515945acacbbffcf2356c7a5', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80245) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:42:46.451895 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8f4512e-1a1c-4408-a9e0-e7dca03bfaa0 admin admin] REQUEST_METHOD: `GET` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:42:46.452179 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8f4512e-1a1c-4408-a9e0-e7dca03bfaa0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:42:46.452456 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8f4512e-1a1c-4408-a9e0-e7dca03bfaa0 admin admin] PATH_INFO: `/v3/roles` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:42:46.453188 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8f4512e-1a1c-4408-a9e0-e7dca03bfaa0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:42:46.454759 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8f4512e-1a1c-4408-a9e0-e7dca03bfaa0 admin admin] RBAC: Authorization granted {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:42:46.464448 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: [pid: 80245|app: 0|req: 1424/2839] 10.222.0.47 () {58 vars in 1121 bytes} [Thu Jul 21 08:42:46 2022] GET /identity/v3/roles => generated 1420 bytes in 78 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 08:42:46.468958 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc9b3f25-cd8f-4078-a9fe-dd9f3152821b admin admin] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:42:46.479839 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc9b3f25-cd8f-4078-a9fe-dd9f3152821b admin admin] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:42:46.485343 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75ba10a2-7659-486b-9eb2-88ebfa0bae5b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '42d15fc0515945acacbbffcf2356c7a5', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:42:46.486235 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75ba10a2-7659-486b-9eb2-88ebfa0bae5b admin admin] REQUEST_METHOD: `PUT` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:42:46.486543 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75ba10a2-7659-486b-9eb2-88ebfa0bae5b admin admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:42:46.486818 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75ba10a2-7659-486b-9eb2-88ebfa0bae5b admin admin] PATH_INFO: `/v3/projects/d57e7e762f164e48975b4a7fae9bc114/users/37fac0d86a9047ac8bedfae0556d82b0/roles/e29870f6e9e2411a9db0d5bf45f32ca5` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:42:46.488822 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75ba10a2-7659-486b-9eb2-88ebfa0bae5b admin admin] RBAC: Authorizing `identity:create_grant(project_id=d57e7e762f164e48975b4a7fae9bc114, user_id=37fac0d86a9047ac8bedfae0556d82b0, role_id=e29870f6e9e2411a9db0d5bf45f32ca5)` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:42:46.491479 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75ba10a2-7659-486b-9eb2-88ebfa0bae5b admin admin] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:42:46.502902 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 1416/2840] 10.222.0.47 () {60 vars in 1482 bytes} [Thu Jul 21 08:42:46 2022] PUT /identity/v3/projects/d57e7e762f164e48975b4a7fae9bc114/users/37fac0d86a9047ac8bedfae0556d82b0/roles/e29870f6e9e2411a9db0d5bf45f32ca5 => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 08:42:47.122458 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5842a23f-9a57-4a69-8b77-e7392e6d6f16 None None] REQUEST_METHOD: `POST` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:42:47.122877 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5842a23f-9a57-4a69-8b77-e7392e6d6f16 None None] SCRIPT_NAME: `/identity` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:42:47.123752 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5842a23f-9a57-4a69-8b77-e7392e6d6f16 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:42:47.153846 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.auth.core [None req-5842a23f-9a57-4a69-8b77-e7392e6d6f16 None None] MFA Rules not processed for user `37fac0d86a9047ac8bedfae0556d82b0`. Rule list: `[]` (Enabled: `True`). {{(pid=80245) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 08:42:47.155740 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.fernet_utils [None req-5842a23f-9a57-4a69-8b77-e7392e6d6f16 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80245) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 08:42:47.230651 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: [pid: 80245|app: 0|req: 1425/2841] 10.222.0.47 () {58 vars in 957 bytes} [Thu Jul 21 08:42:47 2022] POST /identity/v3/auth/tokens => generated 3305 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 21 08:42:47.242768 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75ba10a2-7659-486b-9eb2-88ebfa0bae5b admin admin] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:42:47.244496 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.fernet_utils [None req-75ba10a2-7659-486b-9eb2-88ebfa0bae5b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80244) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 08:42:47.314920 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75ba10a2-7659-486b-9eb2-88ebfa0bae5b admin admin] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:42:47.318474 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1f675c7-8f0a-40d5-99ac-b577a0012eec service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4f40410b7674687ab5362ac982248ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4faef13aa50e4e36b6c265fb1398fa14', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:42:47.319113 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1f675c7-8f0a-40d5-99ac-b577a0012eec service cinder] REQUEST_METHOD: `GET` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:42:47.319421 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1f675c7-8f0a-40d5-99ac-b577a0012eec service cinder] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:42:47.319695 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1f675c7-8f0a-40d5-99ac-b577a0012eec service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:42:47.376329 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1f675c7-8f0a-40d5-99ac-b577a0012eec service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:42:47.378015 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1f675c7-8f0a-40d5-99ac-b577a0012eec service cinder] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:42:47.386798 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 1417/2842] 10.222.0.47 () {60 vars in 1365 bytes} [Thu Jul 21 08:42:47 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 145 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 08:42:48.183463 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5842a23f-9a57-4a69-8b77-e7392e6d6f16 None None] Authenticating user token {{(pid=80245) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:42:48.184902 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.fernet_utils [None req-5842a23f-9a57-4a69-8b77-e7392e6d6f16 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80245) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 08:42:48.215179 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG dbcounter [-] [80245] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80245) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:42:48.249647 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5842a23f-9a57-4a69-8b77-e7392e6d6f16 None None] Validating token access rules against request {{(pid=80245) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:42:48.254384 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a51ad70-825e-4215-8c0b-0c9ed43c0e9f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '42d15fc0515945acacbbffcf2356c7a5', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80245) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:42:48.255250 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a51ad70-825e-4215-8c0b-0c9ed43c0e9f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:42:48.255643 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a51ad70-825e-4215-8c0b-0c9ed43c0e9f admin admin] SCRIPT_NAME: `/identity` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:42:48.255957 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a51ad70-825e-4215-8c0b-0c9ed43c0e9f admin admin] PATH_INFO: `/v3/users/0bd219159bc841d1affb14bb0a916834` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:42:48.257036 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a51ad70-825e-4215-8c0b-0c9ed43c0e9f admin admin] RBAC: Authorizing `identity:delete_user(user_id=0bd219159bc841d1affb14bb0a916834)` {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:42:48.258941 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a51ad70-825e-4215-8c0b-0c9ed43c0e9f admin admin] RBAC: Authorization granted {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:42:48.304275 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.notifications [None req-0a51ad70-825e-4215-8c0b-0c9ed43c0e9f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0bd219159bc841d1affb14bb0a916834'} {{(pid=80245) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 08:42:48.311553 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.notifications [None req-0a51ad70-825e-4215-8c0b-0c9ed43c0e9f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0bd219159bc841d1affb14bb0a916834'} {{(pid=80245) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 08:42:48.323288 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.notifications [None req-0a51ad70-825e-4215-8c0b-0c9ed43c0e9f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0bd219159bc841d1affb14bb0a916834'} {{(pid=80245) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 08:42:48.330045 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.notifications [None req-0a51ad70-825e-4215-8c0b-0c9ed43c0e9f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0bd219159bc841d1affb14bb0a916834'} {{(pid=80245) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 08:42:48.330685 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.notifications [None req-0a51ad70-825e-4215-8c0b-0c9ed43c0e9f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0bd219159bc841d1affb14bb0a916834'} {{(pid=80245) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 08:42:48.336577 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: [pid: 80245|app: 0|req: 1426/2843] 10.222.0.47 () {58 vars in 1223 bytes} [Thu Jul 21 08:42:48 2022] DELETE /identity/v3/users/0bd219159bc841d1affb14bb0a916834 => generated 0 bytes in 154 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 08:42:48.341101 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d10f1706-0011-41c0-ac33-be9f4861e7e1 None None] REQUEST_METHOD: `POST` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:42:48.342113 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d10f1706-0011-41c0-ac33-be9f4861e7e1 None None] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:42:48.342226 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d10f1706-0011-41c0-ac33-be9f4861e7e1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:42:48.360252 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.auth.core [None req-d10f1706-0011-41c0-ac33-be9f4861e7e1 None None] MFA Rules not processed for user `2dce40b8d74c41ec94cc36e6ff05d412`. Rule list: `[]` (Enabled: `True`). {{(pid=80244) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 08:42:48.361551 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.fernet_utils [None req-d10f1706-0011-41c0-ac33-be9f4861e7e1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80244) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 08:42:48.385194 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG dbcounter [-] [80244] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80244) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:42:48.422565 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 1418/2844] 10.222.0.47 () {58 vars in 957 bytes} [Thu Jul 21 08:42:48 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 08:42:48.432149 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a51ad70-825e-4215-8c0b-0c9ed43c0e9f admin admin] Authenticating user token {{(pid=80245) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:42:48.433533 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.fernet_utils [None req-0a51ad70-825e-4215-8c0b-0c9ed43c0e9f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80245) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 08:42:48.491877 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a51ad70-825e-4215-8c0b-0c9ed43c0e9f admin admin] Validating token access rules against request {{(pid=80245) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:42:48.495681 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df7b07f4-adf6-4346-a311-8b945399a2a0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20ca3c3bd3774691b3374cfbfec6e494', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4faef13aa50e4e36b6c265fb1398fa14', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80245) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:42:48.496336 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df7b07f4-adf6-4346-a311-8b945399a2a0 service neutron] REQUEST_METHOD: `GET` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:42:48.496621 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df7b07f4-adf6-4346-a311-8b945399a2a0 service neutron] SCRIPT_NAME: `/identity` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:42:48.496902 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df7b07f4-adf6-4346-a311-8b945399a2a0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:42:48.506257 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df7b07f4-adf6-4346-a311-8b945399a2a0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:42:48.507721 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df7b07f4-adf6-4346-a311-8b945399a2a0 service neutron] RBAC: Authorization granted {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:42:48.517619 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: [pid: 80245|app: 0|req: 1427/2845] 10.222.0.47 () {60 vars in 1365 bytes} [Thu Jul 21 08:42:48 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 08:42:48.744231 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d10f1706-0011-41c0-ac33-be9f4861e7e1 None None] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:42:48.745768 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.fernet_utils [None req-d10f1706-0011-41c0-ac33-be9f4861e7e1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80244) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 08:42:48.804821 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d10f1706-0011-41c0-ac33-be9f4861e7e1 None None] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:42:48.809107 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df7b07f4-adf6-4346-a311-8b945399a2a0 service neutron] Authenticating user token {{(pid=80245) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:42:48.810902 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.fernet_utils [None req-df7b07f4-adf6-4346-a311-8b945399a2a0 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80245) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 08:42:48.811736 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9190641-2232-4942-a05c-649f6906b58e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4f40410b7674687ab5362ac982248ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4faef13aa50e4e36b6c265fb1398fa14', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:42:48.812454 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9190641-2232-4942-a05c-649f6906b58e service cinder] REQUEST_METHOD: `GET` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:42:48.812739 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9190641-2232-4942-a05c-649f6906b58e service cinder] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:42:48.812960 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9190641-2232-4942-a05c-649f6906b58e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:42:48.814923 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.fernet_utils [None req-e9190641-2232-4942-a05c-649f6906b58e service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80244) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 08:42:48.824976 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df7b07f4-adf6-4346-a311-8b945399a2a0 service neutron] Validating token access rules against request {{(pid=80245) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:42:48.831451 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92cdd981-0b38-4a21-bbfa-3eeb0f34f629 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '42d15fc0515945acacbbffcf2356c7a5', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80245) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:42:48.832331 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92cdd981-0b38-4a21-bbfa-3eeb0f34f629 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:42:48.832617 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92cdd981-0b38-4a21-bbfa-3eeb0f34f629 admin admin] SCRIPT_NAME: `/identity` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:42:48.832898 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92cdd981-0b38-4a21-bbfa-3eeb0f34f629 admin admin] PATH_INFO: `/v3/projects/0993312e2717405eb3fa20e28a802c3b` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:42:48.834094 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92cdd981-0b38-4a21-bbfa-3eeb0f34f629 admin admin] RBAC: Authorizing `identity:delete_project(project_id=0993312e2717405eb3fa20e28a802c3b)` {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:42:48.836059 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92cdd981-0b38-4a21-bbfa-3eeb0f34f629 admin admin] RBAC: Authorization granted {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:42:48.884441 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG dbcounter [-] [80245] Writing DB stats keystone:SELECT=86,keystone:DELETE=13,keystone:INSERT=1 {{(pid=80245) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:42:48.887183 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.notifications [None req-92cdd981-0b38-4a21-bbfa-3eeb0f34f629 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0993312e2717405eb3fa20e28a802c3b', 'request_id': 'req-92cdd981-0b38-4a21-bbfa-3eeb0f34f629'} {{(pid=80245) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 08:42:48.890091 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9190641-2232-4942-a05c-649f6906b58e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:42:48.891847 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9190641-2232-4942-a05c-649f6906b58e service cinder] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:42:48.895036 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.notifications [None req-92cdd981-0b38-4a21-bbfa-3eeb0f34f629 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0993312e2717405eb3fa20e28a802c3b', 'request_id': 'req-92cdd981-0b38-4a21-bbfa-3eeb0f34f629'} {{(pid=80245) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 08:42:48.900857 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.notifications [None req-92cdd981-0b38-4a21-bbfa-3eeb0f34f629 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0993312e2717405eb3fa20e28a802c3b', 'request_id': 'req-92cdd981-0b38-4a21-bbfa-3eeb0f34f629'} {{(pid=80245) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 08:42:48.902520 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 1419/2846] 10.222.0.47 () {60 vars in 1365 bytes} [Thu Jul 21 08:42:48 2022] GET /identity/v3/auth/tokens => generated 3305 bytes in 159 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 08:42:48.913805 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.notifications [None req-92cdd981-0b38-4a21-bbfa-3eeb0f34f629 admin admin] The token cache is being invalidate because project 0993312e2717405eb3fa20e28a802c3b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80245) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 21 08:42:48.914094 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.notifications [None req-92cdd981-0b38-4a21-bbfa-3eeb0f34f629 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80245) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 08:42:48.914728 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.notifications [None req-92cdd981-0b38-4a21-bbfa-3eeb0f34f629 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80245) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 08:42:48.916476 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: [pid: 80245|app: 0|req: 1428/2847] 10.222.0.47 () {58 vars in 1232 bytes} [Thu Jul 21 08:42:48 2022] DELETE /identity/v3/projects/0993312e2717405eb3fa20e28a802c3b => generated 0 bytes in 108 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 21 08:42:48.922208 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f98d19e2-e676-4dc7-8201-3943176ae98f None None] REQUEST_METHOD: `POST` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:42:48.922546 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f98d19e2-e676-4dc7-8201-3943176ae98f None None] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:42:48.922776 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f98d19e2-e676-4dc7-8201-3943176ae98f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:42:48.941585 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.auth.core [None req-f98d19e2-e676-4dc7-8201-3943176ae98f None None] MFA Rules not processed for user `2dce40b8d74c41ec94cc36e6ff05d412`. Rule list: `[]` (Enabled: `True`). {{(pid=80244) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 08:42:48.942794 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.fernet_utils [None req-f98d19e2-e676-4dc7-8201-3943176ae98f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80244) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 08:42:49.008238 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 1420/2848] 10.222.0.47 () {58 vars in 957 bytes} [Thu Jul 21 08:42:48 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 08:42:49.016944 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29da6620-16d6-42cd-ba89-ed34ec872933 None None] REQUEST_METHOD: `POST` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:42:49.017240 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29da6620-16d6-42cd-ba89-ed34ec872933 None None] SCRIPT_NAME: `/identity` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:42:49.017520 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29da6620-16d6-42cd-ba89-ed34ec872933 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:42:49.036699 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.auth.core [None req-29da6620-16d6-42cd-ba89-ed34ec872933 None None] MFA Rules not processed for user `2dce40b8d74c41ec94cc36e6ff05d412`. Rule list: `[]` (Enabled: `True`). {{(pid=80245) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 08:42:49.037906 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.fernet_utils [None req-29da6620-16d6-42cd-ba89-ed34ec872933 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80245) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 08:42:49.046786 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: [pid: 80245|app: 0|req: 1429/2849] 10.222.0.47 () {58 vars in 957 bytes} [Thu Jul 21 08:42:49 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 08:42:49.050857 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f98d19e2-e676-4dc7-8201-3943176ae98f None None] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:42:49.062703 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f98d19e2-e676-4dc7-8201-3943176ae98f None None] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:42:49.069637 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0bf9ad2-9d70-4d51-bb68-17aad5e13a03 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '42d15fc0515945acacbbffcf2356c7a5', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:42:49.070453 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0bf9ad2-9d70-4d51-bb68-17aad5e13a03 admin admin] REQUEST_METHOD: `GET` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:42:49.070766 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0bf9ad2-9d70-4d51-bb68-17aad5e13a03 admin admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:42:49.071104 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0bf9ad2-9d70-4d51-bb68-17aad5e13a03 admin admin] PATH_INFO: `/v3/domains` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:42:49.071931 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0bf9ad2-9d70-4d51-bb68-17aad5e13a03 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80244) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 08:42:49.072235 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0bf9ad2-9d70-4d51-bb68-17aad5e13a03 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:42:49.073875 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0bf9ad2-9d70-4d51-bb68-17aad5e13a03 admin admin] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:42:49.090336 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 1421/2850] 10.222.0.47 () {58 vars in 1152 bytes} [Thu Jul 21 08:42:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 08:42:49.092032 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29da6620-16d6-42cd-ba89-ed34ec872933 None None] Authenticating user token {{(pid=80245) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:42:49.101959 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29da6620-16d6-42cd-ba89-ed34ec872933 None None] Validating token access rules against request {{(pid=80245) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:42:49.109406 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59b22603-a03a-4aed-94a0-522bcb567fe3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '42d15fc0515945acacbbffcf2356c7a5', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80245) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:42:49.110400 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59b22603-a03a-4aed-94a0-522bcb567fe3 admin admin] REQUEST_METHOD: `POST` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:42:49.110863 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59b22603-a03a-4aed-94a0-522bcb567fe3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:42:49.111290 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59b22603-a03a-4aed-94a0-522bcb567fe3 admin admin] PATH_INFO: `/v3/projects` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:42:49.112463 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59b22603-a03a-4aed-94a0-522bcb567fe3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:42:49.115754 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59b22603-a03a-4aed-94a0-522bcb567fe3 admin admin] RBAC: Authorization granted {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:42:49.148883 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: [pid: 80245|app: 0|req: 1430/2851] 10.222.0.47 () {60 vars in 1152 bytes} [Thu Jul 21 08:42:49 2022] POST /identity/v3/projects => generated 377 bytes in 58 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 08:42:49.153245 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0bf9ad2-9d70-4d51-bb68-17aad5e13a03 admin admin] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:42:49.164041 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0bf9ad2-9d70-4d51-bb68-17aad5e13a03 admin admin] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:42:49.168049 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG dbcounter [-] [80244] Writing DB stats keystone:SELECT=100 {{(pid=80244) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:42:49.223635 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7cf0a3c-d0cb-45c6-943f-a25b450f564c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '42d15fc0515945acacbbffcf2356c7a5', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:42:49.224239 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7cf0a3c-d0cb-45c6-943f-a25b450f564c admin admin] REQUEST_METHOD: `POST` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:42:49.224526 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7cf0a3c-d0cb-45c6-943f-a25b450f564c admin admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:42:49.224805 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7cf0a3c-d0cb-45c6-943f-a25b450f564c admin admin] PATH_INFO: `/v3/users` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:42:49.225518 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7cf0a3c-d0cb-45c6-943f-a25b450f564c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:42:49.227532 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7cf0a3c-d0cb-45c6-943f-a25b450f564c admin admin] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:42:49.247963 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 1422/2852] 10.222.0.47 () {60 vars in 1143 bytes} [Thu Jul 21 08:42:49 2022] POST /identity/v3/users => generated 289 bytes in 95 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 08:42:49.252545 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59b22603-a03a-4aed-94a0-522bcb567fe3 admin admin] Authenticating user token {{(pid=80245) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:42:49.264553 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59b22603-a03a-4aed-94a0-522bcb567fe3 admin admin] Validating token access rules against request {{(pid=80245) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:42:49.268850 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33baab46-c410-4704-9fd3-374ba923e96f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '42d15fc0515945acacbbffcf2356c7a5', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80245) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:42:49.269549 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33baab46-c410-4704-9fd3-374ba923e96f admin admin] REQUEST_METHOD: `GET` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:42:49.269835 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33baab46-c410-4704-9fd3-374ba923e96f admin admin] SCRIPT_NAME: `/identity` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:42:49.270088 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33baab46-c410-4704-9fd3-374ba923e96f admin admin] PATH_INFO: `/v3/roles` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:42:49.270814 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33baab46-c410-4704-9fd3-374ba923e96f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:42:49.272566 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33baab46-c410-4704-9fd3-374ba923e96f admin admin] RBAC: Authorization granted {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:42:49.283928 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: [pid: 80245|app: 0|req: 1431/2853] 10.222.0.47 () {58 vars in 1121 bytes} [Thu Jul 21 08:42:49 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 08:42:49.287983 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7cf0a3c-d0cb-45c6-943f-a25b450f564c admin admin] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:42:49.298145 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7cf0a3c-d0cb-45c6-943f-a25b450f564c admin admin] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:42:49.303485 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a36f7350-9320-4530-bd71-69d02288db35 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '42d15fc0515945acacbbffcf2356c7a5', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:42:49.304153 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a36f7350-9320-4530-bd71-69d02288db35 admin admin] REQUEST_METHOD: `GET` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:42:49.304441 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a36f7350-9320-4530-bd71-69d02288db35 admin admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:42:49.304720 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a36f7350-9320-4530-bd71-69d02288db35 admin admin] PATH_INFO: `/v3/roles` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:42:49.305445 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a36f7350-9320-4530-bd71-69d02288db35 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:42:49.307056 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a36f7350-9320-4530-bd71-69d02288db35 admin admin] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:42:49.317040 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 1423/2854] 10.222.0.47 () {58 vars in 1121 bytes} [Thu Jul 21 08:42:49 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 08:42:49.321176 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33baab46-c410-4704-9fd3-374ba923e96f admin admin] Authenticating user token {{(pid=80245) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:42:49.331542 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33baab46-c410-4704-9fd3-374ba923e96f admin admin] Validating token access rules against request {{(pid=80245) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:42:49.336077 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a3d075b-1e6b-4e9e-86eb-b57e20992adf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '42d15fc0515945acacbbffcf2356c7a5', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80245) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:42:49.336949 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a3d075b-1e6b-4e9e-86eb-b57e20992adf admin admin] REQUEST_METHOD: `PUT` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:42:49.337226 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a3d075b-1e6b-4e9e-86eb-b57e20992adf admin admin] SCRIPT_NAME: `/identity` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:42:49.337502 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a3d075b-1e6b-4e9e-86eb-b57e20992adf admin admin] PATH_INFO: `/v3/projects/7858221a62ef4997979bd2c376629140/users/855719ced58a4b55b4b8d31affd62a7b/roles/3a074738221e4601b5205715ba87d240` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:42:49.353435 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a3d075b-1e6b-4e9e-86eb-b57e20992adf admin admin] RBAC: Authorizing `identity:create_grant(project_id=7858221a62ef4997979bd2c376629140, user_id=855719ced58a4b55b4b8d31affd62a7b, role_id=3a074738221e4601b5205715ba87d240)` {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:42:49.356067 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a3d075b-1e6b-4e9e-86eb-b57e20992adf admin admin] RBAC: Authorization granted {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:42:49.368793 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: [pid: 80245|app: 0|req: 1432/2855] 10.222.0.47 () {60 vars in 1482 bytes} [Thu Jul 21 08:42:49 2022] PUT /identity/v3/projects/7858221a62ef4997979bd2c376629140/users/855719ced58a4b55b4b8d31affd62a7b/roles/3a074738221e4601b5205715ba87d240 => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 08:42:49.373752 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3275889-99f5-475d-a3d0-054fecabf494 None None] REQUEST_METHOD: `POST` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:42:49.374245 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3275889-99f5-475d-a3d0-054fecabf494 None None] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:42:49.374347 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3275889-99f5-475d-a3d0-054fecabf494 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:42:49.392771 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.auth.core [None req-d3275889-99f5-475d-a3d0-054fecabf494 None None] MFA Rules not processed for user `2dce40b8d74c41ec94cc36e6ff05d412`. Rule list: `[]` (Enabled: `True`). {{(pid=80244) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 08:42:49.394002 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.fernet_utils [None req-d3275889-99f5-475d-a3d0-054fecabf494 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80244) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 08:42:49.464123 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 1424/2856] 10.222.0.47 () {58 vars in 957 bytes} [Thu Jul 21 08:42:49 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 08:42:49.475158 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a3d075b-1e6b-4e9e-86eb-b57e20992adf admin admin] Authenticating user token {{(pid=80245) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:42:49.477067 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.fernet_utils [None req-3a3d075b-1e6b-4e9e-86eb-b57e20992adf admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80245) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 08:42:49.535904 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a3d075b-1e6b-4e9e-86eb-b57e20992adf admin admin] Validating token access rules against request {{(pid=80245) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:42:49.539624 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf49a4f4-7498-4fc5-9375-ebba87db29f5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20ca3c3bd3774691b3374cfbfec6e494', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4faef13aa50e4e36b6c265fb1398fa14', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80245) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:42:49.540255 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf49a4f4-7498-4fc5-9375-ebba87db29f5 service neutron] REQUEST_METHOD: `GET` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:42:49.540539 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf49a4f4-7498-4fc5-9375-ebba87db29f5 service neutron] SCRIPT_NAME: `/identity` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:42:49.540818 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf49a4f4-7498-4fc5-9375-ebba87db29f5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:42:49.549911 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf49a4f4-7498-4fc5-9375-ebba87db29f5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:42:49.551314 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf49a4f4-7498-4fc5-9375-ebba87db29f5 service neutron] RBAC: Authorization granted {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:42:49.560772 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: [pid: 80245|app: 0|req: 1433/2857] 10.222.0.47 () {60 vars in 1365 bytes} [Thu Jul 21 08:42:49 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 08:42:53.881717 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48d36c36-093d-4608-9e10-8f0a9d9b6ab5 None None] REQUEST_METHOD: `POST` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:42:53.882298 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48d36c36-093d-4608-9e10-8f0a9d9b6ab5 None None] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:42:53.882298 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48d36c36-093d-4608-9e10-8f0a9d9b6ab5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:42:53.912583 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.auth.core [None req-48d36c36-093d-4608-9e10-8f0a9d9b6ab5 None None] MFA Rules not processed for user `855719ced58a4b55b4b8d31affd62a7b`. Rule list: `[]` (Enabled: `True`). {{(pid=80244) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 08:42:53.913800 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.fernet_utils [None req-48d36c36-093d-4608-9e10-8f0a9d9b6ab5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80244) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 08:42:53.994669 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 1425/2858] 10.222.0.47 () {58 vars in 957 bytes} [Thu Jul 21 08:42:53 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 08:42:54.000895 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf49a4f4-7498-4fc5-9375-ebba87db29f5 service neutron] Authenticating user token {{(pid=80245) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:42:54.012193 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf49a4f4-7498-4fc5-9375-ebba87db29f5 service neutron] Validating token access rules against request {{(pid=80245) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:42:54.017373 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d349f7f2-06b1-421b-a8ab-e5f202947e41 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '42d15fc0515945acacbbffcf2356c7a5', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80245) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:42:54.018055 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d349f7f2-06b1-421b-a8ab-e5f202947e41 admin admin] REQUEST_METHOD: `POST` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:42:54.018375 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d349f7f2-06b1-421b-a8ab-e5f202947e41 admin admin] SCRIPT_NAME: `/identity` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:42:54.018757 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d349f7f2-06b1-421b-a8ab-e5f202947e41 admin admin] PATH_INFO: `/v3/projects` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:42:54.019352 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d349f7f2-06b1-421b-a8ab-e5f202947e41 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:42:54.021365 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d349f7f2-06b1-421b-a8ab-e5f202947e41 admin admin] RBAC: Authorization granted {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:42:54.053278 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: [pid: 80245|app: 0|req: 1434/2859] 10.222.0.47 () {60 vars in 1152 bytes} [Thu Jul 21 08:42:53 2022] POST /identity/v3/projects => generated 379 bytes in 54 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 08:42:54.057488 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48d36c36-093d-4608-9e10-8f0a9d9b6ab5 None None] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:42:54.069791 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48d36c36-093d-4608-9e10-8f0a9d9b6ab5 None None] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:42:54.083478 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG dbcounter [-] [80244] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80244) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:42:54.132853 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d617115e-266c-4cc6-85b7-c81c5deb22a2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '42d15fc0515945acacbbffcf2356c7a5', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:42:54.133442 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d617115e-266c-4cc6-85b7-c81c5deb22a2 admin admin] REQUEST_METHOD: `POST` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:42:54.133726 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d617115e-266c-4cc6-85b7-c81c5deb22a2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:42:54.134001 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d617115e-266c-4cc6-85b7-c81c5deb22a2 admin admin] PATH_INFO: `/v3/users` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:42:54.134698 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d617115e-266c-4cc6-85b7-c81c5deb22a2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:42:54.136682 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d617115e-266c-4cc6-85b7-c81c5deb22a2 admin admin] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:42:54.157388 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 1426/2860] 10.222.0.47 () {60 vars in 1143 bytes} [Thu Jul 21 08:42:54 2022] POST /identity/v3/users => generated 296 bytes in 100 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 08:42:54.161684 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d349f7f2-06b1-421b-a8ab-e5f202947e41 admin admin] Authenticating user token {{(pid=80245) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:42:54.173033 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d349f7f2-06b1-421b-a8ab-e5f202947e41 admin admin] Validating token access rules against request {{(pid=80245) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:42:54.177801 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56f1ba3c-1f4d-4799-9ebc-bee48693cb78 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '42d15fc0515945acacbbffcf2356c7a5', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80245) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:42:54.178466 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56f1ba3c-1f4d-4799-9ebc-bee48693cb78 admin admin] REQUEST_METHOD: `GET` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:42:54.178750 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56f1ba3c-1f4d-4799-9ebc-bee48693cb78 admin admin] SCRIPT_NAME: `/identity` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:42:54.179029 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56f1ba3c-1f4d-4799-9ebc-bee48693cb78 admin admin] PATH_INFO: `/v3/roles` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:42:54.179796 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56f1ba3c-1f4d-4799-9ebc-bee48693cb78 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:42:54.181389 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56f1ba3c-1f4d-4799-9ebc-bee48693cb78 admin admin] RBAC: Authorization granted {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:42:54.192090 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: [pid: 80245|app: 0|req: 1435/2861] 10.222.0.47 () {58 vars in 1121 bytes} [Thu Jul 21 08:42:54 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 08:42:54.196257 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d617115e-266c-4cc6-85b7-c81c5deb22a2 admin admin] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:42:54.206176 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d617115e-266c-4cc6-85b7-c81c5deb22a2 admin admin] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:42:54.210750 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cec7ba32-a40e-4a1d-9d9a-0f12cb4907e6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '42d15fc0515945acacbbffcf2356c7a5', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:42:54.211673 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cec7ba32-a40e-4a1d-9d9a-0f12cb4907e6 admin admin] REQUEST_METHOD: `PUT` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:42:54.211954 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cec7ba32-a40e-4a1d-9d9a-0f12cb4907e6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:42:54.212234 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cec7ba32-a40e-4a1d-9d9a-0f12cb4907e6 admin admin] PATH_INFO: `/v3/projects/351502a4a7fa4c978e2e3d1ddd192f04/users/9d86bc543db44944973fca1e7c3f4556/roles/e29870f6e9e2411a9db0d5bf45f32ca5` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:42:54.227440 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cec7ba32-a40e-4a1d-9d9a-0f12cb4907e6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=351502a4a7fa4c978e2e3d1ddd192f04, user_id=9d86bc543db44944973fca1e7c3f4556, role_id=e29870f6e9e2411a9db0d5bf45f32ca5)` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:42:54.230123 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cec7ba32-a40e-4a1d-9d9a-0f12cb4907e6 admin admin] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:42:54.243822 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 1427/2862] 10.222.0.47 () {60 vars in 1482 bytes} [Thu Jul 21 08:42:54 2022] PUT /identity/v3/projects/351502a4a7fa4c978e2e3d1ddd192f04/users/9d86bc543db44944973fca1e7c3f4556/roles/e29870f6e9e2411a9db0d5bf45f32ca5 => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 08:42:54.247888 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56f1ba3c-1f4d-4799-9ebc-bee48693cb78 admin admin] Authenticating user token {{(pid=80245) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:42:54.257990 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56f1ba3c-1f4d-4799-9ebc-bee48693cb78 admin admin] Validating token access rules against request {{(pid=80245) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:42:54.297779 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG dbcounter [-] [80245] Writing DB stats keystone:DELETE=4,keystone:SELECT=92,keystone:INSERT=4 {{(pid=80245) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:42:54.319710 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abe64afd-7f84-408c-8e38-a6d7d96dca03 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '42d15fc0515945acacbbffcf2356c7a5', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80245) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:42:54.320397 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abe64afd-7f84-408c-8e38-a6d7d96dca03 admin admin] REQUEST_METHOD: `GET` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:42:54.320685 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abe64afd-7f84-408c-8e38-a6d7d96dca03 admin admin] SCRIPT_NAME: `/identity` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:42:54.320960 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abe64afd-7f84-408c-8e38-a6d7d96dca03 admin admin] PATH_INFO: `/v3/roles` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:42:54.321690 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abe64afd-7f84-408c-8e38-a6d7d96dca03 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:42:54.323305 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abe64afd-7f84-408c-8e38-a6d7d96dca03 admin admin] RBAC: Authorization granted {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:42:54.333305 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: [pid: 80245|app: 0|req: 1436/2863] 10.222.0.47 () {58 vars in 1121 bytes} [Thu Jul 21 08:42:54 2022] GET /identity/v3/roles => generated 1420 bytes in 86 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 08:42:54.337672 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cec7ba32-a40e-4a1d-9d9a-0f12cb4907e6 admin admin] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:42:54.348919 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cec7ba32-a40e-4a1d-9d9a-0f12cb4907e6 admin admin] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:42:54.354118 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9b4d4da-5b46-431d-a611-567fcacd7af4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '42d15fc0515945acacbbffcf2356c7a5', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:42:54.355006 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9b4d4da-5b46-431d-a611-567fcacd7af4 admin admin] REQUEST_METHOD: `PUT` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:42:54.355437 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9b4d4da-5b46-431d-a611-567fcacd7af4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:42:54.355746 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9b4d4da-5b46-431d-a611-567fcacd7af4 admin admin] PATH_INFO: `/v3/projects/351502a4a7fa4c978e2e3d1ddd192f04/users/9d86bc543db44944973fca1e7c3f4556/roles/e29870f6e9e2411a9db0d5bf45f32ca5` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:42:54.358433 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9b4d4da-5b46-431d-a611-567fcacd7af4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=351502a4a7fa4c978e2e3d1ddd192f04, user_id=9d86bc543db44944973fca1e7c3f4556, role_id=e29870f6e9e2411a9db0d5bf45f32ca5)` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:42:54.361237 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9b4d4da-5b46-431d-a611-567fcacd7af4 admin admin] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:42:54.372266 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 1428/2864] 10.222.0.47 () {60 vars in 1482 bytes} [Thu Jul 21 08:42:54 2022] PUT /identity/v3/projects/351502a4a7fa4c978e2e3d1ddd192f04/users/9d86bc543db44944973fca1e7c3f4556/roles/e29870f6e9e2411a9db0d5bf45f32ca5 => generated 0 bytes in 35 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 08:42:59.133970 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5b1b557-8a62-4d26-8481-4c12b44f49f8 None None] REQUEST_METHOD: `POST` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:42:59.134667 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5b1b557-8a62-4d26-8481-4c12b44f49f8 None None] SCRIPT_NAME: `/identity` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:42:59.134845 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5b1b557-8a62-4d26-8481-4c12b44f49f8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:42:59.169827 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.auth.core [None req-f5b1b557-8a62-4d26-8481-4c12b44f49f8 None None] MFA Rules not processed for user `9d86bc543db44944973fca1e7c3f4556`. Rule list: `[]` (Enabled: `True`). {{(pid=80245) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 08:42:59.171235 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.fernet_utils [None req-f5b1b557-8a62-4d26-8481-4c12b44f49f8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80245) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 08:42:59.276501 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: [pid: 80245|app: 0|req: 1437/2865] 10.222.0.47 () {58 vars in 957 bytes} [Thu Jul 21 08:42:59 2022] POST /identity/v3/auth/tokens => generated 3313 bytes in 145 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 08:42:59.298169 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9b4d4da-5b46-431d-a611-567fcacd7af4 admin admin] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:42:59.301951 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.fernet_utils [None req-d9b4d4da-5b46-431d-a611-567fcacd7af4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80244) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 08:42:59.389408 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9b4d4da-5b46-431d-a611-567fcacd7af4 admin admin] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:42:59.395957 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4bd439b-72af-48b9-9aaf-784db09ffddf service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2b5b99a36442148fe743269054e991', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4faef13aa50e4e36b6c265fb1398fa14', 'project_domain_id': 'default', 'roles': ['member', 'service', 'ResellerAdmin', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:42:59.396667 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4bd439b-72af-48b9-9aaf-784db09ffddf service nova] REQUEST_METHOD: `GET` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:42:59.396956 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4bd439b-72af-48b9-9aaf-784db09ffddf service nova] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:42:59.397230 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4bd439b-72af-48b9-9aaf-784db09ffddf service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:42:59.460728 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4bd439b-72af-48b9-9aaf-784db09ffddf service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:42:59.465138 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4bd439b-72af-48b9-9aaf-784db09ffddf service nova] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:42:59.481821 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 1429/2866] 10.222.0.47 () {60 vars in 1365 bytes} [Thu Jul 21 08:42:59 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 184 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 21 08:43:02.969694 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5b1b557-8a62-4d26-8481-4c12b44f49f8 None None] Authenticating user token {{(pid=80245) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:43:02.971899 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.fernet_utils [None req-f5b1b557-8a62-4d26-8481-4c12b44f49f8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80245) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 08:43:03.045910 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5b1b557-8a62-4d26-8481-4c12b44f49f8 None None] Validating token access rules against request {{(pid=80245) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:43:03.051521 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7b49317-bb83-4e0a-aa8e-e92410419a8c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '42d15fc0515945acacbbffcf2356c7a5', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80245) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:43:03.052767 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7b49317-bb83-4e0a-aa8e-e92410419a8c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:43:03.053069 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7b49317-bb83-4e0a-aa8e-e92410419a8c admin admin] SCRIPT_NAME: `/identity` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:43:03.053540 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7b49317-bb83-4e0a-aa8e-e92410419a8c admin admin] PATH_INFO: `/v3/users/8325d08910cb422487f3d2f01410e49f` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:43:03.054947 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7b49317-bb83-4e0a-aa8e-e92410419a8c admin admin] RBAC: Authorizing `identity:delete_user(user_id=8325d08910cb422487f3d2f01410e49f)` {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:43:03.057891 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7b49317-bb83-4e0a-aa8e-e92410419a8c admin admin] RBAC: Authorization granted {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:43:03.112908 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.notifications [None req-b7b49317-bb83-4e0a-aa8e-e92410419a8c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8325d08910cb422487f3d2f01410e49f'} {{(pid=80245) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 08:43:03.120779 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.notifications [None req-b7b49317-bb83-4e0a-aa8e-e92410419a8c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8325d08910cb422487f3d2f01410e49f'} {{(pid=80245) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 08:43:03.129380 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG dbcounter [-] [80245] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=80245) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:43:03.141644 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.notifications [None req-b7b49317-bb83-4e0a-aa8e-e92410419a8c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8325d08910cb422487f3d2f01410e49f'} {{(pid=80245) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 08:43:03.150296 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.notifications [None req-b7b49317-bb83-4e0a-aa8e-e92410419a8c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8325d08910cb422487f3d2f01410e49f'} {{(pid=80245) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 08:43:03.151063 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.notifications [None req-b7b49317-bb83-4e0a-aa8e-e92410419a8c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8325d08910cb422487f3d2f01410e49f'} {{(pid=80245) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 08:43:03.157753 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: [pid: 80245|app: 0|req: 1438/2867] 10.222.0.47 () {58 vars in 1223 bytes} [Thu Jul 21 08:43:02 2022] DELETE /identity/v3/users/8325d08910cb422487f3d2f01410e49f => generated 0 bytes in 189 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 08:43:03.362720 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4bd439b-72af-48b9-9aaf-784db09ffddf service nova] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:43:03.371660 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG dbcounter [-] [80244] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80244) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:43:03.379354 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4bd439b-72af-48b9-9aaf-784db09ffddf service nova] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:43:03.445291 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01d05b00-0bc6-44a3-8b9a-7e72dedd1541 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '42d15fc0515945acacbbffcf2356c7a5', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:43:03.446064 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01d05b00-0bc6-44a3-8b9a-7e72dedd1541 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:43:03.446343 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01d05b00-0bc6-44a3-8b9a-7e72dedd1541 admin admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:43:03.446620 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01d05b00-0bc6-44a3-8b9a-7e72dedd1541 admin admin] PATH_INFO: `/v3/projects/4813fdf8c25f4b988cf6a2b26d5b9c71` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:43:03.447885 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01d05b00-0bc6-44a3-8b9a-7e72dedd1541 admin admin] RBAC: Authorizing `identity:delete_project(project_id=4813fdf8c25f4b988cf6a2b26d5b9c71)` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:43:03.450123 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01d05b00-0bc6-44a3-8b9a-7e72dedd1541 admin admin] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:43:03.511681 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.notifications [None req-01d05b00-0bc6-44a3-8b9a-7e72dedd1541 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4813fdf8c25f4b988cf6a2b26d5b9c71', 'request_id': 'req-01d05b00-0bc6-44a3-8b9a-7e72dedd1541'} {{(pid=80244) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 08:43:03.524489 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.notifications [None req-01d05b00-0bc6-44a3-8b9a-7e72dedd1541 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4813fdf8c25f4b988cf6a2b26d5b9c71', 'request_id': 'req-01d05b00-0bc6-44a3-8b9a-7e72dedd1541'} {{(pid=80244) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 08:43:03.535951 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.notifications [None req-01d05b00-0bc6-44a3-8b9a-7e72dedd1541 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4813fdf8c25f4b988cf6a2b26d5b9c71', 'request_id': 'req-01d05b00-0bc6-44a3-8b9a-7e72dedd1541'} {{(pid=80244) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 08:43:03.547971 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.notifications [None req-01d05b00-0bc6-44a3-8b9a-7e72dedd1541 admin admin] The token cache is being invalidate because project 4813fdf8c25f4b988cf6a2b26d5b9c71 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80244) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 21 08:43:03.548260 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.notifications [None req-01d05b00-0bc6-44a3-8b9a-7e72dedd1541 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80244) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 08:43:03.548922 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.notifications [None req-01d05b00-0bc6-44a3-8b9a-7e72dedd1541 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80244) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 08:43:03.550669 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 1430/2868] 10.222.0.47 () {58 vars in 1232 bytes} [Thu Jul 21 08:43:03 2022] DELETE /identity/v3/projects/4813fdf8c25f4b988cf6a2b26d5b9c71 => generated 0 bytes in 188 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 21 08:43:03.554555 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7b49317-bb83-4e0a-aa8e-e92410419a8c admin admin] Authenticating user token {{(pid=80245) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:43:03.555981 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.fernet_utils [None req-b7b49317-bb83-4e0a-aa8e-e92410419a8c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80245) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 08:43:03.569053 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7b49317-bb83-4e0a-aa8e-e92410419a8c admin admin] Validating token access rules against request {{(pid=80245) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:43:03.630203 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-361c7b2d-7bdf-4ad0-a3e1-98c0f5e7a411 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '42d15fc0515945acacbbffcf2356c7a5', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80245) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:43:03.631132 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-361c7b2d-7bdf-4ad0-a3e1-98c0f5e7a411 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:43:03.631456 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-361c7b2d-7bdf-4ad0-a3e1-98c0f5e7a411 admin admin] SCRIPT_NAME: `/identity` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:43:03.631730 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-361c7b2d-7bdf-4ad0-a3e1-98c0f5e7a411 admin admin] PATH_INFO: `/v3/users/37fac0d86a9047ac8bedfae0556d82b0` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:43:03.632947 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-361c7b2d-7bdf-4ad0-a3e1-98c0f5e7a411 admin admin] RBAC: Authorizing `identity:delete_user(user_id=37fac0d86a9047ac8bedfae0556d82b0)` {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:43:03.634875 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-361c7b2d-7bdf-4ad0-a3e1-98c0f5e7a411 admin admin] RBAC: Authorization granted {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:43:03.686151 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.notifications [None req-361c7b2d-7bdf-4ad0-a3e1-98c0f5e7a411 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '37fac0d86a9047ac8bedfae0556d82b0'} {{(pid=80245) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 08:43:03.693509 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.notifications [None req-361c7b2d-7bdf-4ad0-a3e1-98c0f5e7a411 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '37fac0d86a9047ac8bedfae0556d82b0'} {{(pid=80245) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 08:43:03.706139 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.notifications [None req-361c7b2d-7bdf-4ad0-a3e1-98c0f5e7a411 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '37fac0d86a9047ac8bedfae0556d82b0'} {{(pid=80245) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 08:43:03.712827 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.notifications [None req-361c7b2d-7bdf-4ad0-a3e1-98c0f5e7a411 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '37fac0d86a9047ac8bedfae0556d82b0'} {{(pid=80245) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 08:43:03.713502 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.notifications [None req-361c7b2d-7bdf-4ad0-a3e1-98c0f5e7a411 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '37fac0d86a9047ac8bedfae0556d82b0'} {{(pid=80245) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 08:43:03.720051 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: [pid: 80245|app: 0|req: 1439/2869] 10.222.0.47 () {58 vars in 1223 bytes} [Thu Jul 21 08:43:03 2022] DELETE /identity/v3/users/37fac0d86a9047ac8bedfae0556d82b0 => generated 0 bytes in 166 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 21 08:43:03.896707 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01d05b00-0bc6-44a3-8b9a-7e72dedd1541 admin admin] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:43:03.898316 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.fernet_utils [None req-01d05b00-0bc6-44a3-8b9a-7e72dedd1541 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80244) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 08:43:03.911386 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01d05b00-0bc6-44a3-8b9a-7e72dedd1541 admin admin] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:43:03.971229 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77ea1434-e8d4-4602-8c1c-b334f1b722f0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '42d15fc0515945acacbbffcf2356c7a5', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:43:03.972068 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77ea1434-e8d4-4602-8c1c-b334f1b722f0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:43:03.972352 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77ea1434-e8d4-4602-8c1c-b334f1b722f0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:43:03.972657 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77ea1434-e8d4-4602-8c1c-b334f1b722f0 admin admin] PATH_INFO: `/v3/projects/d57e7e762f164e48975b4a7fae9bc114` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:43:03.973848 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77ea1434-e8d4-4602-8c1c-b334f1b722f0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d57e7e762f164e48975b4a7fae9bc114)` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:43:03.976498 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77ea1434-e8d4-4602-8c1c-b334f1b722f0 admin admin] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:43:04.024292 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG dbcounter [-] [80244] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80244) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:43:04.041075 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.notifications [None req-77ea1434-e8d4-4602-8c1c-b334f1b722f0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd57e7e762f164e48975b4a7fae9bc114', 'request_id': 'req-77ea1434-e8d4-4602-8c1c-b334f1b722f0'} {{(pid=80244) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 08:43:04.049625 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.notifications [None req-77ea1434-e8d4-4602-8c1c-b334f1b722f0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd57e7e762f164e48975b4a7fae9bc114', 'request_id': 'req-77ea1434-e8d4-4602-8c1c-b334f1b722f0'} {{(pid=80244) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 08:43:04.056114 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.notifications [None req-77ea1434-e8d4-4602-8c1c-b334f1b722f0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd57e7e762f164e48975b4a7fae9bc114', 'request_id': 'req-77ea1434-e8d4-4602-8c1c-b334f1b722f0'} {{(pid=80244) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 08:43:04.067150 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.notifications [None req-77ea1434-e8d4-4602-8c1c-b334f1b722f0 admin admin] The token cache is being invalidate because project d57e7e762f164e48975b4a7fae9bc114 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80244) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 21 08:43:04.067468 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.notifications [None req-77ea1434-e8d4-4602-8c1c-b334f1b722f0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80244) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 08:43:04.068089 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.notifications [None req-77ea1434-e8d4-4602-8c1c-b334f1b722f0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80244) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 08:43:04.069613 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 1431/2870] 10.222.0.47 () {58 vars in 1232 bytes} [Thu Jul 21 08:43:03 2022] DELETE /identity/v3/projects/d57e7e762f164e48975b4a7fae9bc114 => generated 0 bytes in 173 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 08:43:04.074707 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1073d174-a7c5-4bbb-8531-d2b5a0438c27 None None] REQUEST_METHOD: `POST` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:43:04.075813 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1073d174-a7c5-4bbb-8531-d2b5a0438c27 None None] SCRIPT_NAME: `/identity` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:43:04.075927 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1073d174-a7c5-4bbb-8531-d2b5a0438c27 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:43:04.094326 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.auth.core [None req-1073d174-a7c5-4bbb-8531-d2b5a0438c27 None None] MFA Rules not processed for user `2dce40b8d74c41ec94cc36e6ff05d412`. Rule list: `[]` (Enabled: `True`). {{(pid=80245) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 08:43:04.095607 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.fernet_utils [None req-1073d174-a7c5-4bbb-8531-d2b5a0438c27 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80245) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 08:43:04.146697 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG dbcounter [-] [80245] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=80245) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:43:04.156456 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: [pid: 80245|app: 0|req: 1440/2871] 10.222.0.47 () {58 vars in 957 bytes} [Thu Jul 21 08:43:04 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 08:43:04.165204 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b0b0b61-5877-44a7-ad8a-af968563e9b0 None None] REQUEST_METHOD: `POST` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:43:04.165494 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b0b0b61-5877-44a7-ad8a-af968563e9b0 None None] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:43:04.165774 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b0b0b61-5877-44a7-ad8a-af968563e9b0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:43:04.184375 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.auth.core [None req-8b0b0b61-5877-44a7-ad8a-af968563e9b0 None None] MFA Rules not processed for user `2dce40b8d74c41ec94cc36e6ff05d412`. Rule list: `[]` (Enabled: `True`). {{(pid=80244) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 08:43:04.186091 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.fernet_utils [None req-8b0b0b61-5877-44a7-ad8a-af968563e9b0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80244) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 08:43:04.193998 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 1432/2872] 10.222.0.47 () {58 vars in 957 bytes} [Thu Jul 21 08:43:04 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 08:43:04.198409 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1073d174-a7c5-4bbb-8531-d2b5a0438c27 None None] Authenticating user token {{(pid=80245) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:43:04.208993 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1073d174-a7c5-4bbb-8531-d2b5a0438c27 None None] Validating token access rules against request {{(pid=80245) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:43:04.214425 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c46b484-01f1-4578-9fe6-ab27df6348e0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '42d15fc0515945acacbbffcf2356c7a5', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80245) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:43:04.215113 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c46b484-01f1-4578-9fe6-ab27df6348e0 admin admin] REQUEST_METHOD: `GET` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:43:04.215430 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c46b484-01f1-4578-9fe6-ab27df6348e0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:43:04.215740 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c46b484-01f1-4578-9fe6-ab27df6348e0 admin admin] PATH_INFO: `/v3/domains` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:43:04.216467 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c46b484-01f1-4578-9fe6-ab27df6348e0 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80245) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 08:43:04.216778 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c46b484-01f1-4578-9fe6-ab27df6348e0 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:43:04.220643 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c46b484-01f1-4578-9fe6-ab27df6348e0 admin admin] RBAC: Authorization granted {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:43:04.243460 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: [pid: 80245|app: 0|req: 1441/2873] 10.222.0.47 () {58 vars in 1152 bytes} [Thu Jul 21 08:43:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 08:43:04.244896 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b0b0b61-5877-44a7-ad8a-af968563e9b0 None None] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:43:04.255410 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b0b0b61-5877-44a7-ad8a-af968563e9b0 None None] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:43:04.260108 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f546c534-654f-4323-bdc2-a97fdb6d7297 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '42d15fc0515945acacbbffcf2356c7a5', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:43:04.260766 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f546c534-654f-4323-bdc2-a97fdb6d7297 admin admin] REQUEST_METHOD: `POST` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:43:04.261049 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f546c534-654f-4323-bdc2-a97fdb6d7297 admin admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:43:04.261358 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f546c534-654f-4323-bdc2-a97fdb6d7297 admin admin] PATH_INFO: `/v3/projects` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:43:04.262062 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f546c534-654f-4323-bdc2-a97fdb6d7297 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:43:04.264152 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f546c534-654f-4323-bdc2-a97fdb6d7297 admin admin] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:43:04.291475 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 1433/2874] 10.222.0.47 () {60 vars in 1152 bytes} [Thu Jul 21 08:43:04 2022] POST /identity/v3/projects => generated 373 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 08:43:04.296212 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c46b484-01f1-4578-9fe6-ab27df6348e0 admin admin] Authenticating user token {{(pid=80245) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:43:04.309316 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c46b484-01f1-4578-9fe6-ab27df6348e0 admin admin] Validating token access rules against request {{(pid=80245) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:43:04.377370 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8a0e88f-1283-4f76-afd6-44dd6c156385 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '42d15fc0515945acacbbffcf2356c7a5', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80245) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:43:04.378196 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8a0e88f-1283-4f76-afd6-44dd6c156385 admin admin] REQUEST_METHOD: `POST` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:43:04.378591 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8a0e88f-1283-4f76-afd6-44dd6c156385 admin admin] SCRIPT_NAME: `/identity` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:43:04.379006 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8a0e88f-1283-4f76-afd6-44dd6c156385 admin admin] PATH_INFO: `/v3/users` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:43:04.379824 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8a0e88f-1283-4f76-afd6-44dd6c156385 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:43:04.381790 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8a0e88f-1283-4f76-afd6-44dd6c156385 admin admin] RBAC: Authorization granted {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:43:04.409816 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: [pid: 80245|app: 0|req: 1442/2875] 10.222.0.47 () {60 vars in 1143 bytes} [Thu Jul 21 08:43:04 2022] POST /identity/v3/users => generated 287 bytes in 114 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 21 08:43:04.413912 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f546c534-654f-4323-bdc2-a97fdb6d7297 admin admin] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:43:04.428513 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f546c534-654f-4323-bdc2-a97fdb6d7297 admin admin] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:43:04.433778 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9003fed1-b6bc-47e0-b977-60d76c7eecf0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '42d15fc0515945acacbbffcf2356c7a5', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:43:04.434455 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9003fed1-b6bc-47e0-b977-60d76c7eecf0 admin admin] REQUEST_METHOD: `GET` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:43:04.435395 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9003fed1-b6bc-47e0-b977-60d76c7eecf0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:43:04.435670 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9003fed1-b6bc-47e0-b977-60d76c7eecf0 admin admin] PATH_INFO: `/v3/roles` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:43:04.436402 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9003fed1-b6bc-47e0-b977-60d76c7eecf0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:43:04.438003 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9003fed1-b6bc-47e0-b977-60d76c7eecf0 admin admin] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:43:04.449603 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 1434/2876] 10.222.0.47 () {58 vars in 1121 bytes} [Thu Jul 21 08:43:04 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 08:43:04.453715 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8a0e88f-1283-4f76-afd6-44dd6c156385 admin admin] Authenticating user token {{(pid=80245) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:43:04.464145 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8a0e88f-1283-4f76-afd6-44dd6c156385 admin admin] Validating token access rules against request {{(pid=80245) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:43:04.469483 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1e6dc20-dea3-4258-b660-ad6ab3f7831b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '42d15fc0515945acacbbffcf2356c7a5', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80245) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:43:04.470291 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1e6dc20-dea3-4258-b660-ad6ab3f7831b admin admin] REQUEST_METHOD: `GET` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:43:04.475221 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1e6dc20-dea3-4258-b660-ad6ab3f7831b admin admin] SCRIPT_NAME: `/identity` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:43:04.475221 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1e6dc20-dea3-4258-b660-ad6ab3f7831b admin admin] PATH_INFO: `/v3/roles` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:43:04.475221 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1e6dc20-dea3-4258-b660-ad6ab3f7831b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:43:04.475221 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1e6dc20-dea3-4258-b660-ad6ab3f7831b admin admin] RBAC: Authorization granted {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:43:04.487072 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: [pid: 80245|app: 0|req: 1443/2877] 10.222.0.47 () {58 vars in 1121 bytes} [Thu Jul 21 08:43:04 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 08:43:04.491276 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9003fed1-b6bc-47e0-b977-60d76c7eecf0 admin admin] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:43:04.503302 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9003fed1-b6bc-47e0-b977-60d76c7eecf0 admin admin] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:43:04.508269 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d5d8a96-3ab7-404c-9989-b5f1249b7209 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '42d15fc0515945acacbbffcf2356c7a5', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:43:04.509169 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d5d8a96-3ab7-404c-9989-b5f1249b7209 admin admin] REQUEST_METHOD: `PUT` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:43:04.509453 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d5d8a96-3ab7-404c-9989-b5f1249b7209 admin admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:43:04.509728 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d5d8a96-3ab7-404c-9989-b5f1249b7209 admin admin] PATH_INFO: `/v3/projects/099ef6c722224dc8ac631682903803f7/users/9b8cd5349ae147e1928214c0ba6d8e5c/roles/3a074738221e4601b5205715ba87d240` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:43:04.532606 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d5d8a96-3ab7-404c-9989-b5f1249b7209 admin admin] RBAC: Authorizing `identity:create_grant(project_id=099ef6c722224dc8ac631682903803f7, user_id=9b8cd5349ae147e1928214c0ba6d8e5c, role_id=3a074738221e4601b5205715ba87d240)` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:43:04.535276 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d5d8a96-3ab7-404c-9989-b5f1249b7209 admin admin] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:43:04.552191 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 1435/2878] 10.222.0.47 () {60 vars in 1482 bytes} [Thu Jul 21 08:43:04 2022] PUT /identity/v3/projects/099ef6c722224dc8ac631682903803f7/users/9b8cd5349ae147e1928214c0ba6d8e5c/roles/3a074738221e4601b5205715ba87d240 => generated 0 bytes in 61 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 08:43:04.622190 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20c5a621-0188-4bbf-b9c0-2e9ccb39d382 None None] REQUEST_METHOD: `POST` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:43:04.622726 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20c5a621-0188-4bbf-b9c0-2e9ccb39d382 None None] SCRIPT_NAME: `/identity` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:43:04.623101 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20c5a621-0188-4bbf-b9c0-2e9ccb39d382 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:43:04.663589 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.auth.core [None req-20c5a621-0188-4bbf-b9c0-2e9ccb39d382 None None] MFA Rules not processed for user `9b8cd5349ae147e1928214c0ba6d8e5c`. Rule list: `[]` (Enabled: `True`). {{(pid=80245) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 08:43:04.664850 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.fernet_utils [None req-20c5a621-0188-4bbf-b9c0-2e9ccb39d382 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80245) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 08:43:04.743674 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: [pid: 80245|app: 0|req: 1444/2879] 10.222.0.47 () {58 vars in 957 bytes} [Thu Jul 21 08:43:04 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 21 08:43:04.758006 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d5d8a96-3ab7-404c-9989-b5f1249b7209 admin admin] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:43:04.759773 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.fernet_utils [None req-3d5d8a96-3ab7-404c-9989-b5f1249b7209 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80244) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 08:43:04.818429 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d5d8a96-3ab7-404c-9989-b5f1249b7209 admin admin] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:43:04.821998 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5226849-40c8-47ec-ae95-36a0814c9a97 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4f40410b7674687ab5362ac982248ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4faef13aa50e4e36b6c265fb1398fa14', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:43:04.822642 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5226849-40c8-47ec-ae95-36a0814c9a97 service cinder] REQUEST_METHOD: `GET` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:43:04.822925 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5226849-40c8-47ec-ae95-36a0814c9a97 service cinder] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:43:04.823203 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5226849-40c8-47ec-ae95-36a0814c9a97 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:43:04.831695 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5226849-40c8-47ec-ae95-36a0814c9a97 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:43:04.833154 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5226849-40c8-47ec-ae95-36a0814c9a97 service cinder] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:43:04.842504 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 1436/2880] 10.222.0.47 () {60 vars in 1365 bytes} [Thu Jul 21 08:43:04 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 08:43:04.862815 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20c5a621-0188-4bbf-b9c0-2e9ccb39d382 None None] Authenticating user token {{(pid=80245) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:43:04.921842 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG dbcounter [-] [80245] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80245) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:43:04.928664 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20c5a621-0188-4bbf-b9c0-2e9ccb39d382 None None] Validating token access rules against request {{(pid=80245) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:43:04.933491 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bdcab17-04b5-465e-993d-e21ce1fc4c04 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '42d15fc0515945acacbbffcf2356c7a5', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80245) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:43:04.934367 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bdcab17-04b5-465e-993d-e21ce1fc4c04 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:43:04.934680 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bdcab17-04b5-465e-993d-e21ce1fc4c04 admin admin] SCRIPT_NAME: `/identity` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:43:04.934959 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bdcab17-04b5-465e-993d-e21ce1fc4c04 admin admin] PATH_INFO: `/v3/users/9b8cd5349ae147e1928214c0ba6d8e5c` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:43:04.936195 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bdcab17-04b5-465e-993d-e21ce1fc4c04 admin admin] RBAC: Authorizing `identity:delete_user(user_id=9b8cd5349ae147e1928214c0ba6d8e5c)` {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:43:04.938102 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bdcab17-04b5-465e-993d-e21ce1fc4c04 admin admin] RBAC: Authorization granted {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:43:04.986788 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.notifications [None req-8bdcab17-04b5-465e-993d-e21ce1fc4c04 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9b8cd5349ae147e1928214c0ba6d8e5c'} {{(pid=80245) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 08:43:04.994481 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.notifications [None req-8bdcab17-04b5-465e-993d-e21ce1fc4c04 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9b8cd5349ae147e1928214c0ba6d8e5c'} {{(pid=80245) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 08:43:05.006761 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.notifications [None req-8bdcab17-04b5-465e-993d-e21ce1fc4c04 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9b8cd5349ae147e1928214c0ba6d8e5c'} {{(pid=80245) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 08:43:05.014221 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.notifications [None req-8bdcab17-04b5-465e-993d-e21ce1fc4c04 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9b8cd5349ae147e1928214c0ba6d8e5c'} {{(pid=80245) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 08:43:05.014896 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.notifications [None req-8bdcab17-04b5-465e-993d-e21ce1fc4c04 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9b8cd5349ae147e1928214c0ba6d8e5c'} {{(pid=80245) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 08:43:05.021312 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: [pid: 80245|app: 0|req: 1445/2881] 10.222.0.47 () {58 vars in 1223 bytes} [Thu Jul 21 08:43:04 2022] DELETE /identity/v3/users/9b8cd5349ae147e1928214c0ba6d8e5c => generated 0 bytes in 159 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 08:43:05.026124 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6fbaa47-83da-4412-bd90-58645beafeaa None None] REQUEST_METHOD: `POST` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:43:05.026426 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6fbaa47-83da-4412-bd90-58645beafeaa None None] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:43:05.026710 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6fbaa47-83da-4412-bd90-58645beafeaa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:43:05.047600 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.auth.core [None req-f6fbaa47-83da-4412-bd90-58645beafeaa None None] MFA Rules not processed for user `2dce40b8d74c41ec94cc36e6ff05d412`. Rule list: `[]` (Enabled: `True`). {{(pid=80244) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 08:43:05.048852 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.fernet_utils [None req-f6fbaa47-83da-4412-bd90-58645beafeaa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80244) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 08:43:05.112689 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 1437/2882] 10.222.0.47 () {58 vars in 957 bytes} [Thu Jul 21 08:43:05 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 08:43:05.122268 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bdcab17-04b5-465e-993d-e21ce1fc4c04 admin admin] Authenticating user token {{(pid=80245) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:43:05.123729 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.fernet_utils [None req-8bdcab17-04b5-465e-993d-e21ce1fc4c04 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80245) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 08:43:05.182130 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bdcab17-04b5-465e-993d-e21ce1fc4c04 admin admin] Validating token access rules against request {{(pid=80245) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:43:05.185789 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05d1acc3-cf0a-4ac8-92a6-5890c170d4b2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20ca3c3bd3774691b3374cfbfec6e494', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4faef13aa50e4e36b6c265fb1398fa14', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80245) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:43:05.186446 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05d1acc3-cf0a-4ac8-92a6-5890c170d4b2 service neutron] REQUEST_METHOD: `GET` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:43:05.186851 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05d1acc3-cf0a-4ac8-92a6-5890c170d4b2 service neutron] SCRIPT_NAME: `/identity` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:43:05.187135 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05d1acc3-cf0a-4ac8-92a6-5890c170d4b2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:43:05.196457 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05d1acc3-cf0a-4ac8-92a6-5890c170d4b2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:43:05.197861 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05d1acc3-cf0a-4ac8-92a6-5890c170d4b2 service neutron] RBAC: Authorization granted {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:43:05.209818 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: [pid: 80245|app: 0|req: 1446/2883] 10.222.0.47 () {60 vars in 1365 bytes} [Thu Jul 21 08:43:05 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 08:43:05.518752 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6fbaa47-83da-4412-bd90-58645beafeaa None None] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:43:05.520250 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.fernet_utils [None req-f6fbaa47-83da-4412-bd90-58645beafeaa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80244) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 08:43:05.529854 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG dbcounter [-] [80244] Writing DB stats keystone:SELECT=93,keystone:DELETE=4,keystone:INSERT=3 {{(pid=80244) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:43:05.534546 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6fbaa47-83da-4412-bd90-58645beafeaa None None] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:43:05.544767 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-770e2e37-4c07-48b4-a684-c8f852fad894 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '42d15fc0515945acacbbffcf2356c7a5', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:43:05.546282 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-770e2e37-4c07-48b4-a684-c8f852fad894 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:43:05.546742 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-770e2e37-4c07-48b4-a684-c8f852fad894 admin admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:43:05.547156 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-770e2e37-4c07-48b4-a684-c8f852fad894 admin admin] PATH_INFO: `/v3/projects/099ef6c722224dc8ac631682903803f7` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:43:05.548651 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-770e2e37-4c07-48b4-a684-c8f852fad894 admin admin] RBAC: Authorizing `identity:delete_project(project_id=099ef6c722224dc8ac631682903803f7)` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:43:05.550599 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-770e2e37-4c07-48b4-a684-c8f852fad894 admin admin] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:43:05.603233 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.notifications [None req-770e2e37-4c07-48b4-a684-c8f852fad894 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '099ef6c722224dc8ac631682903803f7', 'request_id': 'req-770e2e37-4c07-48b4-a684-c8f852fad894'} {{(pid=80244) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 08:43:05.611114 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.notifications [None req-770e2e37-4c07-48b4-a684-c8f852fad894 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '099ef6c722224dc8ac631682903803f7', 'request_id': 'req-770e2e37-4c07-48b4-a684-c8f852fad894'} {{(pid=80244) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 08:43:05.617888 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.notifications [None req-770e2e37-4c07-48b4-a684-c8f852fad894 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '099ef6c722224dc8ac631682903803f7', 'request_id': 'req-770e2e37-4c07-48b4-a684-c8f852fad894'} {{(pid=80244) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 08:43:05.629029 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.notifications [None req-770e2e37-4c07-48b4-a684-c8f852fad894 admin admin] The token cache is being invalidate because project 099ef6c722224dc8ac631682903803f7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80244) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 21 08:43:05.629310 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.notifications [None req-770e2e37-4c07-48b4-a684-c8f852fad894 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80244) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 08:43:05.629968 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.notifications [None req-770e2e37-4c07-48b4-a684-c8f852fad894 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80244) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 08:43:05.631561 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 1438/2884] 10.222.0.47 () {58 vars in 1232 bytes} [Thu Jul 21 08:43:05 2022] DELETE /identity/v3/projects/099ef6c722224dc8ac631682903803f7 => generated 0 bytes in 113 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 08:43:15.203660 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG dbcounter [-] [80245] Writing DB stats keystone:SELECT=54,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80245) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:43:15.625756 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG dbcounter [-] [80244] Writing DB stats keystone:SELECT=25,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80244) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:43:15.883352 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a284f1a1-cc2a-407a-9ba8-7851e0d18fb6 None None] REQUEST_METHOD: `POST` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:43:15.883654 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a284f1a1-cc2a-407a-9ba8-7851e0d18fb6 None None] SCRIPT_NAME: `/identity` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:43:15.883937 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a284f1a1-cc2a-407a-9ba8-7851e0d18fb6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:43:15.902398 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.auth.core [None req-a284f1a1-cc2a-407a-9ba8-7851e0d18fb6 None None] MFA Rules not processed for user `855719ced58a4b55b4b8d31affd62a7b`. Rule list: `[]` (Enabled: `True`). {{(pid=80245) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 08:43:15.903676 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.fernet_utils [None req-a284f1a1-cc2a-407a-9ba8-7851e0d18fb6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80245) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 08:43:15.959895 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: [pid: 80245|app: 0|req: 1447/2885] 10.222.0.47 () {58 vars in 957 bytes} [Thu Jul 21 08:43:15 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 78 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 08:43:16.272956 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-770e2e37-4c07-48b4-a684-c8f852fad894 admin admin] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:43:16.274364 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.fernet_utils [None req-770e2e37-4c07-48b4-a684-c8f852fad894 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80244) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 08:43:16.332493 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-770e2e37-4c07-48b4-a684-c8f852fad894 admin admin] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:43:16.336114 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-932659a4-784b-4eb5-beb5-4d788038e863 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20ca3c3bd3774691b3374cfbfec6e494', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4faef13aa50e4e36b6c265fb1398fa14', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:43:16.336762 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-932659a4-784b-4eb5-beb5-4d788038e863 service neutron] REQUEST_METHOD: `GET` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:43:16.337046 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-932659a4-784b-4eb5-beb5-4d788038e863 service neutron] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:43:16.337322 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-932659a4-784b-4eb5-beb5-4d788038e863 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:43:16.339107 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.fernet_utils [None req-932659a4-784b-4eb5-beb5-4d788038e863 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80244) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 08:43:16.349112 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-932659a4-784b-4eb5-beb5-4d788038e863 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:43:16.350531 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-932659a4-784b-4eb5-beb5-4d788038e863 service neutron] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:43:16.358741 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 1439/2886] 10.222.0.47 () {60 vars in 1365 bytes} [Thu Jul 21 08:43:16 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 08:43:25.947846 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a284f1a1-cc2a-407a-9ba8-7851e0d18fb6 None None] Authenticating user token {{(pid=80245) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:43:25.949253 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.fernet_utils [None req-a284f1a1-cc2a-407a-9ba8-7851e0d18fb6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80245) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 08:43:25.952167 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG dbcounter [-] [80245] Writing DB stats keystone:SELECT=25 {{(pid=80245) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:43:26.019874 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a284f1a1-cc2a-407a-9ba8-7851e0d18fb6 None None] Validating token access rules against request {{(pid=80245) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:43:26.025236 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eee5fc0e-8149-48ef-92f5-dfe7bdb76771 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3379b32543474d0a8e897484e85c7d04', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4faef13aa50e4e36b6c265fb1398fa14', 'project_domain_id': 'default', 'roles': ['service', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80245) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:43:26.025882 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eee5fc0e-8149-48ef-92f5-dfe7bdb76771 service placement] REQUEST_METHOD: `GET` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:43:26.026169 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eee5fc0e-8149-48ef-92f5-dfe7bdb76771 service placement] SCRIPT_NAME: `/identity` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:43:26.026465 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eee5fc0e-8149-48ef-92f5-dfe7bdb76771 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:43:26.028832 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.fernet_utils [None req-eee5fc0e-8149-48ef-92f5-dfe7bdb76771 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80245) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 08:43:26.040441 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eee5fc0e-8149-48ef-92f5-dfe7bdb76771 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:43:26.042039 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eee5fc0e-8149-48ef-92f5-dfe7bdb76771 service placement] RBAC: Authorization granted {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:43:26.052070 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: [pid: 80245|app: 0|req: 1448/2887] 10.222.0.47 () {60 vars in 1365 bytes} [Thu Jul 21 08:43:25 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 105 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 08:43:26.354912 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG dbcounter [-] [80244] Writing DB stats keystone:SELECT=26 {{(pid=80244) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:43:28.963464 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-932659a4-784b-4eb5-beb5-4d788038e863 service neutron] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:43:28.965468 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.fernet_utils [None req-932659a4-784b-4eb5-beb5-4d788038e863 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80244) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 08:43:29.041703 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-932659a4-784b-4eb5-beb5-4d788038e863 service neutron] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:43:29.047272 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-212df775-92c9-4615-b345-9f25983990ff service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2b5b99a36442148fe743269054e991', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4faef13aa50e4e36b6c265fb1398fa14', 'project_domain_id': 'default', 'roles': ['member', 'service', 'ResellerAdmin', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:43:29.047948 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-212df775-92c9-4615-b345-9f25983990ff service nova] REQUEST_METHOD: `GET` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:43:29.048231 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-212df775-92c9-4615-b345-9f25983990ff service nova] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:43:29.048508 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-212df775-92c9-4615-b345-9f25983990ff service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:43:29.050402 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.fernet_utils [None req-212df775-92c9-4615-b345-9f25983990ff service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80244) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 08:43:29.062640 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-212df775-92c9-4615-b345-9f25983990ff service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:43:29.064383 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-212df775-92c9-4615-b345-9f25983990ff service nova] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:43:29.075616 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 1440/2888] 10.222.0.47 () {60 vars in 1365 bytes} [Thu Jul 21 08:43:28 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 113 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 08:43:31.132011 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eee5fc0e-8149-48ef-92f5-dfe7bdb76771 service placement] Authenticating user token {{(pid=80245) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:43:31.133426 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.fernet_utils [None req-eee5fc0e-8149-48ef-92f5-dfe7bdb76771 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80245) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 08:43:31.145206 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eee5fc0e-8149-48ef-92f5-dfe7bdb76771 service placement] Validating token access rules against request {{(pid=80245) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:43:31.148793 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2a6eeb5-3244-494f-8c12-f296d3652edb service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20ca3c3bd3774691b3374cfbfec6e494', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4faef13aa50e4e36b6c265fb1398fa14', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80245) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:43:31.149435 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2a6eeb5-3244-494f-8c12-f296d3652edb service neutron] REQUEST_METHOD: `GET` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:43:31.149721 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2a6eeb5-3244-494f-8c12-f296d3652edb service neutron] SCRIPT_NAME: `/identity` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:43:31.150000 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2a6eeb5-3244-494f-8c12-f296d3652edb service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:43:31.151755 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.fernet_utils [None req-a2a6eeb5-3244-494f-8c12-f296d3652edb service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80245) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 08:43:31.215963 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2a6eeb5-3244-494f-8c12-f296d3652edb service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:43:31.217391 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2a6eeb5-3244-494f-8c12-f296d3652edb service neutron] RBAC: Authorization granted {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:43:31.227084 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: [pid: 80245|app: 0|req: 1449/2889] 10.222.0.47 () {60 vars in 1365 bytes} [Thu Jul 21 08:43:31 2022] GET /identity/v3/auth/tokens => generated 3313 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 21 08:43:39.070476 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG dbcounter [-] [80244] Writing DB stats keystone:SELECT=34 {{(pid=80244) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:43:41.222710 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG dbcounter [-] [80245] Writing DB stats keystone:SELECT=62 {{(pid=80245) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:45:25.194595 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-212df775-92c9-4615-b345-9f25983990ff service nova] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:45:25.204631 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-212df775-92c9-4615-b345-9f25983990ff service nova] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:45:25.208522 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04ce6fb2-e73d-426c-8f74-35a6f800c346 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20ca3c3bd3774691b3374cfbfec6e494', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4faef13aa50e4e36b6c265fb1398fa14', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:45:25.209181 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04ce6fb2-e73d-426c-8f74-35a6f800c346 service neutron] REQUEST_METHOD: `GET` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:45:25.209470 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04ce6fb2-e73d-426c-8f74-35a6f800c346 service neutron] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:45:25.209751 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04ce6fb2-e73d-426c-8f74-35a6f800c346 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:45:25.213610 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.fernet_utils [None req-04ce6fb2-e73d-426c-8f74-35a6f800c346 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80244) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 08:45:25.228512 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04ce6fb2-e73d-426c-8f74-35a6f800c346 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:45:25.229949 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04ce6fb2-e73d-426c-8f74-35a6f800c346 service neutron] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:45:25.241249 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 1441/2890] 10.222.0.47 () {60 vars in 1365 bytes} [Thu Jul 21 08:45:25 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 47 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 08:45:26.421831 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2a6eeb5-3244-494f-8c12-f296d3652edb service neutron] Authenticating user token {{(pid=80245) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:45:26.430574 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2a6eeb5-3244-494f-8c12-f296d3652edb service neutron] Validating token access rules against request {{(pid=80245) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:45:26.434200 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f90d1f98-5376-4b55-9b8f-113f770c67a2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20ca3c3bd3774691b3374cfbfec6e494', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4faef13aa50e4e36b6c265fb1398fa14', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80245) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:45:26.434845 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f90d1f98-5376-4b55-9b8f-113f770c67a2 service neutron] REQUEST_METHOD: `GET` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:45:26.435127 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f90d1f98-5376-4b55-9b8f-113f770c67a2 service neutron] SCRIPT_NAME: `/identity` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:45:26.435434 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f90d1f98-5376-4b55-9b8f-113f770c67a2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:45:26.437272 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.fernet_utils [None req-f90d1f98-5376-4b55-9b8f-113f770c67a2 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80245) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 08:45:26.450894 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f90d1f98-5376-4b55-9b8f-113f770c67a2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:45:26.452283 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f90d1f98-5376-4b55-9b8f-113f770c67a2 service neutron] RBAC: Authorization granted {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:45:26.463626 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: [pid: 80245|app: 0|req: 1450/2891] 10.222.0.47 () {60 vars in 1365 bytes} [Thu Jul 21 08:45:26 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 08:45:28.748805 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1111ffa5-0f6b-45a5-a261-5a8a7d92b3db None None] REQUEST_METHOD: `POST` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:45:28.748805 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1111ffa5-0f6b-45a5-a261-5a8a7d92b3db None None] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:45:28.749304 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1111ffa5-0f6b-45a5-a261-5a8a7d92b3db None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:45:28.767713 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.auth.core [None req-1111ffa5-0f6b-45a5-a261-5a8a7d92b3db None None] MFA Rules not processed for user `855719ced58a4b55b4b8d31affd62a7b`. Rule list: `[]` (Enabled: `True`). {{(pid=80244) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 08:45:28.768924 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.fernet_utils [None req-1111ffa5-0f6b-45a5-a261-5a8a7d92b3db None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80244) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 08:45:28.775899 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 1442/2892] 10.222.0.47 () {58 vars in 957 bytes} [Thu Jul 21 08:45:28 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 08:45:36.458308 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG dbcounter [-] [80245] Writing DB stats keystone:SELECT=6 {{(pid=80245) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:45:38.762387 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG dbcounter [-] [80244] Writing DB stats keystone:SELECT=9 {{(pid=80244) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:46:27.679817 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f90d1f98-5376-4b55-9b8f-113f770c67a2 service neutron] Authenticating user token {{(pid=80245) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:46:27.688616 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f90d1f98-5376-4b55-9b8f-113f770c67a2 service neutron] Validating token access rules against request {{(pid=80245) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:46:27.692251 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-634e1b08-e233-4981-b1a2-54113363e4b6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20ca3c3bd3774691b3374cfbfec6e494', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4faef13aa50e4e36b6c265fb1398fa14', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80245) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:46:27.692894 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-634e1b08-e233-4981-b1a2-54113363e4b6 service neutron] REQUEST_METHOD: `GET` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:46:27.693180 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-634e1b08-e233-4981-b1a2-54113363e4b6 service neutron] SCRIPT_NAME: `/identity` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:46:27.693457 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-634e1b08-e233-4981-b1a2-54113363e4b6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:46:27.695969 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.fernet_utils [None req-634e1b08-e233-4981-b1a2-54113363e4b6 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80245) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 08:46:27.705435 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-634e1b08-e233-4981-b1a2-54113363e4b6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:46:27.706882 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-634e1b08-e233-4981-b1a2-54113363e4b6 service neutron] RBAC: Authorization granted {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:46:27.715568 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: [pid: 80245|app: 0|req: 1451/2893] 10.222.0.47 () {60 vars in 1365 bytes} [Thu Jul 21 08:46:27 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 21 08:46:37.711946 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG dbcounter [-] [80245] Writing DB stats keystone:SELECT=6 {{(pid=80245) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:47:19.394259 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9204b57d-489b-4e28-baf5-2b00203290d9 None None] REQUEST_METHOD: `POST` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:47:19.394775 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9204b57d-489b-4e28-baf5-2b00203290d9 None None] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:47:19.394912 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9204b57d-489b-4e28-baf5-2b00203290d9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:47:19.412576 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.auth.core [None req-9204b57d-489b-4e28-baf5-2b00203290d9 None None] MFA Rules not processed for user `855719ced58a4b55b4b8d31affd62a7b`. Rule list: `[]` (Enabled: `True`). {{(pid=80244) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 08:47:19.413770 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.fernet_utils [None req-9204b57d-489b-4e28-baf5-2b00203290d9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80244) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 08:47:19.420629 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 1443/2894] 10.222.0.47 () {58 vars in 957 bytes} [Thu Jul 21 08:47:19 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 27 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 08:47:19.737224 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-634e1b08-e233-4981-b1a2-54113363e4b6 service neutron] Authenticating user token {{(pid=80245) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:47:19.745870 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-634e1b08-e233-4981-b1a2-54113363e4b6 service neutron] Validating token access rules against request {{(pid=80245) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:47:19.749441 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-842bacc6-ae13-41f9-8fbd-547a03422986 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20ca3c3bd3774691b3374cfbfec6e494', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4faef13aa50e4e36b6c265fb1398fa14', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80245) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:47:19.750112 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-842bacc6-ae13-41f9-8fbd-547a03422986 service neutron] REQUEST_METHOD: `GET` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:47:19.750394 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-842bacc6-ae13-41f9-8fbd-547a03422986 service neutron] SCRIPT_NAME: `/identity` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:47:19.750670 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-842bacc6-ae13-41f9-8fbd-547a03422986 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:47:19.752718 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.fernet_utils [None req-842bacc6-ae13-41f9-8fbd-547a03422986 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80245) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 08:47:19.762044 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-842bacc6-ae13-41f9-8fbd-547a03422986 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:47:19.763492 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-842bacc6-ae13-41f9-8fbd-547a03422986 service neutron] RBAC: Authorization granted {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:47:19.772171 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: [pid: 80245|app: 0|req: 1452/2895] 10.222.0.47 () {60 vars in 1365 bytes} [Thu Jul 21 08:47:19 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 35 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 08:47:20.232589 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9204b57d-489b-4e28-baf5-2b00203290d9 None None] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:47:20.234097 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.fernet_utils [None req-9204b57d-489b-4e28-baf5-2b00203290d9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80244) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 08:47:20.250877 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9204b57d-489b-4e28-baf5-2b00203290d9 None None] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:47:20.256982 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-919bd82d-7d6d-4a0e-a342-28b58b61ecb9 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3379b32543474d0a8e897484e85c7d04', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4faef13aa50e4e36b6c265fb1398fa14', 'project_domain_id': 'default', 'roles': ['service', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:47:20.257656 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-919bd82d-7d6d-4a0e-a342-28b58b61ecb9 service placement] REQUEST_METHOD: `GET` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:47:20.257938 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-919bd82d-7d6d-4a0e-a342-28b58b61ecb9 service placement] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:47:20.258208 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-919bd82d-7d6d-4a0e-a342-28b58b61ecb9 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:47:20.260084 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.fernet_utils [None req-919bd82d-7d6d-4a0e-a342-28b58b61ecb9 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80244) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 08:47:20.277930 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-919bd82d-7d6d-4a0e-a342-28b58b61ecb9 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:47:20.279405 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-919bd82d-7d6d-4a0e-a342-28b58b61ecb9 service placement] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:47:20.290227 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 1444/2896] 10.222.0.47 () {60 vars in 1365 bytes} [Thu Jul 21 08:47:20 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 58 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 08:47:22.561077 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-842bacc6-ae13-41f9-8fbd-547a03422986 service neutron] Authenticating user token {{(pid=80245) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:47:22.575700 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-842bacc6-ae13-41f9-8fbd-547a03422986 service neutron] Validating token access rules against request {{(pid=80245) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:47:22.582414 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e52017c-da9f-4f75-829c-d2e77fd5c12a service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac2b5b99a36442148fe743269054e991', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4faef13aa50e4e36b6c265fb1398fa14', 'project_domain_id': 'default', 'roles': ['member', 'service', 'ResellerAdmin', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80245) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:47:22.603537 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e52017c-da9f-4f75-829c-d2e77fd5c12a service nova] REQUEST_METHOD: `GET` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:47:22.603958 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e52017c-da9f-4f75-829c-d2e77fd5c12a service nova] SCRIPT_NAME: `/identity` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:47:22.604247 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e52017c-da9f-4f75-829c-d2e77fd5c12a service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:47:22.611504 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.fernet_utils [None req-6e52017c-da9f-4f75-829c-d2e77fd5c12a service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80245) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 08:47:22.631824 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e52017c-da9f-4f75-829c-d2e77fd5c12a service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:47:22.634331 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e52017c-da9f-4f75-829c-d2e77fd5c12a service nova] RBAC: Authorization granted {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:47:22.647494 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: [pid: 80245|app: 0|req: 1453/2897] 10.222.0.47 () {60 vars in 1365 bytes} [Thu Jul 21 08:47:22 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 08:47:30.285374 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG dbcounter [-] [80244] Writing DB stats keystone:SELECT=9 {{(pid=80244) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:47:32.641589 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG dbcounter [-] [80245] Writing DB stats keystone:SELECT=12 {{(pid=80245) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:48:32.011405 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-919bd82d-7d6d-4a0e-a342-28b58b61ecb9 service placement] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:48:32.077495 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-919bd82d-7d6d-4a0e-a342-28b58b61ecb9 service placement] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:48:32.082782 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09c27f36-cafa-4716-85d6-c4a2024214d4 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3379b32543474d0a8e897484e85c7d04', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4faef13aa50e4e36b6c265fb1398fa14', 'project_domain_id': 'default', 'roles': ['service', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:48:32.083487 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09c27f36-cafa-4716-85d6-c4a2024214d4 service placement] REQUEST_METHOD: `GET` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:48:32.083774 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09c27f36-cafa-4716-85d6-c4a2024214d4 service placement] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:48:32.084060 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09c27f36-cafa-4716-85d6-c4a2024214d4 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:48:32.094002 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09c27f36-cafa-4716-85d6-c4a2024214d4 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:48:32.095661 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09c27f36-cafa-4716-85d6-c4a2024214d4 service placement] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:48:32.105733 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 1445/2898] 10.222.0.47 () {60 vars in 1365 bytes} [Thu Jul 21 08:48:32 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 08:48:42.101384 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG dbcounter [-] [80244] Writing DB stats keystone:SELECT=27 {{(pid=80244) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:49:15.903110 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e52017c-da9f-4f75-829c-d2e77fd5c12a service nova] Authenticating user token {{(pid=80245) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:49:15.940871 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e52017c-da9f-4f75-829c-d2e77fd5c12a service nova] Validating token access rules against request {{(pid=80245) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:49:15.944503 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-900921fb-aeda-4b66-b08a-9fe47e7d52ac service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20ca3c3bd3774691b3374cfbfec6e494', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4faef13aa50e4e36b6c265fb1398fa14', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80245) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:49:15.945148 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-900921fb-aeda-4b66-b08a-9fe47e7d52ac service neutron] REQUEST_METHOD: `GET` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:49:15.945432 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-900921fb-aeda-4b66-b08a-9fe47e7d52ac service neutron] SCRIPT_NAME: `/identity` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:49:15.945703 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-900921fb-aeda-4b66-b08a-9fe47e7d52ac service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:49:15.948104 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.fernet_utils [None req-900921fb-aeda-4b66-b08a-9fe47e7d52ac service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80245) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 08:49:15.968505 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-900921fb-aeda-4b66-b08a-9fe47e7d52ac service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:49:15.969940 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-900921fb-aeda-4b66-b08a-9fe47e7d52ac service neutron] RBAC: Authorization granted {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:49:15.979284 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: [pid: 80245|app: 0|req: 1454/2899] 10.222.0.47 () {60 vars in 1365 bytes} [Thu Jul 21 08:49:15 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 77 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 08:49:25.974713 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG dbcounter [-] [80245] Writing DB stats keystone:SELECT=22 {{(pid=80245) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:49:29.459245 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09c27f36-cafa-4716-85d6-c4a2024214d4 service placement] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:49:29.468776 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09c27f36-cafa-4716-85d6-c4a2024214d4 service placement] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:49:29.472460 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4037b338-cdc4-4646-a57d-ec6b28789aeb service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20ca3c3bd3774691b3374cfbfec6e494', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4faef13aa50e4e36b6c265fb1398fa14', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:49:29.473103 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4037b338-cdc4-4646-a57d-ec6b28789aeb service neutron] REQUEST_METHOD: `GET` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:49:29.473389 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4037b338-cdc4-4646-a57d-ec6b28789aeb service neutron] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:49:29.473664 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4037b338-cdc4-4646-a57d-ec6b28789aeb service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:49:29.475513 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.fernet_utils [None req-4037b338-cdc4-4646-a57d-ec6b28789aeb service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80244) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 08:49:29.564342 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4037b338-cdc4-4646-a57d-ec6b28789aeb service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:49:29.565782 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4037b338-cdc4-4646-a57d-ec6b28789aeb service neutron] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:49:29.584451 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 1446/2900] 10.222.0.47 () {60 vars in 1365 bytes} [Thu Jul 21 08:49:29 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 126 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 08:49:32.698323 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-900921fb-aeda-4b66-b08a-9fe47e7d52ac service neutron] Authenticating user token {{(pid=80245) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:49:32.711733 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-900921fb-aeda-4b66-b08a-9fe47e7d52ac service neutron] Validating token access rules against request {{(pid=80245) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:49:32.717585 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da564a25-6372-4fdb-9657-14b4d2dff604 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3379b32543474d0a8e897484e85c7d04', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4faef13aa50e4e36b6c265fb1398fa14', 'project_domain_id': 'default', 'roles': ['service', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80245) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:49:32.718364 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da564a25-6372-4fdb-9657-14b4d2dff604 service placement] REQUEST_METHOD: `GET` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:49:32.718644 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da564a25-6372-4fdb-9657-14b4d2dff604 service placement] SCRIPT_NAME: `/identity` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:49:32.718918 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da564a25-6372-4fdb-9657-14b4d2dff604 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:49:32.720785 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.fernet_utils [None req-da564a25-6372-4fdb-9657-14b4d2dff604 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80245) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 08:49:32.755658 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da564a25-6372-4fdb-9657-14b4d2dff604 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:49:32.758149 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da564a25-6372-4fdb-9657-14b4d2dff604 service placement] RBAC: Authorization granted {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:49:32.772161 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: [pid: 80245|app: 0|req: 1455/2901] 10.222.0.47 () {60 vars in 1365 bytes} [Thu Jul 21 08:49:32 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 74 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 08:49:35.956794 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4037b338-cdc4-4646-a57d-ec6b28789aeb service neutron] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:49:35.971758 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4037b338-cdc4-4646-a57d-ec6b28789aeb service neutron] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:49:35.978918 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b7ae6c4-204e-4cb5-a1e2-bcac68fe716c service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3379b32543474d0a8e897484e85c7d04', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4faef13aa50e4e36b6c265fb1398fa14', 'project_domain_id': 'default', 'roles': ['service', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:49:35.980062 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b7ae6c4-204e-4cb5-a1e2-bcac68fe716c service placement] REQUEST_METHOD: `GET` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:49:35.980369 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b7ae6c4-204e-4cb5-a1e2-bcac68fe716c service placement] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:49:35.980644 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b7ae6c4-204e-4cb5-a1e2-bcac68fe716c service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:49:35.982475 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.fernet_utils [None req-4b7ae6c4-204e-4cb5-a1e2-bcac68fe716c service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80244) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 08:49:35.995410 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b7ae6c4-204e-4cb5-a1e2-bcac68fe716c service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:49:35.997245 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b7ae6c4-204e-4cb5-a1e2-bcac68fe716c service placement] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:49:36.015647 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 1447/2902] 10.222.0.47 () {60 vars in 1365 bytes} [Thu Jul 21 08:49:35 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 60 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 21 08:49:36.046092 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da564a25-6372-4fdb-9657-14b4d2dff604 service placement] Authenticating user token {{(pid=80245) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:49:36.047177 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.fernet_utils [None req-da564a25-6372-4fdb-9657-14b4d2dff604 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80245) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 08:49:36.064851 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da564a25-6372-4fdb-9657-14b4d2dff604 service placement] Validating token access rules against request {{(pid=80245) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:49:36.070185 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11041720-7346-43ad-8058-f9f7fd0c5ce5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '42d15fc0515945acacbbffcf2356c7a5', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80245) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:49:36.072060 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11041720-7346-43ad-8058-f9f7fd0c5ce5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:49:36.072342 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11041720-7346-43ad-8058-f9f7fd0c5ce5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:49:36.073603 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11041720-7346-43ad-8058-f9f7fd0c5ce5 admin admin] PATH_INFO: `/v3/users/855719ced58a4b55b4b8d31affd62a7b` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:49:36.075223 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11041720-7346-43ad-8058-f9f7fd0c5ce5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=855719ced58a4b55b4b8d31affd62a7b)` {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:49:36.077348 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11041720-7346-43ad-8058-f9f7fd0c5ce5 admin admin] RBAC: Authorization granted {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:49:36.128554 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.notifications [None req-11041720-7346-43ad-8058-f9f7fd0c5ce5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '855719ced58a4b55b4b8d31affd62a7b'} {{(pid=80245) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 08:49:36.135735 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.notifications [None req-11041720-7346-43ad-8058-f9f7fd0c5ce5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '855719ced58a4b55b4b8d31affd62a7b'} {{(pid=80245) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 08:49:36.147893 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.notifications [None req-11041720-7346-43ad-8058-f9f7fd0c5ce5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '855719ced58a4b55b4b8d31affd62a7b'} {{(pid=80245) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 08:49:36.155145 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.notifications [None req-11041720-7346-43ad-8058-f9f7fd0c5ce5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '855719ced58a4b55b4b8d31affd62a7b'} {{(pid=80245) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 08:49:36.155866 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.notifications [None req-11041720-7346-43ad-8058-f9f7fd0c5ce5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '855719ced58a4b55b4b8d31affd62a7b'} {{(pid=80245) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 08:49:36.161869 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: [pid: 80245|app: 0|req: 1456/2903] 10.222.0.47 () {58 vars in 1223 bytes} [Thu Jul 21 08:49:36 2022] DELETE /identity/v3/users/855719ced58a4b55b4b8d31affd62a7b => generated 0 bytes in 117 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 08:49:36.344788 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b7ae6c4-204e-4cb5-a1e2-bcac68fe716c service placement] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:49:36.356439 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b7ae6c4-204e-4cb5-a1e2-bcac68fe716c service placement] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:49:36.421043 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99e8ce77-3bcd-425f-814a-3f30a46f908c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '42d15fc0515945acacbbffcf2356c7a5', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:49:36.421823 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99e8ce77-3bcd-425f-814a-3f30a46f908c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:49:36.422103 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99e8ce77-3bcd-425f-814a-3f30a46f908c admin admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:49:36.422375 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99e8ce77-3bcd-425f-814a-3f30a46f908c admin admin] PATH_INFO: `/v3/projects/7858221a62ef4997979bd2c376629140` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:49:36.423578 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99e8ce77-3bcd-425f-814a-3f30a46f908c admin admin] RBAC: Authorizing `identity:delete_project(project_id=7858221a62ef4997979bd2c376629140)` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:49:36.425514 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99e8ce77-3bcd-425f-814a-3f30a46f908c admin admin] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:49:36.472496 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.notifications [None req-99e8ce77-3bcd-425f-814a-3f30a46f908c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7858221a62ef4997979bd2c376629140', 'request_id': 'req-99e8ce77-3bcd-425f-814a-3f30a46f908c'} {{(pid=80244) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 08:49:36.476654 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG dbcounter [-] [80244] Writing DB stats keystone:SELECT=94,keystone:DELETE=5,keystone:INSERT=1 {{(pid=80244) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:49:36.482057 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.notifications [None req-99e8ce77-3bcd-425f-814a-3f30a46f908c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7858221a62ef4997979bd2c376629140', 'request_id': 'req-99e8ce77-3bcd-425f-814a-3f30a46f908c'} {{(pid=80244) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 08:49:36.489957 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.notifications [None req-99e8ce77-3bcd-425f-814a-3f30a46f908c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7858221a62ef4997979bd2c376629140', 'request_id': 'req-99e8ce77-3bcd-425f-814a-3f30a46f908c'} {{(pid=80244) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 08:49:36.501214 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.notifications [None req-99e8ce77-3bcd-425f-814a-3f30a46f908c admin admin] The token cache is being invalidate because project 7858221a62ef4997979bd2c376629140 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80244) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 21 08:49:36.501525 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.notifications [None req-99e8ce77-3bcd-425f-814a-3f30a46f908c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80244) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 08:49:36.502176 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.notifications [None req-99e8ce77-3bcd-425f-814a-3f30a46f908c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80244) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 08:49:36.503751 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 1448/2904] 10.222.0.47 () {58 vars in 1232 bytes} [Thu Jul 21 08:49:36 2022] DELETE /identity/v3/projects/7858221a62ef4997979bd2c376629140 => generated 0 bytes in 160 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 08:49:36.507818 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11041720-7346-43ad-8058-f9f7fd0c5ce5 admin admin] Authenticating user token {{(pid=80245) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:49:36.509237 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.fernet_utils [None req-11041720-7346-43ad-8058-f9f7fd0c5ce5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80245) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 08:49:36.522887 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11041720-7346-43ad-8058-f9f7fd0c5ce5 admin admin] Validating token access rules against request {{(pid=80245) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:49:36.582970 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5d37e1b-ef2c-4f0c-9566-22bef9e26e02 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '42d15fc0515945acacbbffcf2356c7a5', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80245) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:49:36.583872 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5d37e1b-ef2c-4f0c-9566-22bef9e26e02 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:49:36.584090 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5d37e1b-ef2c-4f0c-9566-22bef9e26e02 admin admin] SCRIPT_NAME: `/identity` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:49:36.584366 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5d37e1b-ef2c-4f0c-9566-22bef9e26e02 admin admin] PATH_INFO: `/v3/users/9d86bc543db44944973fca1e7c3f4556` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:49:36.585635 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5d37e1b-ef2c-4f0c-9566-22bef9e26e02 admin admin] RBAC: Authorizing `identity:delete_user(user_id=9d86bc543db44944973fca1e7c3f4556)` {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:49:36.587577 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5d37e1b-ef2c-4f0c-9566-22bef9e26e02 admin admin] RBAC: Authorization granted {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:49:36.633313 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.notifications [None req-d5d37e1b-ef2c-4f0c-9566-22bef9e26e02 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9d86bc543db44944973fca1e7c3f4556'} {{(pid=80245) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 08:49:36.636423 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG dbcounter [-] [80245] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=80245) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:49:36.642387 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.notifications [None req-d5d37e1b-ef2c-4f0c-9566-22bef9e26e02 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9d86bc543db44944973fca1e7c3f4556'} {{(pid=80245) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 08:49:36.655155 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.notifications [None req-d5d37e1b-ef2c-4f0c-9566-22bef9e26e02 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9d86bc543db44944973fca1e7c3f4556'} {{(pid=80245) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 08:49:36.662404 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.notifications [None req-d5d37e1b-ef2c-4f0c-9566-22bef9e26e02 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9d86bc543db44944973fca1e7c3f4556'} {{(pid=80245) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 08:49:36.663092 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.notifications [None req-d5d37e1b-ef2c-4f0c-9566-22bef9e26e02 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9d86bc543db44944973fca1e7c3f4556'} {{(pid=80245) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 08:49:36.669215 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: [pid: 80245|app: 0|req: 1457/2905] 10.222.0.47 () {58 vars in 1223 bytes} [Thu Jul 21 08:49:36 2022] DELETE /identity/v3/users/9d86bc543db44944973fca1e7c3f4556 => generated 0 bytes in 162 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 08:49:36.839158 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99e8ce77-3bcd-425f-814a-3f30a46f908c admin admin] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:49:36.840645 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.fernet_utils [None req-99e8ce77-3bcd-425f-814a-3f30a46f908c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80244) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 08:49:36.855991 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99e8ce77-3bcd-425f-814a-3f30a46f908c admin admin] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:49:36.915757 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5d11aaf-7b20-42c8-98b4-9f412d82010b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dce40b8d74c41ec94cc36e6ff05d412', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '42d15fc0515945acacbbffcf2356c7a5', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:49:36.916662 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5d11aaf-7b20-42c8-98b4-9f412d82010b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:49:36.916950 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5d11aaf-7b20-42c8-98b4-9f412d82010b admin admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:49:36.917231 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5d11aaf-7b20-42c8-98b4-9f412d82010b admin admin] PATH_INFO: `/v3/projects/351502a4a7fa4c978e2e3d1ddd192f04` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:49:36.918427 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5d11aaf-7b20-42c8-98b4-9f412d82010b admin admin] RBAC: Authorizing `identity:delete_project(project_id=351502a4a7fa4c978e2e3d1ddd192f04)` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:49:36.920384 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5d11aaf-7b20-42c8-98b4-9f412d82010b admin admin] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:49:36.969629 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.notifications [None req-c5d11aaf-7b20-42c8-98b4-9f412d82010b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '351502a4a7fa4c978e2e3d1ddd192f04', 'request_id': 'req-c5d11aaf-7b20-42c8-98b4-9f412d82010b'} {{(pid=80244) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 08:49:36.976656 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.notifications [None req-c5d11aaf-7b20-42c8-98b4-9f412d82010b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '351502a4a7fa4c978e2e3d1ddd192f04', 'request_id': 'req-c5d11aaf-7b20-42c8-98b4-9f412d82010b'} {{(pid=80244) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 08:49:36.983477 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.notifications [None req-c5d11aaf-7b20-42c8-98b4-9f412d82010b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '351502a4a7fa4c978e2e3d1ddd192f04', 'request_id': 'req-c5d11aaf-7b20-42c8-98b4-9f412d82010b'} {{(pid=80244) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 08:49:36.993813 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.notifications [None req-c5d11aaf-7b20-42c8-98b4-9f412d82010b admin admin] The token cache is being invalidate because project 351502a4a7fa4c978e2e3d1ddd192f04 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80244) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 21 08:49:36.994101 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.notifications [None req-c5d11aaf-7b20-42c8-98b4-9f412d82010b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80244) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 08:49:36.995382 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG keystone.notifications [None req-c5d11aaf-7b20-42c8-98b4-9f412d82010b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80244) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 08:49:36.996935 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 1449/2906] 10.222.0.47 () {58 vars in 1232 bytes} [Thu Jul 21 08:49:36 2022] DELETE /identity/v3/projects/351502a4a7fa4c978e2e3d1ddd192f04 => generated 0 bytes in 159 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 21 08:49:46.665535 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG dbcounter [-] [80245] Writing DB stats keystone:DELETE=3,keystone:SELECT=12 {{(pid=80245) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:49:46.990725 dv-cinder-850320-1-iscsi devstack@keystone.service[80244]: DEBUG dbcounter [-] [80244] Writing DB stats keystone:SELECT=57,keystone:DELETE=9,keystone:INSERT=1 {{(pid=80244) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 08:53:38.046534 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5d37e1b-ef2c-4f0c-9566-22bef9e26e02 admin admin] Authenticating user token {{(pid=80245) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 08:53:38.047954 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.fernet_utils [None req-d5d37e1b-ef2c-4f0c-9566-22bef9e26e02 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80245) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 08:53:38.117271 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5d37e1b-ef2c-4f0c-9566-22bef9e26e02 admin admin] Validating token access rules against request {{(pid=80245) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 08:53:38.181831 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87a8a7a4-f51c-4b06-b08c-1794fb6bc8b7 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3379b32543474d0a8e897484e85c7d04', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4faef13aa50e4e36b6c265fb1398fa14', 'project_domain_id': 'default', 'roles': ['service', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80245) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 08:53:38.182481 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87a8a7a4-f51c-4b06-b08c-1794fb6bc8b7 service placement] REQUEST_METHOD: `GET` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 08:53:38.182767 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87a8a7a4-f51c-4b06-b08c-1794fb6bc8b7 service placement] SCRIPT_NAME: `/identity` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 08:53:38.183040 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87a8a7a4-f51c-4b06-b08c-1794fb6bc8b7 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80245) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 08:53:38.184851 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.fernet_utils [None req-87a8a7a4-f51c-4b06-b08c-1794fb6bc8b7 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80245) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 08:53:38.195976 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87a8a7a4-f51c-4b06-b08c-1794fb6bc8b7 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 08:53:38.197531 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87a8a7a4-f51c-4b06-b08c-1794fb6bc8b7 service placement] RBAC: Authorization granted {{(pid=80245) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 08:53:38.207390 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: [pid: 80245|app: 0|req: 1458/2907] 10.222.0.47 () {60 vars in 1365 bytes} [Thu Jul 21 08:53:38 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 161 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 08:53:48.203099 dv-cinder-850320-1-iscsi devstack@keystone.service[80245]: DEBUG dbcounter [-] [80245] Writing DB stats keystone:SELECT=58 {{(pid=80245) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}}